Is it possible to use spot instances for batch transform? Don’t see in either of these places
https://sagemaker.readthedocs.io/en/stable/frameworks/huggingface/sagemaker.huggingface.html
Trying to pass “use_spot_instances=True” to either the HuggingFaceModel, HuggingFaceModel.transformer or HuggingFaceModel.transformer.transform() gives an error. I am using this notebook as an example.
{
"cells": [
{
"cell_type": "markdown",
"source": [
"# Huggingface Sagemaker-sdk - Run a batch transform inference job with 🤗 Transformers\n"
],
"metadata": {}
},
{
"cell_type": "markdown",
"source": [
"1. [Introduction](#Introduction) \n",
"2. [Run Batch Transform after training a model](#Run-Batch-Transform-after-training-a-model) \n",
"3. [Run Batch Transform Inference Job with a fine-tuned model using `jsonl`](#Run-Batch-Transform-Inference-Job-with-a-fine-tuned-model-using-jsonl) \n",
"\n",
"Welcome to this getting started guide, we will use the new Hugging Face Inference DLCs and Amazon SageMaker Python SDK to deploy two transformer model for inference. \n",
"In the first example we deploy a trained Hugging Face Transformer model on to SageMaker for inference.\n",
"In the second example we directly deploy one of the 10 000+ Hugging Face Transformers from the [Hub](https://huggingface.co/models) to Amazon SageMaker for Inference.<"
],
show original
THanks
not possible today. If you want to use Spot for inference, you can write inference code as a custom python script and run inference in the Training API / HuggingFace Estimator
2 Likes
