Deploying HG Pipelines on AWS Sagemaker

Amazon SageMaker
1

I need some help with deploying HG Pipelines on AWS Sagemaker

This is how I deploy a pre-trained model in a pipeline in my local environment and get a prediction.

local_model = BertForSequenceClassification.from_pretrained(“local_model_path”)
tokenizer = BertTokenizer.from_pretrained(“"bert-base-uncased", padding=True, truncation=True)
pipe = pipeline("text-classification", model= local_model, tokenizer=.tokenizer)
output = pipe(input)

I could not find a way to deploy this HG Pipeline on Sagemaker. I followed the following documentation Deploy models to Amazon SageMaker and tried to use HuggingFaceModel. However, I am not sure how I can provide the other pipeline parameters.

huggingface_model = HuggingFaceModel(
   model_data="s3://model.tar.gz",
   role=role,
   transformers_version="4.12",
   pytorch_version="1.9",
   py_version='py38'
)

Where and how can I provide the task (text-classification) and tokenizer information? If this is not possible, should I first tokenize my data and give that as an input to predictor? predictor.predict(data_tokenized)?

Any other suggestions? Thanks so much!!! :pray:

2

Hi Tom, I think you are on the right path. Provided your model is already on S3 and packaged as tar.gz file you can just use the deploy() method to deploy the model. To get predictions you can use predictor.predict, as you have already pointed out.

This notebook by @philschmid should be useful: notebooks/deploy_transformer_model_from_s3.ipynb at master · huggingface/notebooks · GitHub

Hope that helps, let me know if there are any questions.

Cheers
Heiko

3

Thanks for the note. I followed @philschmid’s notebook. However, I am still wondering where and how I should be providing the tokenizer parameter that I usually provide in a HG pipeline. Please see below:
pipe = pipeline(“text-classification”, model= local_model, tokenizer=.tokenizer)

any ideas?

4

The tokenizer goes into the model.tar.gz file, together with the model and all other required files for your model to run.

5

Hello @TomKc,

You need to create a model.tar.gz including your tokenizer and model files. Instructions can be found here: Deploy models to Amazon SageMaker or below.
{repository} must include those files.

cd {repository}
tar zcvf model.tar.gz *
aws s3 cp model.tar.gz <s3://{my-s3-path}>

and then you can use the env parameter to provide your task.

huggingface_model = HuggingFaceModel(
   model_data="s3://{my-s3-path}/model.tar.gz",
   role=role,
   transformers_version="4.12",
   pytorch_version="1.9",
   py_version='py38', 
   env = {"HF_TASK":"text-classification"}
)