Deploying Sentence Transformer as sagemaker endpoint

Hello,

Has anyone deployed sentence transformer into sagemaker as endpoint? The link provides task based approach Deploy Hugging Face models easily with Amazon SageMaker but this is only applicable for nlp pipeline and I cant apply for sentence transformer.

Thanks

I’m going through the exact same thing. I’ve found this. It works, and I have a serverless inference endpoint deployed.

I’d prefer to use the sentence-transformer package as I’m not familiar with how to work with AutoTokenizer.

Thanks a lot for this nice article. So we need to change to ensure this is created as serverless?

# deploy the endpoint endpoint
predictor = huggingface_model.deploy(
    initial_instance_count=1,
    instance_type="ml.g4dn.xlarge"
    )

Yes by changing the instance configuration you can deploy the model as serverless endpoint.