Infer on sagemaker with custom pipeline

Amazon SageMaker
1

I have a model that I want to use a custom pipeline for, how would I modify the following to use my custom pipeline code?

from sagemaker.huggingface.model import HuggingFaceModel
hub = {
  'HF_TASK':'my-custom-pipeline',
}
huggingface_model = HuggingFaceModel(
   model_data="s3://model/model.tar.gz",  # path to your trained SageMaker model
   env=hub,
   role=role,                                            # IAM role with permissions to create an endpoint
   transformers_version="4.28",                           # Transformers version used
   pytorch_version="2.0",                                # PyTorch version used
   py_version='py310',                                    # Python version used
)
2

see: Creating document embeddings with Hugging Face's Transformers & Amazon SageMaker

3

Seems like if I don’t pass in an env, it wouldn’t work, even if I have my own custom inference.py

It’s forcing me to choose from one of the pre-existing pipelines but I have custom code for inference. Am I forced to provide a parameter here?