Can text-to-image models be deployed to a SageMaker endpoint?

Amazon SageMaker
1

I have had trouble finding any resources pertaining to deploying text-to-image models on SageMaker. I’m working with DALLE-Mega and I am able to run it in a notebook on SageMaker but when it comes to deploying it with the SageMaker Hugging Face Inference Toolkit I am having trouble.

This is from my SageMaker notebook to deploy the endpoint:

from sagemaker.huggingface import HuggingFaceModel
import sagemaker

sess = sagemaker.Session()
role = sagemaker.get_execution_role()

# create Hugging Face Model Class
huggingface_model = HuggingFaceModel(
    transformers_version='4.6',
    pytorch_version='1.7',
    py_version='py36',
    model_data='s3://example/dalle-mega',
    role=role,
    env={ 'HF_TASK':'text-to-image' },
)
# deploy model to SageMaker Inference
predictor = huggingface_model.deploy(initial_instance_count=1,instance_type="ml.g4dn.xlarge")

Because text-to-image is not a valid HF_TASK I get this error:

ModelError: An error occurred (ModelError) when calling the InvokeEndpoint operation: Received client error (400) from primary with message "{
  "code": 400,
  "type": "InternalServerException",
  "message": "\"Unknown task text-to-image, available tasks are [\u0027feature-extraction\u0027, \u0027text-classification\u0027, \u0027token-classification\u0027, \u0027question-answering\u0027, \u0027table-question-answering\u0027, \u0027fill-mask\u0027, \u0027summarization\u0027, \u0027translation\u0027, \u0027text2text-generation\u0027, \u0027text-generation\u0027, \u0027zero-shot-classification\u0027, \u0027conversational\u0027, \u0027image-classification\u0027, \u0027translation_XX_to_YY\u0027]\""
}

Removing the “env={ ‘HF_TASK’:‘text-to-image’ },” line will allow the endpoint to be created without error, but whenever I attempt to call the endpoint it generates an error about not knowing what kind of task it is.

All that to say, the possible tasks listed in the error clearly do not include “text-to-image” so is it even possible to deploy a model like DALLE mini/mega to SageMaker?

2

Hey @ColinPM,

The Inference toolkit supports 0 code deployments for transformers pipelines currently there is no text-to-image pipeline support.
So to get your model working you need to create a inference.py similar to the one here: notebooks/sagemaker-notebook.ipynb at main · huggingface/notebooks · GitHub

1 Like