InternalServerException when running a model loaded on S3

Hi there,

I am trying to deploy a model loaded on S3, following the steps found mainly on this video: [Deploy a Hugging Face Transformers Model from S3 to Amazon SageMaker](

For that I have downloaded a model into a S3 bucket and use this image URI for DLC: image_uri = “

When I run the predictor.predict(data) command, I get this error:

The model I use fot these tests is this one: dccuchile/bert-base-spanish-wwm-uncased, and I could not find the way for letting the model know which action should perform.

I am pretty new with HuggingFace technology, and probably I am missing the point for fixing that.

Please, could you let me know what should I do for informing the model about what to do?

Thank you!

Hey @vicente-m47 ,

Could you please provide the whole code you executed?

If you want to deploy a model from Hugging Face – The AI community building the future. you can use the “deploy” button on each of the model pages.

This will generate a code snippet for you

From reading the code you attached you are trying to send an input for question-answering to a model (dccuchile/bert-base-spanish-wwm-uncased), which is not fine-tuned for question-answering, also you are sending an English input to a Spanish model.

A good starting point for new Hugging Facer is our course at Transformer models - Hugging Face Course.
You can find more information about deploying to sagemaker in the documentation here Deploy models to Amazon SageMaker

Hi, @philschmid,

Sorry, you are right, my question was awfully written. Here is the code I used:

!pip install sagemaker --upgrade

from sagemaker.huggingface import HuggingFaceModel
import sagemaker

role = sagemaker.get_execution_role()

# This model was supposedly trained from the dccuchile Bert model for QA.
model = "s3://xxxxxxx/hg_model.tar.gz"
image_uri = ""

huggingface_model = HuggingFaceModel(
predictor = huggingface_model.deploy(
data = {
    "inputs": {
        "question": "¿Cómo me llamo?",
        "context": "Mi nombre es Juan y vivo en Francia"


Reading your comment, I understand that the problem should be on the trained model. I would review the course you recommend and check the training I did.

Thank you!

You don’t need to provide an image_uri the HuggingFaceModel will select the right one based on pytorch_version and transformers_version.

Yes, you’re right. Thank you, @philschmid.