Fail predict using Falcon-7B-Instruct

🤗Transformers
1

I deployed tiiuae/falcon-7b-instruct in AWS SageMaker , when I will use a prediction it fails with the following error

(ModelError) when calling the InvokeEndpoint operation: Received client error (400) from primary with message "{
“code”: 400,
“type”: “InternalServerException”,
“message”: “Loading /.sagemaker/mms/models/tiiuae__falcon-7b-instruct requires you to execute the configuration file in that repo on your local machine. Make sure you have read the code there to avoid malicious use, then set the option trust_remote_code\u003dTrue to remove this error.”
}

I deployed directly from the hub, should I use another deployment option ?

My deployment code:

## Hub Model configuration. https://huggingface.co/models

hub = {

'HF_MODEL_ID':'tiiuae/falcon-7b-instruct',

'HF_TASK':'text-generation'

}

# create Hugging Face Model Class

huggingface_model = HuggingFaceModel(

transformers_version='4.26.0',

pytorch_version='1.13.1',

py_version='py39',

env=hub,

role=role,

)

# deploy model to SageMaker Inference

predictor = huggingface_model.deploy(

initial_instance_count=1, # number of instances

instance_type='ml.m5.xlarge' # ec2 instance type

)
2 Likes