Register HuggingFaceModel on AWS and deploy

Beginners
1

Hello!

I have a finetuned Falcon 7b model on AWS. I need to register this model in the Sagemaker Model Registry. After the registration, I need to load and deploy this registered model. I am wondering what is the correct way to do the above steps:


from sagemaker.huggingface import HuggingFaceModel

hub = {"HF_MODEL_ID": "tiiuae/falcon-7b", "HF_TASK": "text-generation"}

model = HuggingFaceModel(
    model_data=step_train.properties.ModelArtifacts.S3ModelArtifacts,
    env=hub,
    ...
)

step_model_registration = RegisterModel(
    model=model,
    content_type=["application/json"],
    response_type=["application/json"],
    approval_status="Approved"
)

After I run the sagemaker pipeline the model is registered. So I tried to deploy:

model_inf = sagemaker.ModelPackage(
    ...
)
model_inf.deploy(
   ...
)

Unfortunately, I hit the “set trust_remote_code to true” error when I would like to inference this model. Any suggestion?