I am trying to deploy a Vision Transformer huggingface model to a sagemaker endpoint. I am trying to make an inference.py script to go along with the model artificats, so that I can have my own custom output from predict_fn.
However, the endpoint keeps returning this error when passing the image through the feature_extractor:
ModelError: An error occurred (ModelError) when calling the InvokeEndpoint operation: Received client error (400) from model with message "{
“code”: 400,
“type”: “InternalServerException”,
“message”: “an integer is required (got type str)”
I’ve checked and confirmed that the image is indeed a PIL image (which works when I pass it through the endpoint regularly, without the inference script, just without returning the desired output format). I have even tried to convert the image to a numpy array, but I still get the same error.
This is the script:
def model_fn(model_dir):
logging.info(‘within model_fn’)
model = ViTForImageClassification.from_pretrained(model_dir)
feature_extractor = ViTFeatureExtractor.from_pretrained(model_dir)
logging.info(‘Extracted Model and Extractor’)
return model, feature_extractor
def predict_fn(data, model_extractor):
# destruct model and tokenizer
logging.info(‘Inside Prediction_fn’)
model, feature_extractor = model_extractor
inputs = feature_extractor(images=data['inputs'], return_tensors="pt")
outputs = model(**inputs)
logits = outputs.logits
# return {'outputs': outputs, 'logits': logits}
return {'logits': logits}
Thanks in advance!