I downloaded the same albert-base-v2-imdb model from huggingface. The truncation parameter worked.
However with the same deployment and prediction code, the truncation parameter didn’t with my model. No, I don’t have any custom pipeline code in the model.tar.gz file. I retrained an albert_xx_large, then fine-tuned this model.
Still the same error -
ModelError: An error occurred (ModelError) when calling the InvokeEndpoint operation: Received client error (400) from model with message "{
“code”: 400,
“type”: “InternalServerException”,
“message”: “The size of tensor a (577) must match the size of tensor b (512) at non-singleton dimension 1”
}
". See https://us-east-1.console.aws.amazon.com/cloudwatch/home?region=us-east-1#logEventViewer:group=/aws/sagemaker/Endpoints/huggingface-pytorch-inference-2021-10-06-00-23-07-036 in account 209338229909 for more information.