Inference Hyperparameters

Okay, then we know it is not a SageMaker specific issue.

What happens if you load your model and tokenizer with from_pretrained in a notebook and try to use them with the pipelines and truncation=True?

And just to be sure can you share the request you sent with the predictor to your model here?