Deploying Huggingface Sagemaker Models with Elastic Inference

[philschmid]

Hello @YannAgora,

Yes, it can also be used for T5 or pegasus. You can find more documentation here: Transformers MarianMT Tutorial — AWS Neuron documentation.
You can use the NeuronGeneration code inside the inference.py then.