Training compiler for all NLP models?

Amazon SageMaker
2

Hey @pierreguillou,

could you share how you created your Training Job (python estiomator + hyperparameters) and the training script you use?

Iin addition to this, I am not sure if T5 is well supported yet. As mentioned in my blog

The Amazon Training Compiler works best with Encoder Type models, like BERT , RoBERTa , ALBERT , DistilBERT .

The Training Compiler currently works best with encoder-type models. In the test I ran a couple of weeks back T5 performed worse than the default training.