Hi @marcoabrate
The T5ForConditionalGeneration model returns a tuple which contains ['logits', 'past_key_values', 'encoder_last_hidden_state'].
To be able to calculate generative metrics, we need to generate the seq during evaluation, we can’t calculate these metrics using the logits
The examples/seq2seq here supports seqseq training (summrization, translation) and also computes the appropriate metrics (ROUGE, BLUE etc).
For seq2seq training consider using Seq2SeqTrainer and fintune_trainer.py (which uses Trainer) or finetune.py (which uses pytorch-lightning).