Hi everyone, I’m fine-tuning XLNet for generation. For training, I’ve edited the permutation_mask to predict the target sequence one word at a time. I’m evaluating my trained model and am trying to decide between trainer.evaluate() and model.generate(). Running the same input/model with both meth…

Trainer.evaluate() with text generation

valhalla August 11, 2020, 2:28pm 3

There’s a PR for that, you can try to use it.

1 Like

Topic		Replies	Views
Evaluate model at saved checkpoint 🤗Transformers	0	1303	June 22, 2021
[Urgent] trainer.predict() and model.generate creates totally different predictions 🤗Transformers	4	6960	February 1, 2021
Using Trainer class with T5 - what is returned in EvalPrediction dict? 🤗Transformers	8	5349	February 14, 2022
Seq2seq evaluation speed is slow 🤗Transformers	7	3863	June 20, 2023
Evaluation results (metric) during training is different from the evaluation results at the end 🤗Transformers	4	3290	September 26, 2022