I am fine-tuning T5 for paraphrase generation and want to add a diversity measure for the generated sentences in the loss function. After reading the source code , I still have no clue how to add that.

T5 user defined loss function

mengyahu August 6, 2020, 8:27pm 8

Thanks for the reminder, @chrisdoyleIE ! This is surprising to me! Could you provide me with a solution if I want diversity_loss to have influence on my model?

Force decoder to avoid repetition between generated sentences

Topic		Replies	Views
How to output loss from model.generate()? 🤗Transformers	15	4036	April 26, 2024
T5 - model.generate() issue Beginners	2	523	March 18, 2024
TFT5ForConditionalGeneration with custom loss Beginners	0	380	April 4, 2022
How to negatively penalize a T5 model's generation (Cross entropy doesn't do the job)? 🤗Transformers	0	236	November 19, 2022
T5 weird behavior between model.forward() and model.generate 🤗Transformers	0	45	March 31, 2024

T5 user defined loss function

Related Topics