Hi everybody I ran into some issues when trying to fine-tune bart for summarization using the BartForConditionalGeneration model. The issue evolved around properly masking and ignoring the padding tokens when training. Without the following fix the loss went down but the model produced bad summarie…

Train Bart for Conditional Generation (e.g. Summarization)

Zhengyao December 5, 2021, 7:00am 7

You can ignore the specifying labels index when initial the loss function.

Topic		Replies	Views
Inference/prediction ValueError using BART 🤗Transformers	0	323	April 17, 2022
Pretraining BART for conditional generation 🤗Transformers	1	1029	May 30, 2022
[HELP]Bart summarization output exactly the same as labels 🤗Transformers	3	874	August 4, 2021
Fine-Tune BART using "Fine-Tuning Custom Datasets" doc Beginners	6	9426	October 28, 2020
Question regarding training of BartForConditionalGeneration Models	1	2046	March 2, 2021