Hi everybody I ran into some issues when trying to fine-tune bart for summarization using the BartForConditionalGeneration model. The issue evolved around properly masking and ignoring the padding tokens when training. Without the following fix the loss went down but the model produced bad summarie…

Train Bart for Conditional Generation (e.g. Summarization)

lele November 22, 2023, 8:51am 15

Hello! In the code above, it seems that you did not apply padding to the source and target. What difference would it make compared to the results when padding is used?

Topic		Replies	Views
Inference/prediction ValueError using BART 🤗Transformers	0	312	April 17, 2022
Pretraining BART for conditional generation 🤗Transformers	1	1002	May 30, 2022
[HELP]Bart summarization output exactly the same as labels 🤗Transformers	3	855	August 4, 2021
Fine-Tune BART using "Fine-Tuning Custom Datasets" doc Beginners	6	9369	October 28, 2020
Question regarding training of BartForConditionalGeneration Models	1	2032	March 2, 2021

Train Bart for Conditional Generation (e.g. Summarization)

Related topics