Hi everybody I ran into some issues when trying to fine-tune bart for summarization using the BartForConditionalGeneration model. The issue evolved around properly masking and ignoring the padding tokens when training. Without the following fix the loss went down but the model produced bad summarie…

Train Bart for Conditional Generation (e.g. Summarization)

lvwerra November 22, 2022, 3:31pm 11

I cannot, sorry Maybe 5-10% but I am just guessing.

1 Like