I finetuned bart-base on a sequence to sequence task and I have the following questions:
a) Currently I structured the input and output for the bart model in “t5-style” by adding prefixes in front of each piece of input. For bart how should I give multiple inputs (or train it to return multiple outputs) to the model (is there special token to separate inputs, should I continue the t5-style prefixes, etc.)? Also, how would I do this for gpt-2/gpt-neo?
b) When finetuned with prefixes, the target data is formatted with “output: …”, however, the finetuned-bart returns “outputoutput: …”. Why is this repetiton occurring? Also, does the Bart tokenizer automatically add the eos token?
c) Also does the trainer api automaticallly handle adjust_logits_during_generation and decoder_start_token_id as discussed in this post?
@valhalla Could you please help with this? This is my first project training an nlp model, and I would really appreciate any information you can offer regarding my questions.