I want to finetune a Bart model with an AutoModelForSeq2SeqLM head.
Where do I find the format to preprocess the dataset into?
I think the answer may be the arguments in the forward method of the tokenizer?
So in my case, I would need to make sure that my dataset has the following columns:
input_ids, attention_mask, decoder_input_ids, decoder_attention_mask
?
Would this potentially change depending on what head I use for finetuning, and if so where would I turn to see those requirements?