I am using mbart-large-cc25 (MBartForConditionalGeneration) model to finetune it for multi-lingual tasks.
To evaluate the model, I need the model to generate translations for specific target language each time. How can we force the bos token for mbart?
I know that we can use model.generate with forced_bos_token_id but this is way too slow compared to just forward computing withmodel(**inputs). So I tried to set the forced_token_id by model.config.forced_bos_token_id=[lang_id] once I load a model… but it seems that the model starts generation with a random token regardless.
How can we make the model do the forward computing with forced bos token, without using model.generate?