Hello there!
I’m playing with UdopForConditionalGeneration and found out in the code that the ignore_index is set to -100: I’m wondering if that shouldn’t be set to the value of the tokeninzer’s pad_token_id (which is 0 for that particular model)?
Maybe I got it wrong but my understanding is that the model UdopForConditionalGeneration predicts the IDs of the tokens to decode which in that case could imply that the indices to ignore in the loss calculation are the indices of the tokens to ignore (i.e. the padding tokens)?
Thanks for your help!
Adrien