Hello,
Correct me if I’m wrong but for GPT2, during training mode, when padding_side='left', position_ids should be passed correctly or else the position_ids is created as if right padding is used … making it inconsistent, right ?
I’m referring to this line where the position_ids is created if it is not passed …
transformers/modeling_gpt2.py at main · huggingface/transformers (github.com)
Thank you !