I understand the -100 label id is used so that the predictions for these are not included when calculating the loss. However here , they state “complicated list comprehension here because pad_token_id alone is not good enough to know whether label should be excluded or not”, when replacing pad token…

Pad token vs -100 index_id

jgetner April 1, 2025, 7:10pm 2

Its just for when someone wants to change the pad token id.

2 Likes

Topic		Replies	Views
UdopForConditionalGeneration ignore_index in loss calculation 🤗Transformers	0	100	March 28, 2024
Will Trainer loss functions automatically ignore -100? 🤗Transformers	2	2068	June 29, 2023
Difference between setting label index to -100 & setting attention mask to 0 🤗Transformers	5	2921	March 17, 2021
Seq2seq padding 🤗Transformers	1	64	October 10, 2024
Confusion over use of -100 pad value for GPT2 Causal Modeling Fine-tuning Models	0	279	April 13, 2023