Why does the falcon QLoRA tutorial code use eos_token as pad_token?

[brando]

@Rocketknight1

to the token you do want, or set the label to -100 to mask the label at that position.

why -100? what does this achieve?

The value -100 is a special token ID used by HuggingFace’s Transformers library to indicate that a particular token should be ignored when computing the loss.

why not mask = 0 for the indices you want to not train on?