In GPT-2, how do I obtain the loss that is calculated for individual tokens when I run a forward pass on the input sequence? I understand the loss we obtain for the entire sequence length, but interested in obtaining token-level loss. Any help is greatly appreciated.
Related topics
| Topic | Replies | Views | Activity | |
|---|---|---|---|---|
| Loss in a Seq2Seq task | 0 | 119 | June 5, 2024 | |
| Can we get per word loss from the output of a GPT model | 0 | 361 | March 2, 2022 | |
| How to calculate GPT-2 sentence loss (for each sentence) if batch has 2 or more sentences? | 1 | 819 | April 17, 2023 | |
| Is there a way to get per word loss instead of the average loss for GPT model | 0 | 323 | March 7, 2022 | |
| How to compute per-token loss when doing language modeling? | 3 | 2240 | August 23, 2023 |