Hi, I am using the Trainer API to train a custom Transformer model on a new language. During evaluation I would like to see the bits per character (BPC) instead of the eval_loss and perplexity. I was wondering how I would go about doing this using the Trainer API?
Related topics
| Topic | Replies | Views | Activity | |
|---|---|---|---|---|
| Evaluating with quantized model using Trainer | 1 | 263 | August 10, 2024 | |
| Evaluation on Quantized model yields identical results across bit-precisions | 1 | 101 | June 20, 2024 | |
| Trainer.evaluate() with text generation | 5 | 3472 | December 31, 2021 | |
| Unpacking transformer's trainer.eval() to see every example's output, loss | 4 | 297 | April 9, 2024 | |
| The result of fine tuning will be loss =0,eval_loss=Nan. How can I start learning the right way? | 0 | 1016 | September 10, 2023 |