Could someone give some insight to the “model.compute_loss” function which is used when fine-tuning the models without the trainer API (e.g- Keras native training). How does it work (i.e.- for different tasks how it’s adapted.)? It says that it takes the first element of the output, how is the loss calculated then?
as shown in here in Huggingface documentation, Fine-tuning with custom datasets — transformers 4.7.0 documentation