I am trying to fine tune a whisper model using this source: Fine-Tune Whisper For Multilingual ASR with 🤗 Transformers I want to modify the loss function used to fine tune it. For example, I would like to modify the loss function to be able to distill knowledge from another ASR model. So how do I …

How to modify loss function in a seq2seq trainer?

nielsr August 31, 2024, 8:36am 2

Hi,

You can overwrite the Seq2SeqTrainer’s compute_loss method as shown here.