I am trying to fine tune a whisper model using this source: Fine-Tune Whisper For Multilingual ASR with 🤗 Transformers I want to modify the loss function used to fine tune it. For example, I would like to modify the loss function to be able to distill knowledge from another ASR model. So how do I …

How to modify loss function in a seq2seq trainer?

nielsr August 31, 2024, 8:36am 2

Hi,

You can overwrite the Seq2SeqTrainer’s compute_loss method as shown here.

Topic		Replies	Views
Whisper fine-tuning without Seq2SeqTrainer Intermediate	0	340	December 15, 2023
Custom Training Loss Function for Seq2Seq BART Beginners	1	1704	July 21, 2023
Whisper for Audio Classification 🤗Transformers	3	2491	October 9, 2024
How to use Seq2seq Trainer with my original "[MASK]" Beginners	2	699	October 22, 2020
In SpeechSeq2Seq models, is it possible to pass decoder_input_ids for each sample during the training time using huggingface Trainer? 🤗Transformers	0	23	December 12, 2024