Fine-tuning Whisper for Translation

Beginners
1

Hi @sanchit-gandhi @MariaK, I have been following the guide to fine-tune whisper in the course and also looking at this blog. Fine-Tune Whisper For Multilingual ASR with 🤗 Transformers

I wanted to know that if I want to fine-tune for translation instead of transcription, I only need to change the task in the WhisperAudioProcessor and the metrics (using BLEU instead of WER) in the compute metrics function right?

Do I need to make any other changes to modify the code for translation to English?

Best,
Raj

2

@RajSang have you figured it out…what I think is we might also need a paired speech to English text data.

3

up, I have the same question