Whisper fine-tuning and retaining timestamp decoding

Hi everyone,

I am fine-tuning whisper models on some internal data, but I want whisper to retain its abilities besides just decoding texts.

How do I fine-tune whisper yet retain its abilities of timestamp decoding and langdetect

further more I am also trying to do multilingual fine-tuning where I take multiple langauges and fine-tune them together. how should I go about doing this ?

gently pining @sanchit-gandhi