Hello community.
I’m using whisper-large-v2 for audio transcription.
I have the task of transcribing audio in Kazakh and Russian.
The problem is that in Kazakhstan, people often use Russian words when speaking Kazakh. I tried to fine tune a model where the dataset consisted of 50% data in Kazakh and 50% in Russian, but the result did not please me since the model, for example, recognized audio as " Kazakh language" and did not transcribe Russian words, and vice versa.
Question:
Would it be acceptable to create a tokenizer of the “ru-kk” format to combine 2 languages into one? And will it be possible to fine tune the base model using such a tokenizer?