looking at whisper cookbook: https://github.com/openai/openai-cookbook/blob/main/examples/Whisper_prompting_guide.ipynb
and my couple of experiments, we can only use whisper prompting to correct names or nouns.
I tried prompts where I give whisper a context of the audio say for example:
“This is a recorded call between agent of company A and client B where they discuss subscriptions plans”.
but the results were horrible.
I then tried prompting with fictional prompt that is somewhat related to what an audio could contain like:
“Hello sir, how can I help you today”.
again horrible results.
My conclusion is that whisper is not a general llm like GPT or llama, instead it is focused on one task and hard prompts like those are not suitable for it. Could someone confirm?
That’s a bit disappointing but hey it is only 1.5b.