Gradio with Whisper

🔒 Gradio
1

Can we do soemthing like this with Gradio and Parallel. I tried both microphone and browse button for mp3 files, but both are resulting in the below error.

import gradio as gr

generator1 = gr.Interface.load("models/openai/whisper-base")
generator2 = gr.Interface.load("models/openai/whisper-base")
generator3 = gr.Interface.load("models/openai/whisper-base")

demo = gr.Parallel(generator1, generator2, generator3)

demo.launch()

Error:

/usr/local/anaconda3/envs/aimlstuff/lib/python3.10/site-packages/gradio/processing_utils.py:239: UserWarning: Trying to convert audio automatically from int32 to 16-bit int format.
  warnings.warn(warning.format(data.dtype))
Traceback (most recent call last):
  File "/usr/local/anaconda3/envs/aimlstuff/lib/python3.10/site-packages/gradio/routes.py", line 393, in run_predict
    output = await app.get_blocks().process_api(
  File "/usr/local/anaconda3/envs/aimlstuff/lib/python3.10/site-packages/gradio/blocks.py", line 1069, in process_api
    result = await self.call_function(
  File "/usr/local/anaconda3/envs/aimlstuff/lib/python3.10/site-packages/gradio/blocks.py", line 876, in call_function
    prediction = await fn(*processed_input)
  File "/usr/local/anaconda3/envs/aimlstuff/lib/python3.10/site-packages/gradio/mix.py", line 42, in parallel_fn
    return_values_with_durations = await asyncio.gather(
  File "/usr/local/anaconda3/envs/aimlstuff/lib/python3.10/site-packages/gradio/blocks.py", line 878, in call_function
    prediction = await anyio.to_thread.run_sync(
  File "/usr/local/anaconda3/envs/aimlstuff/lib/python3.10/site-packages/anyio/to_thread.py", line 28, in run_sync
    return await get_asynclib().run_sync_in_worker_thread(func, *args, cancellable=cancellable,
  File "/usr/local/anaconda3/envs/aimlstuff/lib/python3.10/site-packages/anyio/_backends/_asyncio.py", line 818, in run_sync_in_worker_thread
    return await future
  File "/usr/local/anaconda3/envs/aimlstuff/lib/python3.10/site-packages/anyio/_backends/_asyncio.py", line 754, in run
    result = context.run(func, *args)
  File "/usr/local/anaconda3/envs/aimlstuff/lib/python3.10/site-packages/gradio/external.py", line 317, in query_huggingface_api
    data = pipeline["preprocess"](*params)
  File "/usr/local/anaconda3/envs/aimlstuff/lib/python3.10/site-packages/gradio/processing_utils.py", line 46, in to_binary
    return base64.b64decode(base64str.split(",")[1])
IndexError: list index out of range

Is there more documentation on this? Can any one help me?

2

So, I need to convert the data that I am giving in to an audio file. I was using the following code without Parallel.

audio = whisper.load_audio(audio)
audio = whisper.pad_or_trim(audio)

I tried loading the method as an interface too but that throws me an error related to Tensors.
Any help is appreciated.

3

Hey, what are you trying to do?
I run this code locally that will transcribe a video or audio file for demonstrations on using whisper.

import whisper
import gradio as gr

model = whisper.load_model("medium.en") # you can use base instead
def audio_to_text(file):
    result = model.transcribe(file, fp16=False, verbose=True)
    text = result["text"]
    return text

demo = gr.Interface(
    fn=audio_to_text, 
    inputs=gr.Video(),
    outputs="text")
demo.launch(show_error=True)