Different Inference Results of Whisper on x86 and ARM CPU

Hello everyone,

I am working on the Whisper model inference and getting different results on two different CPU architectures. On an Intel CPU, I am able to get normal results. However, on an ARM CPU, the model is unable to predict normal tokens, resulting in empty text. After checking, I found that the GELU function output is different on both CPUs.

I am using the pre-trained Whisper-tiny.en model directly, and the Python and PyTorch versions are the same on both CPUs, with all relevant libraries installed.

The ARM CPU is Cortex-A53 on Xilinx KV260. The x86 CPU is Intel Core i7.

I would like to ask what could be the possible cause of this issue and how to resolve it? Thank you.

I am facing the same issue. Did you get any way to make it work consistently across the CPUs? Thank you.