Wav2vec2 model tuning on Mac Silicon

🀗Transformers
1

I am having difficulties fine tuning the wav2vec2 model on my Mac Silicon (M1) processor. When downstream training on the Commonvoice dataset, I encounter the Jupyter notebook error:

NotImplementedError: The operator ‘aten::_weight_norm_interface’ is not currently implemented for the MPS device. If you want this op to be added in priority during the prototype phase of this feature, please comment on General MPS op coverage tracking issue · Issue #77764 · pytorch/pytorch · GitHub. As a temporary fix, you can set the environment variable PYTORCH_ENABLE_MPS_FALLBACK=1 to use the CPU as a fallback for this op. WARNING: this will be slower than running natively on MPS.

The error suggests creating the `PYTORCH_ENABLE_MPS_FALLBACK=1’ environment variable which I have done in the terminal. I have restarted the kernel and confirmed the environment variable is present and set in the notebook by running %env. I have now hit a brick wall with this. Any solutions to this?

2

With some kernel rebuilding, and setting the environment variable: :`PYTORCH_ENABLE_MPS_FALLBACK=1’ in the virtual environment terminal shell, I am now able to train the model. CPU speeds look to be a limiting factor however!

3

How did you do the “kernel rebuilding”?