I can successfully pre-train patrickvonplaten/wav2vec2-base-v2 with a new language unlabeled wav corpus by following the steps outlined in https://github.com/huggingface/transformers/tree/main/examples/pytorch/speech-pretraining on a single GPU A100.
However, when I want to use it with patrickvonplaten/mms-300 or facebook/wav2vec2-xls-r-300m, it gives error at accelerator.backward(loss).
RuntimeError: handle_0 INTERNAL ASSERT FAILED at “…/c10/cuda/driver_api.cpp”:15, please report a bug to PyTorch.
Any pointer to the resolution of this would be great.