While finetuning w2v-bert, the WER is not decreasing

Beginners
1

While finetuning w2v-bert, the WER is not decreasing. I’ve fine-tuned wav2vec2, xls-r with no problem but w2v-bert has some issune while fine tuning, can anyone help?

Step Training Loss Validation Loss Wer
300 4.636700 inf 0.993919
600 18.166900 nan 1.000000
900 0.000000 nan 1.000000
1200 0.000000 nan 1.000000

Below is my collab notebook.

1 Like
2

I’m not familiar with voice model training, but something doesn’t seem right. Take a look at the following page. Maybe it means there are some environments that work well.

I’ve been modifying some Spaces recently, and it occurred to me that all of the voice-related libraries are very old. I think it is possible that some of the libraries that are not explicitly specified in the pip may be doing something wrong.

In the context of generative AI, even functions that are 6 months old may be obsolete. Even if there is no syntax error, the content may have changed, which is tricky.

1 Like
3

Can you manually try printing the loss and gradient norm ? With debugger maybe ?

See if the train_loss/gradient norm is becoming infinite. In that case, your program might be suffering from gradient explosion problem.

1 Like