Hi, guys!
I was trying to pretrain a wav2vec2 from scratch. However, the loss is continuous remains high and is never reduced.
So I wonder if there is a tutorial for pretrain wav2vec2. Looking forward to your reply.
Regard,
Eilander
1 Like
@Eilander did you find any solution to this problem? If you have pre trained a model can you please share the code
Have you checked out the README here? transformers/examples/pytorch/speech-pretraining at main · huggingface/transformers · GitHub