Hi guys! I note that the most topics are related to fine-tuning a pre-trained model. But if I have got some new unlabeled data, how can I preform the pre-training process via Huggingface?

Pre-training for Wav2Vec2-XLSR via Huggingface

morenolq June 9, 2023, 1:05pm 15

I also switched to fairseq, there are several LR and threshold management things missing in HF I think.

Topic		Replies	Views
Wav2Vec2 Speech Pre-Training After a few epochs the contrastive loss was decreased to zero and the model stopped changing 🤗Transformers	2	757	November 21, 2023
Is there any tutorial 4 pretrain wav2vec2 Beginners	4	1096	November 6, 2024
Why is Wav2Vec pretraining loss not decreasing? Models	10	2696	April 29, 2022
Pre-training loss Wav2Vec2 Conformer Models	0	323	March 13, 2023
Wav2VecForPreTraining - Not able to run trainer.train() Beginners	3	705	October 19, 2021