Wav2vec2.0 quantizer and project_q weights missing

🤗Transformers
1

Hello everyone,

I am currently studying the wav2vec2.0 model in the mode ForPretraining. It gives access to the contrastive loss which require the quantizer module.
Nonetheless the quantizer module weights or codevectors are not present in the model we can download from the hub. (facebook/wav2vec2-base or facebook/wav2vec2-960h)
Note that while these weights are not needed for PreTraining they are required to observe the result of the contrastive loss at the end of a pre-training, which I intend to do.

The displayed error is:
“Some weights of Wav2Vec2ForPreTraining were not initialized from the model checkpoint at ./wave2vecDir-fineTune960h and are newly initialized: [‘quantizer.weight_proj.weight’, ‘quantizer.codevectors’, ‘quantizer.weight_proj.bias’, ‘project_q.weight’, ‘project_hid.weight’, ‘project_q.bias’, ‘project_hid.bias’]
You should probably TRAIN this model on a down-stream task to be able to use it for predictions and inference.”

Do you have any idea of a place where I could obtain such a pretrain model with these weights in the hub or elsewhere?

Thank you really much!