Using custom embeddings for pre-training model for new vocabulary

blackcupcat · December 25, 2023, 9:47pm

Hi,
I have my own custom embeddings that I want to use for pre-trainings the BertMaskedForLM model for an entirely new vocabulary.

How do I pass these custom embeddings ?

Topic		Replies	Views
Fine tunning pretrained bert with new vocabulary Beginners	0	363	October 1, 2020
Pre - Train model with inputs_embeds 🤗Transformers	0	243	July 4, 2023
Custom BERT Masked LM output embedding Beginners	0	370	September 7, 2022
Pre-training BERT Models	0	248	February 10, 2023
Tutorial on Pretraining BERT Beginners	1	433	December 15, 2020