What is the best way to load pertained weights then continue training BERT using my server’s GPU? Should I load the model as usual then use the Trainer() ? The tutorial for continue training seems to be rare. I’m working on prajjwal1/bert-tiny (now) and bert-base-uncased (next step).

yes, you load the model from .from_pretrained(), and use trainer to train. The link below has examples and video tutorials [image] Fine-tune a pretrained model We’re on a journey to advance and democratize artificial intelligence through open source and open science. …

The best way to load pertained weights then continue training BERT?

🤗Transformers

mab May 25, 2022, 11:32am 2

yes, you load the model from .from_pretrained(), and use trainer to train. The link below has examples and video tutorials

1 Like

Topic		Replies	Views
How to continue BERT training 🤗Transformers	1	1324	March 4, 2022
Elegant way to load and save a pretrained model as part of other model? 🤗Transformers	0	841	June 9, 2022
Load Bert model weights to transformers v3 from model trained with transformers v2 🤗Transformers	2	297	November 2, 2020
Load pretrained model's tokenizer with or without vocabulary? Beginners	2	114	August 30, 2024
Further Pretrain Basic BERT for sequence classification 🤗Transformers	4	1777	October 9, 2020

The best way to load pertained weights then continue training BERT?

Related topics