How to evaluate bert model from MLM task result?

JackieShi · October 4, 2023, 1:16pm

Now, I have to train BERT from scratch using my own domain data. And in my situation, I don’t have to do NSP, instead only MLM task is enough.
I’ve trained many times and the loss is always around 2 ~ 4. But I don’t know the “good” value for this task. So I wonder that is there any way to evaluate or is there any standard value which can be a reference?

Topic		Replies	Views
How to train BERT from scratch on a new domain for both MLM and NSP? Models	2	1941	February 6, 2021
Evaluation metrics for BERT-like LMs Research	3	4006	October 5, 2020
Pre-training BERT Models	0	249	February 10, 2023
Using MLM and NSP to fine-tune BERT for question answering Models	0	941	October 11, 2022
Getting the MLM accuracy for the BERT model I am training from scratch Beginners	7	4569	October 5, 2023

How to evaluate bert model from MLM task result?

Related Topics