Hi Huggingfacers I have a number of questions regarding finetuning a language model: How to mask a selective portion of a given input sentence instead of masking randomly. For example, if I am using ALBERT as a model, and I am aiming to do a different kind of loss function than the standard MLM l…

How to do selective masking in Language modeling

leoapolonio August 13, 2020, 2:27pm 2

I think the answer is similar another post: Selective masking in Language modeling

Topic		Replies	Views
Selective masking in Language modeling Beginners	1	2124	August 13, 2020
Is masking still used when finetuning a BERT model? Beginners	1	1306	July 29, 2020
Masking specific token in each input sentence during Masked language modelling 🤗Transformers	0	1025	October 18, 2021
Mask modelling on specific words Beginners	1	1042	March 25, 2021
Fine-Tuning a Text2Text Model using different tokenizer 🤗Transformers	5	54	January 20, 2025