Hi Huggingfacers I have a number of questions regarding finetuning a language model: How to mask a selective portion of a given input sentence instead of masking randomly. For example, if I am using ALBERT as a model, and I am aiming to do a different kind of loss function than the standard MLM l…

How to do selective masking in Language modeling

leoapolonio August 13, 2020, 2:27pm 2

I think the answer is similar another post: Selective masking in Language modeling

Topic		Replies	Views
Selective masking in Language modeling Beginners	1	2177	August 13, 2020
Masking specific token in each input sentence during Masked language modelling 🤗Transformers	0	1047	October 18, 2021
Mask modelling on specific words Beginners	1	1044	March 25, 2021
Fine-tune BERT for Masked Language Modeling 🤗Transformers	3	3027	January 25, 2021
How to make a model predict on only some tokens Beginners	1	601	June 16, 2022