Hi Huggingfacers I have a number of questions regarding finetuning a language model: How to mask a selective portion of a given input sentence instead of masking randomly. For example, if I am using ALBERT as a model, and I am aiming to do a different kind of loss function than the standard MLM l…

How to do selective masking in Language modeling

leoapolonio August 13, 2020, 2:27pm 2

I think the answer is similar another post: Selective masking in Language modeling

Topic		Replies	Views
Is masking still used when finetuning a BERT model? Beginners	1	1131	July 29, 2020
Fine tune Masked Language Model on custom dataset Beginners	5	5399	August 20, 2020
MLM vs CLM, can be exchanged? Models	0	704	August 21, 2022
Usind a fine-tuned sentence completion model in a Masked LM task 🤗Transformers	2	262	September 23, 2021
SpanBERT, ELECTRA, MARGE from scratch? Beginners	5	1101	July 22, 2023