Use BertLMHeadModel to finetunning a language model

davidhu March 30, 2021, 7:38am 1

Is it possible to use BertLMHeadModel to finetune a language model if i set the is_decoder=True ?

Topic		Replies	Views
Questions on the `BertModelLMHeadModel` 🤗Transformers	7	6219	October 5, 2020
Usind a fine-tuned sentence completion model in a Masked LM task 🤗Transformers	2	379	September 23, 2021
Fine-tuning Decoder-only or Encoder-Decoder models for classification 🤗Transformers	0	675	July 17, 2024
Using BERT and RoBERTa for (causal?) language modeling 🤗Transformers	6	5331	October 2, 2021
BertForMaskedLM model require fine-tuning? Beginners	0	644	August 7, 2022