Use BertLMHeadModel to finetunning a language model

davidhu March 30, 2021, 7:38am 1

Is it possible to use BertLMHeadModel to finetune a language model if i set the is_decoder=True ?

Topic		Replies	Views
Questions on the `BertModelLMHeadModel` 🤗Transformers	7	6249	October 5, 2020
BERT2RND EncoderDecoderModel predicts random words for Translation tasks 🤗Transformers	0	380	May 30, 2022
Fine-tuning lm with nsp 🤗Transformers	0	1166	January 19, 2021
BERT2BERT Notebook for Models without GenerationMixin 🤗Transformers	0	286	November 12, 2020
Encoder-Decoder model only generates bos_token's [<s><s><s>] Models	17	3146	December 6, 2022