Hello, I have a few questions about the BertModelLMHeadModel: Is BertModelLMHeadModel used to conduct the regular language modeling (next token prediction), as it is the case for the GPT2LMHeadModel? For GPT2LMHeadModel, I can just specify labels = input_ids for convenience. I just specify t…

Questions on the `BertModelLMHeadModel`

valhalla October 5, 2020, 10:41am 7

That’s probably a mistake,

This might help.