I wanted to know in detail as to how xForMultipleChoiceModel
works ? So prompt and choices are concatenated beforehand (preprocess_func
). I had a look at source code here. So number of options are chosen from input_ids
, followed by flattening of all keys in encoding
object from tokenizer (input_ids
, position_ids
) after which it is sent to LongFormerModel
. Can you please expain what happens exactly after this step ? I see that some padding is performed after which LongFormerEncoder
is called. How does the Multiple choice head work and how is the loss computed ? Thanks.