I am trying to test out a model which uses the HF API on the SUPERGLUE tasks. I have not been able to figure out how to properly preprocess the data to use with my model. So far, from reading a few papers (most specifically Roberta), it is mentioned that the authors use the following approach: sentence + “because” + choice1, sentence + “because” + choice2. This part seems straightforward, however, what I’m not too sure about is writing the custom loss function to work with my Trainer() loop. I can’t really wrap my head around choosing the sentence choice that yields the highest probability and then using that for loss… similarly for the other tasks which also have sentence-pairs and choices
Has anyone come across this before and knows the answer? Much appreicated!