Since token_type_ids are there to tell the model which is the first and which the 2nd sequence - I think I can just remove them for BertForSequenceClassification. Is that right?
You do not need to pass them to the model in that case, no.
Ok. Thanks!