How do I do multi Class (multi head) classification?

[sgugger]

This is all in the loss function, so you can definitely use BertForSequenceClassification with two labels, then use the proper loss function (probably BCEWithLogitsLoss).