Multi-label classification with non-binary label

Beginners
1

Hello, I’m working on a classification problem where, for each text sample in my dataset, I have to predict three labels. Two of these labels are binary, while the remaining one is not and can take up one of four different values:

text_sample {
    label_1: # binary -> A or B
    label_2: # binary -> A or B
    label_3: # non-binary -> A, B, C or D
}

I’d like to try out BERT and from the docs BertForSequenceClassification with “multi_label_classification” as problem type looks almost like what I need. The issue is the third label.

Is there any way I can rely on or customize what the Hub already provides to solve this task? If so, I’d really appreciate a nudge in the right direction.