How to select labels for multilabel zero-shot text classification

Hi, I am using transformers pipeline for zero-shot classification on a large set of more than 1m student reviews of courses conducted in the US and the UK. Example of one review is below:
“Very nice woman, extremely helpful if you go to her office hours, but scheme is a stupid language which makes this class boring and difficult. Very tough exams. Do good on your labs and projects and you’ll be okay.”
I read that choosing proper labels for zero-shot classification, with many domain-specific words, is key. Can you suggest general rules how to create such labels, they should be long or short, single domain specific words or complex sentences. For example there could be such approaches as:
1.
candidate_labels = [“teaching skills”, “interpersonal skills”, “grading fairness”]
or
2.
candidate_labels = [“teacher or professor teaching skills”, “teacher or professor interpersonal skills”, “course grading fairness”]
3.
candidate_labels = [“teacher or professor good or bad teaching skills”, “teacher or professor good or bad interpersonal skills”, “course grading fair or unfair”]
I do not have a labelled test set to compare the accuracy, and I would like to avoid labeling a test set, as it is tedious.
Any suggestions? Maybe there are some papers that deal with this problem?

1 Like