https://discuss.huggingface.co/t/new-pipeline-for-zero-shot-text-classification/681/14#:~:text=Thanks%20for%20the,add%20a%20bit%3A The link above describes the finetuning of zero-shot models. The difference between finetuning of sequences and NLI is that the former uses a custom number of labels and only the sequence to classify and the latter uses three labels in the specific order and the sequence to classify followed by the putative entailment as specified in the link above. That link also suggests to use zero-shot classification only if you don’t have enough labeled data.