How do I fine-tune a zero-shot learning model to my task?

How would you improve the performance of zero-shot models considering you will be obtaining 1) feedback on the predictions made by the model and 2) labeled examples.

I want to do classification in Spanish and with very specialized legal text. The facebook/bart-large-mnli and joeddav/xlm-roberta-large-xnli models seem to work decently. However, I already have a couple of thousand examples labeled that might help to fine-tune the zero-shot learning models. Furthermore, I will be providing feedback on the predictions made by the model.

Thank you and greetings from Mexico!


Hey, Any luck on zero-shot finetuning?

I would also love to hear about ways to fine tune.