I’m trying to understand the name of the task I’m trying to do. It seems that there are lots of models out there that are optimized for specific tasks, so I want to be sure I’m getting the right task/pre-trained model before I try to fine-tune.
My task is to take in a paragraph of text (say about 400 tokens) and output a feedback sentence on the text. This isn’t a summary, more like an “I noticed X, but it should be Y” sort of thing. For example, in a blue-sky world, perhaps my first paragraph in this post would get the feedback “If you don’t understand the task yet, I think you need to do a lot more learning” or something like that.
I can create a large-ish dataset for fine tuning, probably about 1000-2000 examples, although anything much larger than that is probably out of the question given my resources.
So I’m just wondering if there is an obvious task/set of pre-trained models that I might get started with, and if my fine-tuning dataset size sounds reasonable?