What Models to use to categorize projects based on description?

Models
1

Hi all,
I have a database of freelance projects with a long description and a title for each (beside other variables), texts are in German language. Now, I wanna categorize these projects into categories such as “Software Development” or “Product Ownership”.
I have a training- and test-dataframe which I categorized through OpenAI’s GPT-4 API whose results are spot-on.
Now I am looking for a free model on Hugging-Face that will come closest to the quality of the GPT-4 API which I want to fine-tune through my dataframe.
Which model would you recommend? Bloom? RoBERTa? BART? Others?

Many thanks for any help and best regards!

2

I would start with distilbert-base-uncased, which is fast for both training and inference. If it gives good enough results, stop there. If not, try the others you mention.