I want to know if i should use Transformers or my own custom model from scratch to do a Text Classification task, the task is to classify a given text input(programming code) into what programming language it is, as text. it will support 31 programming languages, this is really a beginner project for me to learn more about ML. I don’t necessarily want to take the easy route though, i want it to be high performing on benchmarks, and also fast to run. I am using PyTorch btw.
I am currently making my own custom dataset from Github data.
Let me know some ideas on how to approach this.
Here is the repository: