Further downstream: fine-tuning gpt2 for code for gcode

Hi everyone. I’m looking to generate gcode (geometric code) for 3D printing. I have a dataset of 730 files (11MB) where each file generates a certain 3D print so ideally, my intention is to generate gcode for an entire file that I could 3D print (end-to-end).

The most relevant task I’ve found for this is Casual Language Modelling for code completion. This would entail mentioning a prompt, then completing the code. As for the model, I’ve also found that most tasks for this are trained on text vs code. So I’m planning to further fine-tune this model congcongwang/distilgpt2_fine_tuned_coder, that fine-tuned gpt2 for Python and Java code, with gcode.

All in all, I’m wondering if this is a good way of going about it. I’m also preferring to distilgpt vs. gpt since it’s smaller and would respond to my dataset (11MB) better. Any advice helps. Thanks!