How to fine tune fine tune GitHub Copilot?

We can fine tune language models like BERT, GPT-3.

Can I fine tune GitHub Copilot model?

I have already looked into the https://copilot.github.com/ but cant find the details. Would really appreciate if someone had fine tuned Github Copilot.

1 Like

Hi @neo-benjamin

The Codex model that’s powering the Copilot product is not open sourced. However, there are a few models similar to Codex available on the Hugging Face Hub such as Incoder or CodeGen:

1 Like

How to fine tune Codegen? Are the steps documented?

1 Like

You can have a look at the language modeling examples. The should work for any auto regressive model such as GPT-2 or CodeGen: transformers/examples/pytorch/language-modeling at main · huggingface/transformers · GitHub