I’m fairly new to Hugging Face. I’m working on a project that involves using the Polycoder model, which is a variant of GPT-2 trained on source code. The model was trained using the gpt-neox framework, based on megatron and deepspeed. I have some weights, and I’d like to convert them for use with a Hugging Face model. I’ve poked around a bit, and I can see that there are several different GPT-based models in the library, like GPT-2, GPT-neox, GPT-J etc. I’m not quite sure which model to use, or how I can figure out how to convert the weights. Any help would be appreciated.
Actually, I also just noticed that there’s a note in the README asking about doing this exact thing… anyone able to help?