I’m working on a research regarding GPT2 and want to test few ideas which apply for linear layers.
What’s the problem? GPT2 model consist of Conv1D layers instead of nn.Linear.
Now I know that Conv1D is said to be just like linear, but transposed, and still I’d love if someone can help me out to convert Huggingface pre-trained GPT2 model (the one with the Conv1D) to equivalent GPT2 model with nn.Linear instead.
Thanks for any coming help!