Convert Conv1D to nn.Linear

Hi everyone,

I’m working on a research regarding GPT2 and want to test few ideas which apply for linear layers.

What’s the problem? GPT2 model consist of Conv1D layers instead of nn.Linear.

Now I know that Conv1D is said to be just like linear, but transposed, and still I’d love if someone can help me out to convert Huggingface pre-trained GPT2 model (the one with the Conv1D) to equivalent GPT2 model with nn.Linear instead.

Thanks for any coming help!

#transformers #models #research

Hi @IdoAmit198,

Did you figure out a way? Because I also want to apply some function I’ve written on this model but the function currently works only on Linear classes!

Any help is appreciated