Applying movement-pruning on GPT2

🤗Transformers
1

Hi,
I would like to apply movement pruning on GPT2. This model instead of linear layers uses Conv1D layers, could I replace them with the MaskedLinear layers although they are not linear layers? I appreciate any advice on this. Specially where you return torch.nn.functional.linear after pruning, since GPT-2 uses the version of Conv1D implemented in huggingface, could you kindly tell me if I can still use your method?

Thanks in advance @wolfblue and @VictorSanh

2

I have run into this same issue. The library prints a message that doesn’t really say what to do:

You are loading your model in 8bit or 4bit but no linear modules were found in your model. this can happen for some architectures such as gpt2 that uses Conv1D instead of Linear layers. Please double check your model architecture, or submit an issue on github if you think this is a bug.

From this message it’s not clear whether gpt2-type models are unsupported, or supported via some other approach.

1 Like