'GPT2LMHeadModel' object has no attribute 'push_to_hub'

No that’s incorrect.

Using the push_to_hub method is preferred when you just want to push to the Hub, and your model should have that method. Are you absolutely certain the Transformers version is correct inside your notebook? All PreTrainedModel have the push_to_hub method.