How to load model with .pth and avoid ponderous pytorch_model.bin

When I trained my BERT-based model (using AutoModel.from_pretrained()) I saved .pth file.
Now, when I’m going to use it in a remote container I would like to load as less files as possible (to keep cointaner light).
What can I do to avoid using pytorch_model.bin in "cache_dir " (that is as far as I know weights for base BERT model and it is ~1.6gb) since I’m doing "model.load_state_dict " with my .pth anyway
ps: I cannot just use my .pth file instead of pytorch_model.bin since .pth also has some “classification head” layers.