Storage-efficient ways to store models

[fatemystery]

I am fine-tuning several binary classification BERT models, all using bert-base-uncased. I understand that I can save my trained model and reload it later. I noticed that in the stored folder, the largest file is pytorch_model.bin (478MB), which appeared in every model I saved. If this is the pre-trained model (before finetuning), can I save it only once and later using it when loading different models?