How to load weights which was trained older version of PyTorch?

Beginners
1

Hi,

I trained a RoBERTa model on PyTorch 1.6 and tried to load the weights on different server which had PyTorch 1.4.

But I could not load the weights and got an error which said,

OSError: Unable to load weights from pytorch checkpoint file. If you tried to load a PyTorch model from a TF 2.0 checkpoint, please set from_tf=True.

I can upgrade PyTorch which is not a problem but I’m curious how can I load weights trained on older version of PyTorch?

Environments of both servers:
ubuntu: 18.04 ( both servers )
transformers: 3.1.0 ( both servers )
Python 3.7.9 and Python 3.7.3

Thanks in advance.

1 Like
2

AFAIK PyTorch versions shouldn’t make a difference. Check your path, check that the file is not empty. Also post the code that you use to save and load the model.

3

Thank you for your response.

I used the example code here .
But dataset is not Esperanto.

Anyway I’ll check my codes more.
It seems something happened while reading weights file because of os or python version difference…

4

using torch.save(model.state_dict(), os.path.join(output_dir, ‘pytorch_model.bin’), _use_new_zipfile_serialization=False) instead of model.save_pretrained() or trainer.save_model()

4 Likes
5

It does with PyTorch 1.6 as now torch.save uses a new zip file-based file format (unless you pass a _use_new_zipfile_serialization flag as False). Of course, older PyTorch versions can’t load this format; I run into this issue often.

3 Likes
6

Wow, I didn’t know about this, thanks @BinhMinhs10 and @vblagoje!

1 Like
7

Thank you!!! I updated my torch1.2 to 1.8 and solved the problem!!!

1 Like
8

I encounted this error too when I used pytorch 1.4. But when I use pytorch 1.8, this error dispeared

1 Like
9

Any way to overcome this with a PyTorch version below 1.6? I’m using 1.5 on my system and can’t upgrade anytime soon! :X