Saving a model and loading it

Models
1

So I am using this to get the model
model = T5ForConditionalGeneration.from_pretrained(“t5-small”)

then I try to save the model using
model.save_pretrained(“/home/myname/Desktop/”, from_pt=True)

And I get the these files: config.json, pytorch_model.bin

next i try to load the model using
model = T5ForConditionalGeneration.from_pretrained(“config.json”)

and i get the error:
OSError: Unable to load weights from pytorch checkpoint file for ‘config.json’ at ‘config.json’. If you tried to load a PyTorch model from a TF 2.0 checkpoint, please set from_tf=True.

What am I doing wrong? Am i supposed to do something with the pytorch_model.bin file ?

1 Like
2

Hi Samuel, you need to pass the folder with all the exported files to load the model. For example:

from transformers import T5ForConditionalGeneration

model = T5ForConditionalGeneration.from_pretrained("t5-small")

model.save_pretrained("/home/my_name/Desktop/t5small", from_pt=True)

And then, you can load the model:

model = T5ForConditionalGeneration.from_pretrained("/home/my_name/Desktop/t5small")
4 Likes
3

Thank you so much!! This caused me so much frustration! haha

1 Like
5

Hello guys. I’m new to NLP and I just have trained llama3 on Sentiment Classification and I want to save it. I already used the:
trainer.save_model(“saved_model”)
method. But it only saves the configuration files and I need to re-upload it every time I want to use it:

tokenizer = AutoTokenizer.from_pretrained(checkpoint_path)
model = LlamaForSequenceClassification.from_pretrained(checkpoint_path, num_labels=4)
model.eval()

Which honestly makes me mad. Can I save the model with full weights (similarly, when I download the model using ollama)? Dose Hugging Face support this type of download?

1 Like