Availability of models pushed to Hub

🤗Hub
1

I pushed my first model to the hub using push_to_hub function around an hour ago. I can see it up on the Hub website at https://huggingface.co/zyl1024/bert-base-cased-finetuned-qqp. However, when I try to use it in transformer, by following the instruction on that page,

from transformers import AutoTokenizer, AutoModelForSequenceClassification  
tokenizer = AutoTokenizer.from_pretrained("zyl1024/bert-base-cased-finetuned-qqp")
model = AutoModelForSequenceClassification.from_pretrained("zyl1024/bert-base-cased-finetuned-qqp")

I get an error saying

OSError: Can't load tokenizer for 'zyl1024/bert-base-cased-finetuned-qqp'. Make sure that:
- 'zyl1024/bert-base-cased-finetuned-qqp' is a correct model identifier listed on 'https://huggingface.co/models'
- or 'zyl1024/bert-base-cased-finetuned-qqp' is the correct path to a directory containing relevant tokenizer files

Do I need do something else or just wait for a while (if so, how long?) for it to become available for download?

2

Do I need do something else or just wait for a while (if so, how long?) for it to become available for download?

No, normally it should be directly accessible.

However, I see why it doesn’t work: your model repository only contains modeling files (config.json and pytorch_model.bin), but no tokenizer files (such as vocab.txt). Hence, only loading the model will work. You can easily save the files of a tokenizer as follows:

tokenizer.save_pretrained("path_to_directory")

You can then upload those files to your repo on the hub.

1 Like
3

Ah yes. I thought the model pushing takes care of the tokenizer automatically. After I ran tokenizer.push_to_hub(), it works perfectly. Thanks very much!