"Can't load tokenizer for <model>" after download

I’m trying to use the cardiffnlp/twitter-roberta-base-hate model on some data, and was following the example on the model’s page. The script works the first time, when it’s downloading the model and running it straight away, but after I run it after this I get the message:

Can't load tokenizer for 'cardiffnlp/twitter-roberta-base-hate'. If you were trying to load it from 'https://huggingface.co/models', make sure you don't have a local directory with the same name. Otherwise, make sure 'cardiffnlp/twitter-roberta-base-hate' is the correct path to a directory containing all relevant files for a RobertaTokenizerFast tokenizer.

It’s downloaded the model into a subdirectory of my working directory, which it’s presumably finding. That directory contains two files: config.json and pytorch_model.bin. If I delete that directory it works again for one run, but as it’s a half-gig model I’d rather not have to do that each time!

Hello, have you solved this problem?

Hi, I’m sorry but I think I just used a different model. In this case the Cardiff Hatelab had a more up to date model anyway which was probably better maintained.

Good luck

Hey,The community faced the same issue at Venmo Nerd but I tried solving the issue via help of my friends.