I’m trying to replicate part of the transformers tutorial from fastai, and there is a place one writes:
from transformers import GPT2LMHeadModel, GPT2TokenizerFast
pretrained_weights = 'gpt2'
tokenizer = GPT2TokenizerFast.from_pretrained(pretrained_weights)
model = GPT2LMHeadModel.from_pretrained(pretrained_weights)
However, trying to run it I get
---------------------------------------------------------------------------
OSError Traceback (most recent call last)
<ipython-input-31-b475580d46e5> in <module>
1 from transformers import GPT2LMHeadModel, GPT2TokenizerFast
2 pretrained_weights = 'gpt2'
----> 3 tokenizer = GPT2TokenizerFast.from_pretrained(pretrained_weights)
4 model = GPT2LMHeadModel.from_pretrained(pretrained_weights)
/opt/conda/lib/python3.7/site-packages/transformers/tokenization_utils_base.py in from_pretrained(cls, pretrained_model_name_or_path, *init_inputs, **kwargs)
1589 ", ".join(s3_models),
1590 pretrained_model_name_or_path,
-> 1591 list(cls.vocab_files_names.values()),
1592 )
1593 )
OSError: Model name 'gpt2' was not found in tokenizers model name list (gpt2, gpt2-medium, gpt2-large, gpt2-xl, distilgpt2). We assumed 'gpt2' was a path, a model identifier, or url to a directory containing vocabulary files named ['vocab.json', 'merges.txt', 'tokenizer.json'] but couldn't find such vocabulary files at this path or url.
I find this confusing because gpt2 is in the list. In fact, I encounter the same problem with any transformer model I choose, like for instance distilgpt2 or from another family. In fact, if I comment out that line I also get an error
---------------------------------------------------------------------------
OSError Traceback (most recent call last)
/opt/conda/lib/python3.7/site-packages/transformers/configuration_utils.py in get_config_dict(cls, pretrained_model_name_or_path, **kwargs)
372 if resolved_config_file is None:
--> 373 raise EnvironmentError
374 config_dict = cls._dict_from_json_file(resolved_config_file)
OSError:
During handling of the above exception, another exception occurred:
OSError Traceback (most recent call last)
<ipython-input-32-a4869c5495d6> in <module>
2 pretrained_weights = 'gpt2'
3 #tokenizer = GPT2TokenizerFast.from_pretrained(pretrained_weights)
----> 4 model = GPT2LMHeadModel.from_pretrained(pretrained_weights)
/opt/conda/lib/python3.7/site-packages/transformers/modeling_utils.py in from_pretrained(cls, pretrained_model_name_or_path, *model_args, **kwargs)
874 proxies=proxies,
875 local_files_only=local_files_only,
--> 876 **kwargs,
877 )
878 else:
/opt/conda/lib/python3.7/site-packages/transformers/configuration_utils.py in from_pretrained(cls, pretrained_model_name_or_path, **kwargs)
327
328 """
--> 329 config_dict, kwargs = cls.get_config_dict(pretrained_model_name_or_path, **kwargs)
330 return cls.from_dict(config_dict, **kwargs)
331
/opt/conda/lib/python3.7/site-packages/transformers/configuration_utils.py in get_config_dict(cls, pretrained_model_name_or_path, **kwargs)
380 f"- or '{pretrained_model_name_or_path}' is the correct path to a directory containing a {CONFIG_NAME} file\n\n"
381 )
--> 382 raise EnvironmentError(msg)
383
384 except json.JSONDecodeError:
OSError: Can't load config for 'gpt2'. Make sure that:
- 'gpt2' is a correct model identifier listed on 'https://huggingface.co/models'
- or 'gpt2' is the correct path to a directory containing a config.json file
Everything is run on Kaggle notebooks, in case it’s important
Thanks in advance!
?