Unrecognized model in healx/gpt-2-pubmed-medium

Models
1

Hi, I would like to use and fine-tune the healx/gpt-2-pubmed-medium model, but if I try to load it with the provided snippet of code, or even if I directly try to fine tune it with run_clm.py it gives me the following error:

ValueError: Unrecognized model in healx/gpt-2-pubmed-medium. Should have a `model_type` key in its config.json, or contain one of the following strings in its name: bigbird_pegasus, deit, luke, gpt_neo, big_bird, speech_to_text, vit, wav2vec2, m2m_100, convbert, led, blenderbot-small, retribert, ibert, mt5, t5, mobilebert, distilbert, albert, bert-generation, camembert, xlm-roberta, pegasus, marian, mbart, megatron_bert, mpnet, bart, blenderbot, reformer, longformer, roberta, deberta-v2, deberta, flaubert, fsmt, squeezebert, bert, openai-gpt, gpt2, transfo-xl, xlnet, xlm-prophetnet, prophetnet, xlm, ctrl, electra, encoder-decoder, funnel, lxmert, dpr, layoutlm, rag, tapas

I may try to manually download the model, modify the config.json file adding model_type: 'gpt2', but I am not sure if this would be enough.
Anyway, this seems a problem that should not happen

2

Hi, did you solve this problem? I am having the same problem now. Thanks!

3

Hi, the solution I found was to manually clone the github repo and then add in the config.json file "model_type": "gpt2"

4

Great! Thanks a lot, Vitto! :grinning:

5

An easier solution is to just use

from transformers import GPT2Tokenizer, GPT2Model

instead of AutoTokenizer or AutoModel when using from_pretrained().

This will not raise an error, since it knows the model type explicitly.

1 Like