Why is the "Use in Transformers" button displaying wrong thing?

🤗Hub
1

I am looking at: mosaicml/mpt-1b-redpajama-200b · Hugging Face

If I click the “Use in Transformers” button it shows:

from transformers import MosaicGPT

model = MosaicGPT.from_pretrained("mosaicml/mpt-1b-redpajama-200b")

This fails if I copy+paste it because MosaicGPT is not part of the transformers library.

If I look at the model card they give this example instead:

import transformers
model = transformers.AutoModelForCausalLM.from_pretrained('mosaicml/mpt-1b-redpajama-200b', trust_remote_code=True)

This works, i.e. AutoModelForCausalLM knows how to use the custom code in the model repo.

If I look at the config.json for this model I can see:

{
  "architectures": [
    "MosaicGPT"
  ],
  "auto_map": {
    "AutoConfig": "configuration_mosaic_gpt.MosaicGPTConfig",
    "AutoModelForCausalLM": "mosaic_gpt.MosaicGPT"
  },
}

So it seems like the “Use in Transformers” button is maybe just taking the architectures value and not making use of the auto_map?

But then if I look at distilbert-base-uncased-finetuned-sst-2-english · Hugging Face I see different again. The “Use in Transformers” button gives:

from transformers import AutoTokenizer, AutoModelForSequenceClassification

tokenizer = AutoTokenizer.from_pretrained("distilbert-base-uncased-finetuned-sst-2-english")

model = AutoModelForSequenceClassification.from_pretrained("distilbert-base-uncased-finetuned-sst-2-english")

…providing both a tokenizer and the model, and both using Auto* classes.

The config.json for the model has only:

{
  "architectures": [
    "DistilBertForSequenceClassification"
  ],
}

and no auto_map.

So there’s something more going on :thinking:

2

So… I found these docs: Integrate your library with the Hub

It seems like there is no way to configure a correct model sample code for that button just from within your repo config. Instead you have to make a Pull Request to the HF hub-docs repo and add it to this central file hub-docs/Libraries.ts at main · huggingface/hub-docs · GitHub

Seems a bit clunky (probably why few models have done that) but I guess that makes sense for any models which are unable to make use of the AutoModelFor* machinery.

But for ones that do it seems like the “Use in Transformers” button could make more of an effort to display a working code sample automatically.

And wouldn’t it be nicer if there was also a standard config entry for providing/overriding this snippet?

3

This is a bug in the widget, which looks in the architecture, then at some metadata we have for Transforemrs to convert it to an auto class (that’s why you see an auto-class for DistilBert). Opening an issue internally to get this fixed, thanks for flagging!

1 Like
4

Related… is there a way to introspect from code which models will need to use the from_pretrained(trust_remote_code=True) option when using AutoModelFrom* ?

(I mean other than just trying to load it without the flag and get an error and then trying again with it)

5

You can inspect their config ahead of loading and look for the auto_map key.

6

So presence of the auto_map key means it is a model that uses custom code?

7

Yes, exactly!

1 Like