Why is the "Use in Transformers" button displaying wrong thing?

I am looking at: mosaicml/mpt-1b-redpajama-200b · Hugging Face

If I click the “Use in Transformers” button it shows:

from transformers import MosaicGPT

model = MosaicGPT.from_pretrained("mosaicml/mpt-1b-redpajama-200b")

This fails if I copy+paste it because MosaicGPT is not part of the transformers library.

If I look at the model card they give this example instead:

import transformers
model = transformers.AutoModelForCausalLM.from_pretrained('mosaicml/mpt-1b-redpajama-200b', trust_remote_code=True)

This works, i.e. AutoModelForCausalLM knows how to use the custom code in the model repo.

If I look at the config.json for this model I can see:

  "architectures": [
  "auto_map": {
    "AutoConfig": "configuration_mosaic_gpt.MosaicGPTConfig",
    "AutoModelForCausalLM": "mosaic_gpt.MosaicGPT"

So it seems like the “Use in Transformers” button is maybe just taking the architectures value and not making use of the auto_map?

But then if I look at distilbert-base-uncased-finetuned-sst-2-english · Hugging Face I see different again. The “Use in Transformers” button gives:

from transformers import AutoTokenizer, AutoModelForSequenceClassification

tokenizer = AutoTokenizer.from_pretrained("distilbert-base-uncased-finetuned-sst-2-english")

model = AutoModelForSequenceClassification.from_pretrained("distilbert-base-uncased-finetuned-sst-2-english")

…providing both a tokenizer and the model, and both using Auto* classes.

The config.json for the model has only:

  "architectures": [

and no auto_map.

So there’s something more going on :thinking:

So… I found these docs: Integrate your library with the Hub

It seems like there is no way to configure a correct model sample code for that button just from within your repo config. Instead you have to make a Pull Request to the HF hub-docs repo and add it to this central file hub-docs/Libraries.ts at main · huggingface/hub-docs · GitHub

Seems a bit clunky (probably why few models have done that) but I guess that makes sense for any models which are unable to make use of the AutoModelFor* machinery.

But for ones that do it seems like the “Use in Transformers” button could make more of an effort to display a working code sample automatically.

And wouldn’t it be nicer if there was also a standard config entry for providing/overriding this snippet?

This is a bug in the widget, which looks in the architecture, then at some metadata we have for Transforemrs to convert it to an auto class (that’s why you see an auto-class for DistilBert). Opening an issue internally to get this fixed, thanks for flagging!

1 Like

Related… is there a way to introspect from code which models will need to use the from_pretrained(trust_remote_code=True) option when using AutoModelFrom* ?

(I mean other than just trying to load it without the flag and get an error and then trying again with it)

You can inspect their config ahead of loading and look for the auto_map key.

So presence of the auto_map key means it is a model that uses custom code?

Yes, exactly!

1 Like