Unrecognized configuration class <class 'transformers.models.mixtral.configuration_mixtral.MixtralConfig'> for this kind of AutoModel: AutoModelForSeq2SeqLM

Beginners
1

hello This is my code and I am facing this error.

import os
import pathlib
from transformers import AutoModelForSeq2SeqLM, AutoTokenizer

model_dir = "/projects/mehparmar/apps/mehparmar/research/.cache/huggingface/hub/models--mistralai--Mixtral-8x7B-v0.1/snapshots/985aa055896a8f943d4a9f2572e6ea1341823841/"

dir_list = os.listdir(model_dir)
print("Files and directories in '", model_dir, "' :")
print(dir_list)

# Load the model
model = AutoModelForSeq2SeqLM.from_pretrained(model_dir)

# Load the tokenizer (can move this inside a try-except block for error handling)
tokenizer_path = os.path.join(model_dir, "tokenizer.json")
pathlike = pathlib.Path(tokenizer_path)
tokenizer = AutoTokenizer.from_pretrained(pathlike)
# Load the model
config_path = os.path.join(model_dir, "config.json")
model_path = os.path.join(model_dir, "pytorch_model.bin")


model = AutoModelForSeq2SeqLM.from_pretrained(model_path, config=config_path)

text = "Hello my name is"
inputs = tokenizer(text, return_tensors="pt")

outputs = model.generate(**inputs, max_new_tokens=20)
print(tokenizer.decode(outputs[0], skip_special_tokens=True))

I am loading model directly as internet connection is not allowed from notebook I am working on.

Error

---------------------------------------------------------------------------
ValueError                                Traceback (most recent call last)
Cell In[9], line 12
      9 print(dir_list)
     11 # Load the model
---> 12 model = AutoModelForSeq2SeqLM.from_pretrained(model_dir)
     14 # Load the tokenizer (can move this inside a try-except block for error handling)
     15 tokenizer_path = os.path.join(model_dir, "tokenizer.json")

File /opt/conda/envs/python39/lib/python3.9/site-packages/transformers/models/auto/auto_factory.py:566, in _BaseAutoModelClass.from_pretrained(cls, pretrained_model_name_or_path, *model_args, **kwargs)
    562     model_class = _get_model_class(config, cls._model_mapping)
    563     return model_class.from_pretrained(
    564         pretrained_model_name_or_path, *model_args, config=config, **hub_kwargs, **kwargs
    565     )
--> 566 raise ValueError(
    567     f"Unrecognized configuration class {config.__class__} for this kind of AutoModel: {cls.__name__}.\n"
    568     f"Model type should be one of {', '.join(c.__name__ for c in cls._model_mapping.keys())}."
    569 )

ValueError: Unrecognized configuration class <class 'transformers.models.mixtral.configuration_mixtral.MixtralConfig'> for this kind of AutoModel: AutoModelForSeq2SeqLM.
Model type should be one of BartConfig, BigBirdPegasusConfig, BlenderbotConfig, BlenderbotSmallConfig, EncoderDecoderConfig, FSMTConfig, GPTSanJapaneseConfig, LEDConfig, LongT5Config, M2M100Config, MarianConfig, MBartConfig, MT5Config, MvpConfig, NllbMoeConfig, PegasusConfig, PegasusXConfig, PLBartConfig, ProphetNetConfig, SeamlessM4TConfig, SeamlessM4Tv2Config, SwitchTransformersConfig, T5Config, UMT5Config, XLMProphetNetConfig.
2

@songogeta31 I think this is due to the transformers version that you’re using is a little bit old.
if you can get your hands on some internet connection you can run this in one of the cells in the notebook to update the library (only 1 time in your life then you can delete this)

!pip install -qU transformers

else since you mentioned that there is no internet connection you can plug your phone and use the usb theatering feature and then run the command above (there is no escaping the internet)
else there is another method but involves a usb flash drive to transfer some files from 1 computer to another and it’s too painful to explain, do let me know if you can make any of the above solutions work.

3

Thanks it worked! I am on transformers 4.39.0 now.

But my kernel is dieing the moment I start the model. How can I see logs to find the exact error

import os
import pathlib
from transformers import AutoModelForCausalLM, AutoTokenizer

model_dir = "/projects/mehparmar/apps/mehparmar/research/.cache/huggingface/hub/models--mistralai--Mistral-7B-v0.1/snapshots/26bca36bde8333b5d7f72e9ed20ccda6a618af24/"

dir_list = os.listdir(model_dir)
print("Files and directories in '", model_dir, "' :")
print(dir_list)

# Load the model
model = AutoModelForCausalLM.from_pretrained(model_dir)

# Load the tokenizer (can move this inside a try-except block for error handling)
tokenizer_path = os.path.join(model_dir, "tokenizer.json")
pathlike = pathlib.Path(tokenizer_path)
tokenizer = AutoTokenizer.from_pretrained(pathlike)

I am also using a smaller model now.

4

glad it worked, also the only logical explanation why Mistral-7B-v0.1 is failing is because you are running into some compute power (tldr; Mistral-7B-v0.1 is too big for your pc).
could you share your computer specs ? (ram and gpu)