ValueError in deploying HuggingFace Model to Sagemanker with Multiple Lora Adapters

Ruchika1809 · October 21, 2024, 3:20pm

I am getting below ValueError in deploying HuggingFace Model to Sagemanker with Multiple LoRA Adapters

ValueError: The checkpoint you are trying to load has model type starcoder2 but Transformers does not recognize this architecture. This could be because of an issue with the checkpoint, or because your version of Transformers is out of date.

Let me know if anyone has faced this earlier

John6666 · October 21, 2024, 10:24pm

Maybe this issue.

github.com/bigcode-project/starcoder2

Clash in requirements for finetuning Starcoder2

opened 10:35PM - 10 Mar 24 UTC

Exorust

#Facing the following error while trying to finetune Starcoder2 with the given s…cript. ### Description: For `transformers.AutoModelForCausalLM` to recognize Starcoder2 `transformers>4.39.0` is required. But `trl` is still using `transformers==4.38.2`. Even if I compile from source & use `trl=0.7.12.dev0` I still get an issue. ## Here is the error with using `transformers==4.38.2` ``` KeyError Traceback (most recent call last) [/usr/local/lib/python3.10/dist-packages/transformers/models/auto/configuration_auto.py](https://localhost:8080/#) in from_pretrained(cls, pretrained_model_name_or_path, **kwargs) 1127 try: -> 1128 config_class = CONFIG_MAPPING[config_dict["model_type"]] 1129 except KeyError: 4 frames KeyError: 'starcoder2' During handling of the above exception, another exception occurred: ValueError Traceback (most recent call last) [/usr/local/lib/python3.10/dist-packages/transformers/models/auto/configuration_auto.py](https://localhost:8080/#) in from_pretrained(cls, pretrained_model_name_or_path, **kwargs) 1128 config_class = CONFIG_MAPPING[config_dict["model_type"]] 1129 except KeyError: -> 1130 raise ValueError( 1131 f"The checkpoint you are trying to load has model type `{config_dict['model_type']}` " 1132 "but Transformers does not recognize this architecture. This could be because of an " ValueError: The checkpoint you are trying to load has model type `starcoder2` but Transformers does not recognize this architecture. This could be because of an issue with the checkpoint, or because your version of Transformers is out of date. ``` ## Here is the error when using `transformers==4.39.0` ``` ImportError Traceback (most recent call last) [<ipython-input-2-3ef713ffd06d>](https://localhost:8080/#) in <cell line: 1>() ----> 1 from trl import SFTTrainer 2 print("trl version:", trl.__version__) 1 frames [/usr/local/lib/python3.10/dist-packages/trl/__init__.py](https://localhost:8080/#) in <module> 3 __version__ = "0.7.12.dev0" 4 ----> 5 from .core import set_seed 6 from .environment import TextEnvironment, TextHistory 7 from .extras import BestOfNSampler [/usr/local/lib/python3.10/dist-packages/trl/core.py](https://localhost:8080/#) in <module> 23 import torch.nn.functional as F 24 from torch.nn.utils.rnn import pad_sequence ---> 25 from transformers import top_k_top_p_filtering 26 27 from .import_utils import is_npu_available, is_xpu_available ImportError: cannot import name 'top_k_top_p_filtering' from 'transformers' (/usr/local/lib/python3.10/dist-packages/transformers/__init__.py) --------------------------------------------------------------------------- NOTE: If your import is failing due to a missing package, you can manually install dependencies using either !pip or !apt. To view examples of installing some common dependencies, click the "Open Examples" button below. --------------------------------------------------------------------------- ```

Topic		Replies	Views
Deployment issue in AWS Sagemaker and GCP 🤗Transformers	0	191	April 2, 2024
ValueError: Could not load model /opt/ml/model with any of the following classes: (<class 'transformers.models.auto.modeling_auto.AutoModelForCausalLM'>, <class 'transformers.models.llama.modeling_llama.LlamaForCausalLM'>) Amazon SageMaker	0	373	March 13, 2024
AttributeError: 'TensorParallelColumnLinear' object has no attribute 'base_layer' Models	4	86	January 9, 2025
Getting error in the inference stage of Transformers Model (Hugging Face) 🤗Transformers	0	761	October 11, 2022
Finetuning Vision Encoder Decoder Models with huggingface causes ValueError: expected sequence of length 11 at dim 2 (got 12) Beginners	0	490	March 12, 2023

ValueError in deploying HuggingFace Model to Sagemanker with Multiple Lora Adapters

Related topics