Hi, I am trying to import GPTQConfig from Transformers but am unable to do so. Here is my code:
!pip install transformers optimum accelerate auto-gptq
from transformers import AutoModelForCausalLM, AutoTokenizer, GPTQConfig
model_id = “meta-llama/Llama-2-7b-hf”
tokenizer = AutoTokenizer.from_pretrained(model_id, use_fast=True)
quantization_config = GPTQConfig(bits=4, dataset = “c4”, tokenizer=tokenizer)
However, I get this error:
ImportError: cannot import name ‘GPTQConfig’ from ‘transformers’ (/usr/local/lib/python3.10/dist-packages/transformers/init.py)
Any help will be appreciated!