AutoModelForCausalLM.from_pretrained unable to load model from Huggingface

🤗Transformers
1

I am trying to load a model TheBloke/WizardLM-30B-Uncensored-GPTQ downloaded from Huggingface by using transformers.AutoModelForCausalLM.from_pretrained.

When I try to do this using the below code, i get the error

OSError: Error no file named pytorch_model.bin, tf_model.h5, model.ckpt.index or flax_model.msgpack found in directory /text-generation-webui/models/TheBloke_WizardLM-30B-Uncensored-GPTQ.

I looked into that directory and see the following files

added_tokens.json
config.json
generation_config.json
huggingface-metadata.txt
quantize_config.json
README.md
special_tokens_map.json
tokenizer_config.json
tokenizer.json
tokenizer.model
WizardLM-30B-Uncensored-GPTQ-4bit.act-order.safetensors

What should be the proper way to load this huggingface model?

import torch
from transformers import AutoModelForCausalLM, BitsAndBytesConfig

model_id = "/text-generation-webui/models/TheBloke_WizardLM-30B-Uncensored-GPTQ"
nf4_config = BitsAndBytesConfig(
    load_in_4bit=True,
    bnb_4bit_use_double_quant=True,
    bnb_4bit_quant_type="nf4",
    bnb_4bit_compute_dtype=torch.bfloat16,
)
model = AutoModelForCausalLM.from_pretrained(
    model_id, quantization_config=nf4_config, device_map={"": 0}
)
2

Me to. Ever solve this?