Load downloaded Llama2 model with Transformers

Hello,

I downloaded Llama on MacOs and quantized it with llama.cpp. Now I want to load the model with Transformers, however the path I specified is wrong. So how can I load the downloaded model with transformers?

from transformers import AutoTokenizer, AutoModelForCausalLM

model = AutoModelForCausalLM.from_pretrained(
    "/Users/PATH/llama2/llama.cpp/models/7B",
    local_files_only=True,
    device_map="auto",
    )
tokenizer = AutoTokenizer.from_pretrained("/Users/PATH/llama2/llama/llama/tokenizer.py")

Here is a screenshot of my folder structure:

Thanks in advance!

1 Like