Hello,
I downloaded Llama on MacOs and quantized it with llama.cpp. Now I want to load the model with Transformers, however the path I specified is wrong. So how can I load the downloaded model with transformers?
from transformers import AutoTokenizer, AutoModelForCausalLM
model = AutoModelForCausalLM.from_pretrained(
"/Users/PATH/llama2/llama.cpp/models/7B",
local_files_only=True,
device_map="auto",
)
tokenizer = AutoTokenizer.from_pretrained("/Users/PATH/llama2/llama/llama/tokenizer.py")
Here is a screenshot of my folder structure:
Thanks in advance!