Load downloaded Llama2 model with Transformers


I downloaded Llama on MacOs and quantized it with llama.cpp. Now I want to load the model with Transformers, however the path I specified is wrong. So how can I load the downloaded model with transformers?

from transformers import AutoTokenizer, AutoModelForCausalLM

model = AutoModelForCausalLM.from_pretrained(
tokenizer = AutoTokenizer.from_pretrained("/Users/PATH/llama2/llama/llama/tokenizer.py")

Here is a screenshot of my folder structure:

Thanks in advance!

1 Like