Hi, I’m hosting my app on modal com.
When I define it like this, implying that is supposed to be pulled from the repo it works fine, with exception of the time I have to wait for the model to be pulled.
tokenizer = AutoTokenizer.from_pretrained("NousResearch/Llama-2-7b-chat-hf")
model = AutoModelForCausalLM.from_pretrained("NousResearch/Llama-2-7b-chat-hf",)
but when I download contents from here, and put them into folder in the container and define the model like this so it could be accessed locally it gets stuck:
tokenizer = AutoTokenizer.from_pretrained(MODEL_FILES_PATH)
model = AutoModelForCausalLM.from_pretrained(MODEL_FILES_PATH)
It’s the same environment, the files are accessible. What could be the cause?
P.S. I’ve tried enabling safetensors, but it didn’t change much.