Hi, I am trying to load tokenizer for llama2 using AutoTokenizer but I am facing this issue “”“OSError: Can’t load tokenizer for ‘meta-llama/Llama-2-7b-hf’. If you were trying to load it from ‘ Models - Hugging Face ’, make sure you don’t have a local directory with the same name. Otherwise, make su…

Cannot load tokenizer for llama2

saireddy September 13, 2024, 3:48pm 7

model_id should be the name from hf model repository
for example : meta-llama/Meta-Llama-3-8B

1 Like