Hi, I’m following this tutorial Llama 2: AI Developers Handbook | Pinecone , which explains how to load Llama-2 with quantization. I’ve worked through a couple of error messages that I managed to solve. However, now I am stuck with the following response from the model: “FP4 quantization state not …

Loading Llama 2 with quantization on M1 MacBooks

marabgol December 15, 2023, 9:47pm 3

Salam Iraj
have you found any solution?

1 Like