Seems common with IQ3_M. Example, go here:
Click on the black gguf viewer icon for the IQ3_M. Then choose ollama. Look at the download link:
ollama run bartowski/Llama-3.3-70B-Instruct-ablated-GGUF · Hugging Face
Note what’s not present: the actual GGUF you’re trying to use. Compare to the links for other quantizations:
ollama run hf.co/bartowski/Llama-3.3-70B-Instruct-ablated-GGUF:IQ2_XXS
Note how the quantization is listed after it.
It’s not just an issue with links. If I manually run:
ollama run hf.co/bartowski/Llama-3.3-70B-Instruct-ablated-GGUF:IQ3_M
It fails.
This is not an ollama issue - the link physically does not exist, even though the GGUF model exists (and can be downloaded via LFS). If you just do the original link provided, without the gguf name, it doesn’t get the IQ3_M, it gets the latest Q4.
Okay, I’m confused. Why on Earth did a bug report get flagged as spam?