Ollama uses GGUF format files. There are several methods for quantizing Hugging Face’s Transoformers format to GGUF, but using the conversion script included with Llama.cpp is relatively simple.
Ollama uses GGUF format files. There are several methods for quantizing Hugging Face’s Transoformers format to GGUF, but using the conversion script included with Llama.cpp is relatively simple.