Hey there,
my goal is to run Efficient-Large-Model/VILA-7b on a jetson device through Ollama. As far as I could see there’s no “out-of-the-box” support to convert the model weights into the .gguf format without losing its vision component. I did some digging and found the /app/examples/convert_legacy_llama.py from llama.cpp but it still doesn’t do the trick for the vision tower.
Maybe I’m being ignorant about some obvious solution but so far I couldn’t get around to it. Any ideas how to convert the model properly?
Any pointers, guides, tutorials etc. to achieve the above goal will be much appreciated!