Nemotron-3-Omni GGUFs with working audio + video in llama.cpp (9 tested quants, one-pass A/V)

Show and Tell
1

NVIDIA’s Nemotron-3-Nano-Omni-30B is a genuinely multimodal model: image, audio, and video in. But until now the GGUF ecosystem only carried its vision half. The existing GGUF repos ship vision-only projectors, and the video path did not exist in llama.cpp at all.

I built the missing pieces and released the whole package:

Models: engram-ae/Nemotron-3-Nano-Omni-30B-A3B-Reasoning-GGUF · Hugging Face
Nine quantizations from Q3_K_M to BF16, plus a unified projector file that carries both encoder towers (C-RADIO vision and Parakeet audio). One llama-server instance takes every modality on /v1/chat/completions, including the built-in web UI: drag in a photo, a wav, or an mp4.

Code: GitHub - VincentKaufmann/llama.cpp-omni: llama.cpp fork: Parakeet/FastConformer audio graph for Nemotron-3-Omni (mtmd) · GitHub
A llama.cpp fork with the Parakeet/FastConformer audio graph and the C-RADIO video graph (temporal patches plus EVS token pruning), including one-pass video WITH its soundtrack: the model describes what it sees and quotes what it hears from a single request. Prebuilt Linux arm64 CUDA 13 binaries in the releases (DGX Spark, Jetson Thor).

Verification, since claims are cheap:

  • The video graph matches NVIDIA’s PyTorch reference to 0.0019 percent relative L2, with exact EVS selection parity
  • The audio graph transcribes the test narration word for word
  • Every quantization was tested on image, audio, and video with soundtrack before upload. The graded matrix ships in the repo as TEST-MATRIX.md, so you can see exactly what was checked rather than taking my word for it

Everything was built and tested on a single DGX Spark (GB10). Quick start is in the model card: one server flag pair, then open the browser.

Roadmap: the next release rebases onto current llama.cpp master and adopts the upstream audio projector layout, so image, text and audio will run on stock llama.cpp with these same files; the fork remains for video and one-pass A/V. An upstream PR for the video path is planned.

Base model and encoders are NVIDIA’s, under the NVIDIA Open Model Agreement. I wrote the GGUF conversion, the llama.cpp graphs, and the projector unification. Happy to answer questions or fix what breaks.

1 Like