What is the "Best LLM for Shenava-Koochik ASR"

Models
1

Hi, i have a question. i have used the ASR for Reachy Mini it is okat at ASR but it can not get good disition at LLM what is your suggestion for that?
i am using “farsi” at speack and disition.

2

Worth noting Shenava-Koochik itself is a 114M Persian streaming ASR model (from Reza2kn’s Shenava collection), so ASR shouldn’t be the compute bottleneck here, the LLM stage is where “disition” (decision/intent understanding) is failing.

For Persian-language intent/decision-making specifically, general multilingual models like Qwen2.5 or Gemma tend to do noticeably better in Farsi than most “small” English-first models, that’s usually the actual fix rather than the ASR choice. If you want something Persian-specialized instead, PartAI/Dorna-Llama3-8B-Instruct is fine-tuned specifically for Persian instruction-following and reasoning. If you’re running this on Reachy Mini’s onboard Pi, 8B is probably too heavy for real-time local inference though, you’d want a smaller Qwen2.5 (1.5B/3B) or route the LLM call to HF Inference API/Endpoints instead of running it on-device, keeping ASR local and offloading only the decision step.

3

Thanks for the explanation. That makes sense. I was initially focusing on the ASR side, but your point about the LLM being the bottleneck is helpful.

I’ll keep Shenava-Koochik for Persian ASR and test Qwen2.5 (probably 1.5B/3B) for the intent understanding and decision stage first. If the results are not good enough, I’ll consider using a Persian-focused model like Dorna-Llama3-8B-Instruct or moving the LLM inference to an external API.

Thanks for the guidance!

4

Making local LLM + TTS work for a Persian voice assistant on a GTX 1650 Ti (4GB VRAM) — looking for advice

Hi everyone,

I’m building a real-time, two-way Persian voice conversation pipeline for a small robot project (Reachy Mini) with my professor. Current architecture:

  • ASR: Shenava-Rizeh (~32M params, Persian streaming ASR) via sherpa-onnx — fully local, CPU only, offline-capable
  • VAD: Silero VAD — fully local, CPU only
  • LLM: Qwen2.5-72B-Instruct via Hugging Face Inference API — cloud-only
  • TTS: edge-tts (Microsoft Edge’s cloud service, fa-IR-FaridNeural voice) — cloud-only

Why LLM/TTS ended up cloud-based: I initially tried qwen2.5:3b locally via Ollama (CPU-only, since my venv’s torch build is CPU-only), but the output quality in Persian was poor — frequent code-switching into English/other languages, incoherent responses. Switching to Qwen2.5-72B via HF Inference API fixed this completely; responses are now coherent and consistently Persian. edge-tts also gives good voice quality.

The problem: My internet connection is very slow and unstable (~0.7 Mbps down), so I’m getting periodic ConnectTimeout errors on the API calls (I’ve added retry logic, which helps but doesn’t eliminate the issue). For a robot that needs to work reliably, I’d like to move both LLM and TTS to fully local inference if possible.

My hardware:

  • GPU: GTX 1650 Ti, 4GB VRAM
  • CPU-only PyTorch currently installed in the project venv (I haven’t set up CUDA there yet)
  • Very limited/unstable bandwidth for downloading large model weights

What I’m trying to figure out:

  1. For the LLM stage: with only 4GB VRAM, what’s the best quantized model (Qwen2.5 7B? something Persian-tuned like Dorna-Llama3-8B?) that would fit and give reasonable Persian instruction-following quality, ideally without needing to download tens of GB?
  2. For TTS: are there any local/offline Persian TTS engines that get close to the naturalness of edge-tts? I found Piper has one Persian voice (“Amir”), but I’m not sure how it compares quality-wise to a commercial cloud TTS voice.
  3. More generally — has anyone gone through this same cloud-to-local migration for a low-resource language + constrained GPU setup, and what approach worked for you?

Any pointers to models, quantization strategies, or alternative local Persian TTS projects would be hugely appreciated. Happy to share more details (code, benchmarks) if useful.

Thanks in advance!

5

I’m working with the same GPU class (GTX 1650 Ti, 4GB VRAM), and I’ve run into the exact same issue: Persian ASR is fine, but the LLM stage becomes the real bottleneck, especially when you try to keep everything local and real‑time.

A few practical notes based on hands‑on testing:

  1. Qwen2.5‑3B and 1.5B are the best “small” multilingual models for Persian
    They don’t match the quality of Qwen2.5‑72B, but with good prompting and a tight context window they stay coherent in Farsi. Quantized GGUF builds (Q4_K or Q5_K) run on 4GB VRAM without crashing.

  2. Persian‑tuned models help, but only if you can run them
    Dorna‑Llama3‑8B‑Instruct is excellent for Persian reasoning, but 8B is too heavy for real‑time on a 1650 Ti unless heavily quantized and with reduced context. It’s usable, but not smooth.

  3. The biggest improvement comes from pipeline design, not model choice
    If you keep ASR local and make the LLM handle only short “intent/decision” messages, even a 3B model performs well. Long conversational turns will break small models in Persian.

  4. TTS: Piper “Amir” is the best offline Persian voice right now
    It’s not as natural as edge‑tts, but it’s stable, predictable, and fully offline. For robotics, consistency matters more than perfect prosody.

  5. VRAM stability matters more than raw model size
    Small GPUs fragment VRAM easily during repeated inference cycles. Keeping the context short, clearing GPU state between turns, and avoiding large embeddings helps a lot.

I’m preparing a small demo focused on stable local inference for low‑VRAM GPUs (including Persian pipelines). If you want, I can share it once it’s ready so you can test it on your setup.

But overall: for Persian intent understanding on a 1650 Ti, Qwen2.5‑3B (quantized) is the most realistic local option, and Piper is the most reliable offline TTS.

6

Thanks, this is a really useful data point — especially since you’re on the exact same GPU. It’s interesting that your recommendation (Qwen2.5-3B, quantized) is almost the opposite of what someone else here suggested (Gemma 4 E2B ~4B class), so I think the only way to actually know is to test both myself.

Your point about pipeline design mattering more than model choice makes a lot of sense, actually — my original 3B failure was on long, open-ended conversational turns running on CPU (never confirmed GPU was even being used). If I restrict the LLM to short intent/decision-style prompts and verify it’s actually running on the 1650 Ti, that alone might fix most of what I saw.

My plan now:

  1. Run ollama ps to confirm GPU usage (apparently my CPU-only PyTorch build in the project venv doesn’t necessarily mean Ollama is CPU-only too — someone else here pointed that out).
  2. Test Qwen2.5-3B (Q4_K/Q5_K GGUF) with a short, tight context and short target responses, and separately test a ~4B model, so I can actually compare instead of guessing.
  3. Try Piper “Amir” for TTS — good to hear it’s converging as the recommended offline option from multiple people here.
  4. Keep context short and watch for VRAM fragmentation across repeated turns, as you mentioned.

Appreciate the offer on the low-VRAM demo tool — I’ll hold off on that for now and focus on getting a baseline working with the standard Ollama setup first, but I’m interested once there’s something testable.

Will share my Qwen2.5-3B vs ~4B Persian comparison once I have both running on the GPU.

7

Great plan — and yes, confirming GPU usage is the first thing I would do too.
On the 1650 Ti, the difference between “CPU-only by accident” and “actually running on GPU” is night and day, especially for multilingual models.

A couple of extra notes that might help during your tests:

  1. Qwen2.5-3B (Q4_K/Q5_K) tends to stay stable as long as the context is short and the target output is tight. Persian intent-style prompts are usually fine. Long conversational turns will break it, but that’s expected for 3B.

  2. Gemma 4 E2B is definitely worth testing. In my experience it behaves more “English-first”, while Qwen handles Persian morphology better. But the only real answer is your own comparison.

  3. VRAM fragmentation is real on 4GB cards. If you see latency spikes after several turns, a simple “reset state” between turns can help more than changing the model.

  4. Piper Amir is consistent. Not as natural as cloud TTS, but predictable — which matters a lot in robotics.

Once you have your baseline with Ollama + GPU confirmed, I’d be very interested in your Qwen2.5-3B vs ~4B results. Persian benchmarks on low-VRAM GPUs are rare, so your findings will actually help a lot of people.

Whenever you want to try the low-VRAM demo tool, just let me know — no rush.

By the way — the low‑VRAM demo tool is ready now.
I’ve finished the installer and it’s stable on 4GB cards, so if you ever want to try it, I can upload it to a Hugging Face Space for easy access. No pressure at all — just letting you know it’s available whenever you’re curious.