How to Use TTS - STT with Mistral-7b or Other LLMs for an Offline Voice Assistant?

Models
1

Hello everyone,

I’ve been diving deep into open-source AI research, specifically focusing on LLMs (Large Language Models) and voice assistants. The primary aim of my research is to develop an offline voice assistant, where TTS (Text-to-Speech) plays a pivotal role.

While I’ve come across solutions like Whisper AI and Bark AI, I haven’t found any concrete information regarding their compatibility with LLMs like Mistral-7b. If anyone has any experience or knowledge about this, it would be greatly appreciated.

Moreover, if anyone has access to Google Colab notebooks that demonstrate the use of TTS with LLMs, I’d be incredibly grateful if you could share them. Similarly, if you’re aware of any TTS AI systems that are known to be compatible with LLMs, please do share that information as well.

As this project progresses, I’m committed to sharing any useful findings and solutions with the community to benefit all. Your insights today can help shape better tools for everyone tomorrow.

Thanks in advance for any insights, resources, or advice!

2

So far this is the only information i found

3

Here is another video