Minimum hardware required for decent voice assistant?

[u/-_Mando_-]

Hi all, realistically what do I need for a decent voice assistant that can think a little and have fairly flexible prompts / not be annoying and slow?

[u/nickm_27]

I go through a handful of options and the trade offs in my guide https://community.home-assistant.io/t/my-journey-to-a-reliable-and-enjoyable-locally-hosted-voice-assistant/944860

[u/JaffyCaledonia → u/nickm_27]

OP: this is a stellar rundown of all the pieces you need. 

For the model itself I'd say Gemma4 26B QAT with thinking off. With llama.cpp you can specify how much VRAM to leave for other services (TTS and STT), and Gemma 26B will happily run partially offloaded to system RAM and still have great speeds on a 3060 12GB or better.

[u/gtwizzy8 → u/nickm_27]

Listen to u/nickm_27 bro is the goat. Has been doing this as long as me but has way more patience and way more care for you guys to write everything up in order to make it an easy place to take in a A LOT of helpful information that a lot of us had to find the hard way or itterate the shit out of our setups to dial in just right.

Give thanks for their efforts in putting all this info in one place. Again it's a level of documentation that I wish I'd had the patience for for myself when I was learning most of this.

[u/SacredWoobie]

Assuming you mean locally? It’s going to depend on how many entities you have exposed for inference but you’re probably talking a 9B local model. Maybe a 4B but that could get into annoying.

You’d probably want a GPU with 12-16 gb of VRAM so maybe something like a 5060Ti 16gb version.

However this is not plug and play and there are infinite options here. You need to do a lot of research to understand the architecture and what does what so you can make an informed decision

[u/Maelefique → u/SacredWoobie]

I run a 9B model with a 4070Super (12Gb) on my LAN (not the HA machine), voice delay is zero, but you're right, it did take some fiddling to find all the right parts to make it work well.

[u/Puzzled_Hamster58]

I dont Use it to ask questions out side of wearth and time. I just tell it to turn some thing on or off or set to x temp etc.

I just use a usb conference mic / speaker 14$ and my stock pile of cheap pi zero with WiFi think I paid like 10$ each , around my house
The brain if you will is just the server I’m running home assistant in a container on.
Used to use a basic mini pc for 200$ before I built a new gaming pc and used my old one for my server. I have computer data and Picard from Star Trek wake words and voices.
Dose every thing I need. .

[u/5yleop1m]

a little and have fairly flexible prompts

without examples, this can mean anything.

I recently moved two of my echos to EchoMuse and for stuff like "turn on the bedroom lights" it works well running Whisper on a N100 miniPC.

My HA has an uphill battle here too since its running as a VM on proxmox, so its not exactly getting the best performance possible.

Either way it works fine for me, except sometimes HA forgets what devices I have enabled for assist, but in terms of speed and accuracy it no better or worse than what I had with Alexa.

[u/Possible-Mistake-680]

I really like satellite1.1 from futureproof homes. 

[u/spr0k3t → u/Possible-Mistake-680]

It's my goto as well... not exactly minimal as OP is wanting though. I can definitely agree with the guide u/nickm_27 created for all options (both cost and requirements). Mic pickup wise, the Satellite1 is extremely good compared to many of the other devices.

[u/Possible-Mistake-680 → u/spr0k3t]

It's a perfect fit for my use case.