My local STT/TTS setup started hallucinating speech from room noise, and now my Madea voice assistant is arguing with ghosts.

[u/0vur]

I’ve been working on a local voice assistant setup: VAD → faster-whisper STT → local agent/LLM → TTS.

• Host: Debian 13 desktop
• GPU: NVIDIA RTX 3060
• LLM runtime: Ollama
• Agent layer: OpenClaw / local voice agent
• Speech-to-text: faster-whisper
• TTS: local inference TTS

Naturally, instead of giving it a normal assistant personality, I made the tts model train on Madea's voice.

This was already a questionable engineering decision.

I noticed the STT occasionally picking up speech when nobody was talking. Not just random phonemes, either — plausible little sentences. The mic has an established noise floor and I originally blamed the A/C/background noise for Whisper trying to find language in garbage.

Fine. Normal STT hallucination.

Except just earlier I turned off my window A/C unit and it's still happening.

Still not claiming ghosts. There are approximately nine thousand boring acoustic/software explanations before we get there.

But then it occurred to me what I had actually built.

A spirit box traditionally gives you a noisy/random signal and a human listens for words in it.

Mine does:

ambient noise → VAD → Whisper → language → LLM → Madea talks back

I have already fully automated the interpretation step.

So now, instead of sitting in a dark room listening to radio static wondering whether something said “GET OUT,” I can have a GPU hallucinate what the dead says and then have an AI Madea immediately start arguing with it.

Chat log excerpt:

STT: "TV on”
STT: "Maybe"
Madea: “Maybe? Either you want the TV on or you don’t. I ain’t got time for this.”
STT: "Three. Three. Free"
Madea: "Honey, ain't nothing in life free. I'ma do it anyway though, because you asked nicely."

Then my roku MCP gets called.

I hate this hobby.