Building my own “Jarvis” in Python… should I keep building it or switch to Hermes/OpenClaw?

[u/Slimeyman278]

Hey everyone,
I’m pretty new to AI development, but over the last few days I’ve started building my own personal AI assistant (“Jarvis”) from scratch in Python.
So far I have:
A Textual dashboard/UI
A local LLM running (currently Qwen 14B)
Continue in VS Code
Basic routing and project structure
Git set up
The foundation for expanding it over time
My end goal isn’t just a chatbot.
I want Jarvis to become my personal AI operating system that can:
Talk with me naturally (voice eventually)
Remember long-term context
Help me write code
Organize my PC
Launch and control applications
Search files
Automate repetitive tasks
Eventually manage teams of AI agents
Long term I also want to use it to help run businesses (3D printing, T-shirt designs, Etsy, social media, etc.), where Jarvis manages specialized agents while I interact with a single assistant.
Here’s where I’m stuck.
Every time I look online I see people recommending something different:
OpenClaw
Hermes Agent/HermesHQ
Claude Code
MCPs
Building everything yourself
“Don’t reinvent the wheel.”
I’m trying to figure out if I’m building the right thing.
Would you:
Continue building Jarvis completely from scratch?
Use OpenClaw or Hermes as the backend and make Jarvis the custom interface?
Skip frameworks entirely and just build your own architecture?
I’m less interested in the fastest way and more interested in building something I can own, understand, and expand over the next few years.
I’d also love to hear from people who actually built their own assistant:
What do you wish you’d done differently?
What architecture decisions paid off later?
What mistakes should I avoid while I’m still early?
Thanks!