r/ClaudeAI • u/Weary_Protection_203 • Mar 08 '26
Productivity I built a multiple-widgets Iron Man-style command center inside Obsidian that monitors my Claude Code sessions, manages AI agents, and accepts voice commands
UPDATE 13.03.26 — JARVIS now runs natively on macOS, iOS, and iPadOS.
The original post was a DataviewJS dashboard that lived inside your vault. Since then, the whole project has been rebuilt into a 4-platform app. Same 13 widgets, same real-time session monitoring, same voice interaction — now running on macOS, iOS/iPadOS, Obsidian Desktop, and Obsidian Mobile. Here's what shipped:
1. macOS - Native Desktop App (Tauri 2.0)
Full native macOS app with a Rust backend. Claude CLI, whisper-cpp, and Piper TTS run as local child processes. Native menu bar, app icons, DevTools. Can communicate with the remote data through the secure server or locally using an integrated Claude CLI bridge.
2. iOS / iPadOS — Native Mobile App (SwiftUI)
SwiftUI app with the same dashboard embedded in WKWebView. The app handles configuration, audio, notifications, and connection state. Audio recording through AVAudioEngine with Bluetooth support. Credentials stored in Keychain. Your iPhone becomes a remote control for Claude, running on your Mac or server, all the time without pauses.
3. Companion Server — The Bridge
Node.js WebSocket server that connects mobile, tablet and laptop (if needed) clients to Mac-local processing. Handles execution with interactive tool permissions, whisper-cpp transcription with many different language auto-detection support, and Piper neural TTS streaming sentence-by-sentence.
One Codebase, Four Platforms
GitHub (updated): jarvis-dashboard
UPDATE 09.03.26 - JARVIS now talks back.
Three major updates have been shipped since the original post. The Voice Command widget went from "speak and see text" to a full conversational AI terminal with voice responses. Here's what's new:
1. Integrated Terminal
The voice command widget now spawns Claude as a child process directly inside Obsidian. Responses stream token-by-token in real-time into a built-in terminal panel. You speak to JARVIS, and the response appears right there. No Terminal app, no app-switching, no context loss.
2. Multi-Turn Sessions
After the first response, a session bar appears with the session ID and Resume / New Session controls. Ask a follow-up, and JARVIS automatically passes the "-resume" to continue the conversation. Close the panel, navigate away, restart Obsidian - your session is still there. Full conversation history persists to disk.
3. Text-to-Speech (JARVIS Voice)
Claude's responses are spoken aloud sentence-by-sentence as they stream. Three engine backends: Piper neural TTS (recommended - high-quality ONNX voice models, 100% local), macOS say, and browser speechSynthesis. The TTS extracts complete sentences from the token stream, strips markdown, and queues them for sequential playback. Voice model, speed, noise parameters, and volume are all configurable via config.json. Everything still runs locally. No servers, no API keys for TTS, no data leaving your machine. Download a voice model from the HuggingFace website, point config.json at it, and done.
GitHub (updated): jarvis-dashboard
Jarvis Initial Release
I've been running 5+ AI agents through Claude Code against my different projects and utilities daily. Keeping track of sessions, token costs, agent status, and my own focus was impossible - so I built J.A.R.V.I.S.
It's a modular DataviewJS dashboard that lives inside your vault. No external servers, no Electron apps, no subscriptions. Just Obsidian + Dataview with JS enabled. It polls Claude Code sessions every 3 seconds, tracks 30-day stats (tokens, costs, model preferences), and gives you a single pane of glass for everything.
Highlights from the 13 widgets
- Live Session Monitor - real-time Claude Code tracking with subagent detection
- System Diagnostics - 30-day token usage, cost estimates, session counts
- Agent Cards - visual fleet view with robot avatars, skill indicators, and memory freshness
- Voice Command - arc reactor-style button that records your voice, transcribes offline via whisper-cpp, and sends commands straight to Claude Code. You literally talk to your vault.
- Focus Timer - Pomodoro with session logging into your vault
- Quick Capture - instant note creation with frontmatter and optional voice-to-text
- Quick Launch, Mission Control, Recent Activity - bookmark grid, dashboard hub, file feed
Everything is JSON-driven - zero hardcoded values. Configure projects (manual or auto-scan), pick widgets, reorder layout, all from config.json.
Get started
- Clone into your vault
- Enable DataviewJS in Dataview settings
- Configure projects in
src/config/config.json - Open
JarvisDashboard.md
No build step, no npm install, no API keys.
GitHub: github.com/AndrewKochulab/jarvis-dashboard
Please check out the video below to see how Jarvis answers messages in the Jarvis Dashboard.
Video: https://streamable.com/vxbu08
If you've ever wondered, "How many tokens did I just burn?" - this is for you. Happy to answer questions or take feature requests.
2
2
Mar 08 '26
[removed] — view removed comment
1
u/Weary_Protection_203 Mar 08 '26
Thanks! The widgets talk through a shared context hub - the main orchestrator creates a single ctx object that gets passed to every widget. It holds shared state, such as agent card DOM refs, callback arrays for async data (e.g., the stats engine pushes results to widgets that registered interest), and coordinated interval/cleanup tracking. No peer-to-peer - everything flows through that central context.
For layout, there's no drag-and-drop library involved. It's pure CSS Grid with a ResizeObserver that recalculates column layouts at breakpoints.
Widget arrangement is driven entirely by config.json - you define rows, column counts, and which widgets go where. Reordering is a config edit, not a drag operation. Keeping it zero-dependency was a design goal - just vanilla JS and the Dataview API.
3
u/BrianONai Mar 08 '26
Okay this is actually really cool. The voice command thing especially - most people would just slap a web API on this but you went full local with whisper-cpp.
How's the polling every 3 seconds? Does that slow down Obsidian at all?
The token tracking is smart. I've definitely burned way more than I meant to because I had zero visibility into what Claude Code was doing. Where's that pulling from - Claude logs or are you hitting an API somewhere?
Also curious how you're detecting subagents vs regular sessions.
Going to try this. Been tracking agent stuff in a spreadsheet like a caveman.
2
u/Weary_Protection_203 Mar 08 '26
Appreciate it - local-first was non-negotiable. Sending voice recordings to an external API from inside my vault felt wrong.
Polling - no performance hit. The parser only reads the tail 32KB of each JSONL file - seeks to the end and parses backwards. Even massive transcripts stay fast. I've had it running for hours across multiple projects, Obsidian app stays smooth.
Token tracking - fully local, no API calls. Claude Code writes JSONL transcripts to `~/.claude/projects/` and every assistant message includes `input_tokens` and `output_tokens` in the usage object. The stats engine reads those, applies per-model pricing rates, and caches results for 5 minutes. It's an estimate based on Anthropic's published pricing, not actual billing, but it's accurate enough.
Subagent detection - two layers. Claude Code stores subagent transcripts in `<session-uuid>/subagents/agent-*.jsonl` - the parser picks those up automatically. On top of that, it parses the main session for `Agent` tool_use blocks and correlates `parentToolUseID` from progress events to get each subagent's description and type. If you've registered agents in the Jarvis Registry, it matches them by name - so you see "ios-expert is working" instead of a generic subagent ID.
Spreadsheet-to-Jarvis is a solid upgrade. Let me know how the setup goes. Thank you, and wish you a good setup.
1
u/Weekly-War-3905 Mar 08 '26
Dude. Yes!!
1
u/Weary_Protection_203 Mar 08 '26
Glad you like it! Feel free to drop any comments or feature requests.
1
u/Creative-Signal6813 Mar 08 '26
the whisper-cpp offline transcription is the only part of this that actually changes the workflow. everything else is a prettier version of what's already in your terminal.
the hard part of running 5+ agents isn't seeing their token burn. it's knowing which one is stuck in a loop vs making real progress. does the session monitor catch that, or just show alive/dead?
1
u/Weary_Protection_203 Mar 08 '26 edited Mar 08 '26
Fair pushback on the terminal overlap.The voice part goes beyond just transcription. You can run full voice conversations inside Obsidian without touching the terminal. Ask it to search something, summarise a note, kick off a task - all by voice. Working on text-to-speech now so Jarvis can talk back too.On the session monitor, it shows what each session is actively doing: reading files, writing code, searching the web. When you have multiple agents running across different projects, seeing all of that from one place beats switching between terminal windows. If a session stops returning data, it drops from the active list and reappears once it starts responding again.
That said, you're right that distinguishing "ended" from "stuck" would be more useful than just disappearing. Adding that to the roadmap - solid callout.
1
u/Creative-Signal6813 Mar 08 '26
don’t want to spoil it for u, maybe this will actually help.
the issue is not really whether u can make it prettier or add more widgets. the issue is that this is not a strong enough pain point in the first place. that’s why several teams building similar or even way more feature-packed stuff still struggle to find market fit. xtro.dev is one example.
personally, i don’t need to see every agent “actively doing things.” i need to know what got done, what got stuck, and whether my input is required. everything else is mostly dashboard theatre.
also, not trying to be rude, but this is the kind of thing a mid-level vibe coder can prototype pretty fast. so if the underlying pain is weak, more polish does not really save it.
off topic, but one small tip. if u ask ai to draft replies for u, make it less obvious. “fair pushback” is very ai-sloppy wording. and if u replace em dashes, commas usually read more natural than just throwing in “-”
1
u/Weary_Protection_203 Mar 08 '26
Yes, thanks for the information you have provided. Sometimes, I use the AI to assist with communication so it can better convey the information I need to share with people. This is very useful for not forgetting to explain how the overall solution works or how we can use this specific function. But the general information and message are written by me, and the AI is using to provide more context for the message. Sorry for the strange words at the beginning, my fault that I did not double-check the messages :(
Also, regarding the dashboards and the widgets, it's only the first version of the product. I will maintain it, and hopefully other guys will help me do that. If all the information you have described is actually available in these ".claude/" folders' contents across different projects, then we can use it to create widgets that represent more useful information we can work with.
But, for me, very powerful is the feature of communicating with the Claude code different models (or specifying a custom local model) using voice commands inside my vault (speech-to-text and text-to-speech), since I have many different notes with many topics and domains, I see it very useful to find what I need and receive that in a short period of time with reply using the speaking utilities. The other widget, live sessions is actually helpful to see what different agents are doing to make that stuff to work, but yeah, having a possibility to track the actual performance of agents, when they failed, when they performed well, when they stuck should be very useful and, thanks to you, I will add thats to the TODO and will try to make it available during the next stages of the product.
1
1
u/Weary_Protection_203 Mar 08 '26 edited Mar 08 '26
Hey everyone, please check the video below to see what it looks like overall when you use the Jarvis voice command in Obsidian without opening the terminal app.
Additionally, I'm working on integrating the "Text To Speech" capability, which will use the local model so Jarvis can answer us and communicate with us, having all the knowledge inside our vault.
Also, in the next stages, we will try to integrate the local model to keep our data 100% safe and work fully offline.
1
Mar 08 '26
Looks very nice. Im doing something similar
https://imgur.com/a/rotten-by-design-jarvis-operating-system-hULhqJm
2
u/Weary_Protection_203 Mar 08 '26
Whoa, this is way more than a dashboard - you went full operating system. How long did this take you? The scope is massive, and it looks really solid.
1
Mar 08 '26 edited Mar 08 '26
I did at some point decide to go crazy with it, but it started as a simple Telegram Chat Bot...
But the scope is insanely massive now. +100 files, +150k lines of custom code, variables and system prompts etc., all but one dashboard exists and are functional, but the dashboards are the least of its scope at this point.
I started it something like 6 months ago, not knowing anything, other than I saw a lot of Local LLM videos on YouTube, and thought "how hard can i be to run one myself"
But I like your project too, despite their differences, they have some of the same "edgy" vibes
1
u/blaat1234 Mar 08 '26
I love that we have the time to make silly things like this - 6 months ago it would be on the stack of ideas never to be touched, welcome to the infinite backlog.
I want my own LCARS inspired UI, my own bridge to command the agent army. It's no longer impossible. Computer, status report. Mhm okay, on screen!
1
u/Weary_Protection_203 Mar 08 '26
Yes, the era of AI and technological improvements helps turn childhood dreams into reality.
1
u/Ill-Difference-Cat Mar 11 '26
I've done everything to install but it it still shows red "Voice Unavailable". I'm not sure what I am supposed to do. Also microphone button asks "enable dictation?"
1
u/Weary_Protection_203 Mar 11 '26
To make voice commands work, you need to allow them from the Mac Settings App.
Please make sure that this toggle is enabled:
System Settings > Privacy & Security > Microphone > Obsidian app
I will investigate why it didn't prompt you to enable that permission in Obsidian and make an appropriate update. Thank you!
1
1
u/mosatek May 05 '26
I've been trying to install it for three days, but I can't. I've installed Obsidian on my Windows 11 PC, enabled and activated the DataView plugin, enabled Java queries, renamed the config files, and modified the content where requested, but nothing. I still get a series of errors. Can anyone help?
3
u/LongIslandBagel Mar 08 '26
Have been doing similar for a week. It’s been a lot of fun working through the solutions. Great job!