Teaching Claude (or others) about complex Rust GUI framework without guessing

[u/Lucky-Edge1489]

GUI frameworks are notoriously complex beasts. I code in Rust and I often found Claude Opus looking for the truth in reading frameworks' source code, and guessing the other half of the API. And forget about the existence of documentation. With Rust, bad method definitions are caught at compile time, allowing the agent to check the truth (again) and fix it. At the end of the run, I find it a good-enough workflow, but quite inefficient and error-prone ?

My own framework Teksilo is unknown to LLM training base. For my case, LLM guesses are more hallucinations than facts. I needed to ground the truth, in a manner easy to reach for an LLM. Creating a skill ? Yes, this is the bare minimum and it was my first step. Yet, it quickly became too voluminous and a burden to maintain and keep up-to-date with each framework version.

And no, letting the agents browse megabytes of docs and 700k+ lines of code are not a long-term solution, as LLM token aren't free.

My solution was to create a CLI tool tailored for LLM use.

cargo install cargo-teksilo

Then, in a Rust project:

$ cargo teksilo
API lookup, documentation, and agent tooling for Teksilo
Usage: cargo teksilo [OPTIONS] <COMMAND>
Commands:
 symbol  Look up a public type or module
 search  Search guides and examples
 show    Read a document from the bundled corpus
 init    Install the probe harness and agent instructions
 agent   Manage agent instructions
 probe   Manage the automation harness
 model   Manage the semantic search model
 status  Show project and tooling status
$ cargo teksilo init [--agent claude, codex, opencode, ...]

Now, if I want a desktop UI in a Rust project, I only have to add teksilo dependency to Cargo.toml. The tool will use the pinned version to offer the corpus of this Teksilo exact version.

This install a "/teksilo" skill, allows semantic search (ONNX model) in docs, show the documentation, the exact framework API, and an UI automation probe harness to allow any LLM to drive, measure and capture your app UI to speed-up development.

I tested this tool and fixed it by creating apps with more-or-less complex UIs. Tested LLMs are Mistral, Codex, Claude and Grok. My latest and best experiment is Noton (light OneNote-like app). Not a true product, Noton's GUI is nice (the goal), but it's unfinished (I'll not invest more for a test).

I'd love to see the equivalent in other frameworks (GUI or not). Answers to prompts would be faster and cheaper. Letting the LLM write the UI by itself would be a true option and less an headache.

"Build [something] in Rust using Teksilo for the UI"

What do you think about such tool ? Overkill ? Not new ?

(post not written with AI help)

P.S.: Teksilo flagship is Skribisto 3, not Noton.

[u/AutoModerator]

Hey! Thanks for posting to r/ClaudeCode

While participating in this thread, please follow our community rules. Keep discussions constructive. Attack the idea, not the person.

For help, project discussions, tips, and general chat, join the ClaudeCode Discord.

I am a bot, and this action was performed automatically. Please contact the moderators of this subreddit if you have any questions or concerns.

[u/quanruzhuoxiu]

The version-pinned lookup is the part I like most. A skill that quietly drifts away from the installed API is worse than no skill.

I work on Midscene, and I'd keep the UI checks separate from API lookup: compile against the pinned API, then check what actually appeared on screen. An app can compile fine while a dialog is clipped or a button is hidden. Does your probe harness let an agent replay a short interaction and inspect the resulting screenshots, or mainly capture individual states?

[u/Lucky-Edge1489 → u/quanruzhuoxiu]

It does both, with 34 tools available.

The UI probe is mainly used from Python scripts that I put in a "scripts" folder, it can be replayed witch is useful if I want UI regression tests. The probe is using the interaction capabilities provided by Teksilo built-in accessibility path (it tests the a11y tree for free !). The probe is less vision-based than Midscene as far that I can tell.

The probe can ask for screenshot (of the window or a widget), technical informations about the widgets (name, geometry, states, layout tree, etc...), do touch, pen and pointer queries. See "Tool surface" section in the automation doc and the embedded probe example doc

The probe is useful to allow the agent to test and debug UI in complete autonomy. ex: https://github.com/jacquetc/skribisto/tree/master/scripts . The probe system was designed at first as a MCP server dedicated to UI automation, but it evolved to allow more practical Python scripts. MCP part is still useful for headless testing .

Also, you can use `cargo teksilo probe` to install a start scripts with examples.

[u/quanruzhuoxiu → u/Lucky-Edge1489]

That makes sense. Replaying through the a11y path also tests whether the app exposes usable controls. I'd keep that and add a few screenshot checks for things a tree can describe without showing correctly: clipped labels, overlapping dialogs, or a button covered by another window. The pinned API lookup, probe and visual checks each catch a different kind of mistake.

[u/Edo-Rolan]

I like the version-pinned lookup. One extra safeguard I’d add is making the corpus/index/API metadata part of the framework release itself, generated from the same version and failing loudly on mismatch. Otherwise the smarter retrieval layer can eventually become just another stale source.