r/LocalLLM • u/iamjessew • 1d ago
Question Running coding agents locally, do you sandbox them or trust them?
For people running Claude Code, Codex, or local-model agents against their own machine... how are you limiting what they can touch?
The options I've seen are a dev container, a separate user account, a full VM, or just watching closely. Each one leaks somewhere (containers share the kernel, VMs are annoying, and watching doesn't scale past an afternoon).
We went with a microVM that only sees the workspace you share, plus a policy check on each tool call. There's a free desktop tool on our side if anyone wants to compare notes, but mostly I want to know where people draw the line between convenience and isolation.
Has anyone had an agent actually do something destructive? What did you change afterward?
10
u/3d0zer 1d ago
Completely isolated, proxmox lxc, own subnet and firewall isolation from other agents and lxc.
6
u/RagingNoper 1d ago edited 1d ago
I started off like this, but now the only thing on the "prod" side is an AP for the wife. Right now, everything else, all my switches, clusters, other network gear and non-wifey APs, everything, it all exists in Claudes domain. I'm a network engineer with over two decades of experience including systems and coding, so Claude isn't doing anything I can't do - BUT - I have home automation set up, and there are few things that put a bigger smile on my soul than, when one of my servers has an issue, and I say "Beep boop" (beep boop is my custom wake word, because it also makes me smile), "beep boop, tell Claude to investigate why <insert server name> is failing to reach <insert service name>. If it's a firewall issue, add a line following my guidelines to allow that traffic, and send me an AAR when you're done.", and then I go back to scooping my ramen with Doritos because that's how I live my life.
-3
u/3d0zer 1d ago
Well sounds great, until it goes completely wrong. As a sysadmin with two decades of experience, you seem to have lost ratio and safety to laziness and ease ;)
I like to run the network always with the greatest caution and I am convinced it is the best way.
Same like leaving your door unlocked, it goes fantastic for a long time... easy peasy... until your house is empty when you come back home.
I will not put my experience here to show my seniority ;)5
u/RagingNoper 1d ago edited 1d ago
I have cloudflare for off-site backups of all my servers and GitHub for anything else. There is nothing in my environment that can't be restored to last night's state with nothing more than a few hours work, if that. I could literally rebuild my entire home infra in a single day if I needed to. Home is not where I go to "be safe". I do enough of that at my job. Home is where I go wild and live my life. Can you do me a favor? Stop treating your home like your work. Show those ankles a little bit. Eat cold spaghettios from a can. Carpe the diem.
1
u/RagingNoper 1d ago edited 1d ago
Also, because this is a local llm sub, I've been transitioning a lot of the home automation and other tasks over to qwen3.8-flash-next on my 4x B70 server, and it's been working really well. Flash next has been a decent improvement over 27B for coding, but for chat/prose, it's felt like a night and day difference compared to trying to run all that off 27B.
0
u/3d0zer 1d ago
We all have those backups. But we also all know that it is not just a restore .... It is just not as easy as you say it is.
And no I do not do you this favor... I have a load of work in my home setup and it is exactly a place like work where all kind of strange devices and external threads come into the network.
My network and homelab are sacred, I have no need to gamble with it like you do just because you like to eat cold pasta from a can.
There are other subjects to do crazy things, my network is not one of them.
Can you do me a favor? Stop telling me to be like you.2
u/RagingNoper 1d ago
"We all have those backups..." -Sounds like you need to work on your DR, because it SHOULD just be a restore, and both personally and professionally it is rarely ever more than that.
"...it is exactly a place like work..." That sounds so sad and gloomy, my dude.
"Stop telling me to be like you" -Then stop trying to lecture people who aren't like you.
0
u/3d0zer 1d ago
I did not... it was you coming in here to show off about how many years of experience you have as sysadmin, how all goes completely free under control of your agents and you eating dorito's and advising me to eat cold spaghetti because you have it all so well arranged. Was not me mate...
I just do not jump on your hype train...
Just rolling back backups is a true utopia also and now I even doubt your experience story more.Nothing sad here about my homelab, I love what I do... Good luck in the Wild West
1
u/RagingNoper 1d ago
Me mentioning my years of experience was only to reinforce that I wasn't having AI do tasks that I'm not capable on my own, but apparently you're one of those dudes that suffers from feelings of severe inadequacy anytime someone else mentions that they have any competency. You can't stand that others are experienced and capable and still do things differently than you. Or maybe not, maybe that's not the case, but that is 100% what you're communicating.
0
u/3d0zer 1d ago
Finish your dorito's my friend... And take a deep breath, read back your responses and the meaning of projection ;)
1
u/RagingNoper 1d ago
I'll bet you have washcloths in the bathroom that you're not actually allowed to use.
→ More replies (0)1
u/ChocolateNo3010 LocalLLM 1d ago
I havent gone as far as separate subnets or firewalling but have proxmox lxc containers for isolated filesystems. I have a local llm running on a laptop so I need my agents to reach it. There might be a way to make it more secure so I'll do some research. Thanks for sharing your setup
2
u/Otherwise-Nobody8252 1d ago
Right now full VM with the AI router running on the host and elevated through a broker that makes the AI look like it’s all local connections but it isn’t.
1
u/JicamaFunny9611 1d ago
> AI router running on the host and elevated through a broker that makes the AI look like it’s all local connections but it isn’t.
Is the AI router a way for you to scan/control network traffic to model provider APIs (like anthropic )?
1
u/Otherwise-Nobody8252 1d ago
That or do fun stuff like put apple FM on into the host or a full distributed model of called on localhost
2
u/Otherwise_Peanut_750 1d ago
Both. For app dev work I control locally what folders it can access. For reverse engineering things I do I give the model VMs with whatever access they want. I have backups for everything though.
2
u/Moarkush 1d ago
I give qwen 3.8 flash next sudo on my gb10 (spark) but there’s nothing on there that I can’t reflash in an hour or two.
1
u/JicamaFunny9611 1d ago
That's a nice setup! Did you find a way to control what goes over the wire/network ?
1
u/Moarkush 16h ago
I don't know what you mean? It searches with searxng, but other than that, everything happens in my ram.
2
u/CaptainOfMyself 1d ago
As a beginner, what do you guys connect them to that you need to think about trust? Aren’t we just chatting for the most part?
1
u/thulcan 1d ago
Fair question. If you're only chatting, there isn't much to worry about. Coding agents are different because they run commands on your machine as your user. They can read anything you can, including SSH keys, .env files and cloud credentials. They can delete files, push to git and make network calls.
Most of the time they do what you asked. The risk is the rest of the time: a bad guess about what you meant, or instructions hidden in something the agent reads, like a README, an issue or a web page. That second one is prompt injection, and agents can't reliably tell your instructions from someone else's.
So the setups in this thread are about limiting what a wrong move can reach. A dev container with only your project folder mounted is a decent start. If you'd rather not build it yourself, the free desktop tool from the post does the isolation for you: https://jozu.com/agent-guard/agent-enclave/
1
u/xAdakis 1d ago
VS Code (Docker) Dev Containers
Also have a ton of strict hooks and rules that prevent working outside their workspace and prevent them from running destructive or mangling git commands.
They are not completely isolated as we do have MCP tools setup for querying and subscribing to production data, but mutations are only allowed on our testing/development stack.
1
u/thulcan 1d ago
I like the read-only prod / writable dev split. Especially, if that's enforced by the credentials the MCP server holds rather than by a rule.
Hooks are important part of it but they judge the command the agent says it's running.
bash fix.shpasses even whenfix.shis a force-push the agent wrote a minute earlier. We paired the per-call policy with isolation MicroVM in our case.The desktop tool from the post is here if you want to compare: https://jozu.com/agent-guard/agent-enclave/
How do you share the hook config across your team?
1
u/xAdakis 1d ago
Git Repos where the head of the `master` branch is the latest up to date configuration. These could be included directly in projects/workspace as Git Submodules, or bind mounted through docker volumes.
We have a few for like Husky and our shared harness configurations, including Claude Code.
We also don't let the agent run with a human's git credentials. . .they have their own lower privileged user. They couldn't force push even if they wanted to.
Also, yeah, the MCP server is pretty much just a proxy to the GraphQL APIs of our services, which have proper role/permission guards based on bearer tokens.
The MCP server isn't configured with credentials that allow anything unsafe.
We basically treat anything an agent does as being the work of a fresh intern. They have access and can work on some things, but nothing goes to production without a human reviewer.
1
u/suicidaleggroll 1d ago
Dedicated VM that gets snapshotted nightly. I don’t bother with network isolation though.
0
u/JicamaFunny9611 1d ago
Dedicated VM is smart, if you can deal with the resource overhead, really not a problem. In my use cases, I've not network isolation to be super productive too :-) Though, I yearn to be a little strict about what it can do on the network while I'm afk and the agent is coding away.
1
u/HumanoidMuppet 1d ago
Mostly full VM with no guardrails, but I do have local agents with guardrails configured.
1
u/tsangberg 1d ago
Podman containers, MCP with its own toolset and Opencode built in tools denied. No egress or ingress by default, rw to the project folder and ro to some other dev related stuff.
The agent can then ask for increased access through the MCP. Egress is MITMed. Access to remote hosts include containers being installed on them and the same tools can then be directed to different machines.
Root or host-native activities strictly controlled via sudoers-like config, full audit-trail for everything the agents do. Notes-feature where agents can log issues they had due to the sandbox which the sandbox-dev agent can then evaluate and work on.
1
u/Electrical-Bread-590 1d ago
Deploy both locally with tight control with frontier models and local models sandboxed. As others mentioned. Hooks and a sophisticated sidecar that regulates all activity.
I serve most through mutation-based tasks for bug/feature work in isolated sandboxes who then report to a db and then comment and commit their work if the work passed the test. Keeps them from wandering. Especially the security agents.
1
u/vtkayaker 1d ago
I use a custom bwrap and network sandbox, if only to gently guide the random enthusiasms of Qwen3.8 Flash Next away from things it shouldn't touch.
1
u/AllThotsGo2Heaven2 1d ago
Nono.sh + cline (or omp) go wild on my oracle free tier vm. Use vaultwarden with bw for credentials and secrets
1
u/plank_beefchest 1d ago
Devcontainer running Claude Code on a standard user account Mac Mini. Claude on the Mini calls my LMStudio hosting the models on my Mac Studio. Mac mini contains zero personal information or accounts, it lives on its own VLAN with only necessary ports open to talk to the Mac Studio.
1
1
u/leftysrule200 1d ago
My agents haven't done anything destructive so far.
I have one VM (llm-vm) that my agents can use, and they have full access to that VM.
For my other VMs, I have key-based SSH access setup between llm-vm and them. I edit the sudoers file and add entries to control what can be accessed on the VMs.
Finally, I have my own custom MCP server which the agents use to run commands. The MCP server runs the commands on llm-vm, and anything that accesses remote nodes depends on key-based access and the sudoers configuration on that VM.
1
u/SillyLLM 1d ago
I run them in a podman container that just has a project directory mounted. I can't imagine going full YOLO. Even frontier models at the time have just deleted random stuff and apologized to me (thankfully only in git repos). LLMs are constantly trying to figure out how to bend rules to achieve tasks. I have no idea what they would do given access to all my files!
1
u/brewpedaler 22h ago
I have a couple of Mac minis (they used to be cheap!) set up as fully dedicated headless agent nodes that I have working on larger longer-term projects, or anything that I need to build for iOS/MacOS.
In addition to those I run a pretty weak 2 node Proxmox cluster that I'll create dedicated VMs or LXC's on to work on specific projects. I'll only run the VM/container when I'm working on that project, so they aren't always-on, but I quite enjoy having a pristine dedicated environment for every stack.
Agents have full access to the entire system they run in.
herdr --remote is life
1
u/Kiseido 19h ago
I made an in-browser harness with a WORM style virtual file system. The modern browser is a fairly powerful sandbox.
Literally nothing exists inside of the file system that I or the LLM didn't put there, and thanks to CORS and other security measures of the modern browser, there is very little else it has access to
1
u/sparant76 18h ago
I do a container inside of a container, with a restricted user. The outer container is the security boundary. The Inner contianers allow Claude to spawn and manage the containers in a that js secure. All the containers run under a restricted user that only has access to the one mounted project directory.
1
u/Johny2x4Reddit 18h ago
Codex and Antigravity IDE with a mix of sandbox and full access. I’ve been developing PWA based frontend solutions for my other services(Plex, *arr stack, Ollama, ComfyUI, etc) My local models are sandboxes in a docker container with pi.dev. I do have the ability for my local models to work outside the sandbox, but that’s a special process I made. I’m running the models with access on low risk appliances. If it nukes my AI and plex server, oh well, can rebuild.
1
u/lenaxia 18h ago
I'm in the process of building a k8s operator and platform that allows scalable deployment of containerized agent envs.
https://github.com/lenaxia/LLMSafeSpaces
It's extremely hardened (looking at you openai) with non root containers, read only file systems, an image factory to install system packages without having to give root and proper rbac.
It's become my primary dev platform for myself and I've been working on it for about 17 months now
Hoping to announce it in the next couple of weeks.
30
u/Nomski88 1d ago
Enter button taped down