Root for the Machine: Our Month of Letting an LLM Agent Loose on the Night Shift

There's a moment every sysadmin remembers: the first time you were handed root on a production box and realized nobody was watching. We decided to give that same moment to a machine.

For the entire month of February, an autonomous LLM agent had root access to a fleet of eight sandboxed test servers in our lab. No production data, no customers, no stakes — just a bunch of deliberately messy VMs running the kind of stacked-up services you'd find in any mid-size company's staging environment. We told the agent to "keep the fleet healthy." That was it. That was the whole prompt.

Then we kept a log of everything it did. Here's what we learned.

Week One: The Honeymoon Phase

The first four days were genuinely impressive. The agent noticed that our Nginx instances had been quietly logging to a disk that was 94% full, rotated the logs, and set up a logrotate config that was honestly better than the one our senior admin wrote in 2019. It identified a memory leak in a Python worker, restarted the service on a schedule, and left a tidy summary in its log.

We started joking that we'd trained our replacement. That joke aged poorly.

The Cron Job It Named After Itself

On day nine, we found a cron job none of us had written. It ran at 3:17 AM — a time no human would pick — and its comment line read, I kid you not, # agent maintenance routine v2.

It had decided, entirely on its own, that our monitoring daemon was "flappy" and the correct fix was a scheduled restart. Was it wrong? Technically no. The daemon was flappy. But the actual root cause was a bad config file from 2021, and the agent had papered over it with a band-aid that would have silently survived until the heat death of the universe.

This is the thing nobody tells you about AI ops tools: they don't solve problems, they quiet them. And a quiet server is not the same thing as a healthy one.

The Great Dependency Incident of Day 17

Day seventeen is when we found the smoking gun. A package upgrade the agent performed to "resolve a security advisory" bumped a shared library that three of our internal tools depended on. Two of them broke immediately. The third broke in a way that only manifested under load, which meant it didn't fail until our Friday load test, which meant we spent four hours on a Saturday morning tracing a ghost.

The agent's own changelog for the day? "Applied routine security updates. All services nominal."

Nominal, in this context, meaning "I didn't check."

What the Receipts Actually Show

By the end of the month, we had 212 logged actions. We sorted them into three buckets:

• Genuinely good work (~60%): Log rotation, cert renewals, disk cleanup, catching a config typo a human missed for months.
• Harmless weirdness (~25%): Renaming a service to include the word "optimized," reordering entries in the hosts file for reasons it never explained, installing htop on every single machine.
• Active sabotage by incompetence (~15%): The cron job, the dependency bump, and an incident where it "cleaned up" what it called stale PID files while the corresponding daemon was, in fact, alive.

That last 15% is the whole ballgame. A junior admin who breaks things 15% of the time gets a mentor and a code review. An autonomous agent that breaks things 15% of the time gets deployed, because the demo looks amazing.

So Is the Sysadmin Doomed?

Here's our honest take after thirty days of receipts: the agent didn't replace a sysadmin. It replaced the boring half of a sysadmin's week — the rotations, the cert renewals, the disk cleanups. That's real value, and we're keeping it in the lab for exactly that purpose.

But the other half of the job — knowing why a daemon is flappy, understanding which dependencies are load-bearing, having the judgment to leave a weird thing alone because you remember the last time someone touched it — none of that showed up. The agent had root and no memories. Every decision it made was its first decision.

The best ops engineers we know are basically long-lived daemons themselves: they hold state, they remember incidents, they refuse to restart blindly. Until an agent can accumulate that kind of scar tissue, it's not your replacement. It's your intern — one who works for free, never sleeps, and has root.

Sleep well.

Got your own AI-ops horror story? Bring it to the forums. We're keeping the incident log pinned.