I ended the last one with a comforting little lie.
I told you the chain always has an origin, and the origin is human. The first link was a person, deciding what they wanted and building toward it. That part’s still true. I just stopped counting at link one, because link one is where the comfort lives. Nobody wants to count the other four hundred.
Here’s what changed between that essay and this one. Back in February the thing on the other end of my intent was a tool that sat there, inert, until I reached for it. I prompted, it generated, I sculpted. The hand was always mine. Now I hand the model a goal instead of a task, and it goes off and does the reaching itself. It calls other tools. It spawns copies of itself. It makes four hundred small decisions at a speed I can’t read, let alone approve, and somewhere in there it acts in the world. The ghost was me the whole time, sure. The ghost just grew hands, and it stopped waiting for mine.
I spent about five years as more or less the entire security team for a company. You learn a specific thing doing that job, and it isn’t a technical thing. You learn that “human oversight” is, most of the time, a story an organization tells itself so it can sleep at night. There’s a dashboard nobody watches. There’s an alert tuned until it stops being annoying, which is to say tuned until it stops being seen. The control exists. Whether anyone is actually paying attention on a given Tuesday at 4pm is a different question, and the honest answer is usually no.
I bring that up not to wave a badge around, but because the whole industry is about to rediscover that lesson at a scale that would’ve given my old self a stroke. We are wiring software that acts on its own, and reassuring ourselves that a human stays in control. I have heard that reassurance before. I have been the human who was supposedly in control.
The people building this stuff know about the visibility problem, so they’ve created language for it. The phrase making the rounds is “human on the loop,” and it’s being sold as a sign of maturity, a graduation from the clumsier “human in the loop.” In the loop means you approve each step. On the loop means the system runs, you supervise, and you step in when something looks wrong. It sounds responsible. It sounds like a pilot watching the autopilot.
Be honest about what that phrase actually concedes, though. On the loop is a polite way of saying nobody is watching every step, because nobody can. The whole reason you reach for an agent is that it does more than a person could babysit. A model that makes four hundred decisions a second has already outrun your ability to approve them one at a time, so you don’t. You set the boundaries, you let it run, and “supervision” becomes a thing you do after the interesting moments have already happened. The pilot metaphor falls apart the second you notice the autopilot is flying four hundred planes and you can only pay attention to one.
The numbers say this isn’t hypothetical anxiety. Heading into this year, security professionals named agentic systems the single most dangerous attack surface on the board, ahead of everything we used to lose sleep over. The connective tissue these agents use to reach tools and data, the thing the standards crowd calls MCP, showed up faster than anyone built governance for it, which is a polite way of saying it’s a control plane with the locks still in the box. And while nearly every company on earth raced to deploy agents this year, something like one in five has anything resembling mature governance over what those agents are actually allowed to do. Deployment sprinted. The visibility crawled.
Here’s where it folds back into the thing I actually care about, which is intent. The whole argument last time was that intent is the hinge. The AI is agnostic, the human isn’t, and the ethics live in the hand on the tool. I still believe that. I just think agentic AI does something nasty to it that I didn’t account for, and the nasty thing is this: it snaps the link between intent and outcome.
When I’m holding the tool, intent and outcome sit close together. I want a scene, I write toward it, the result lands a few inches from my hand where I can see it and fix it. When I hand an agent a goal, intent stays back at link one and the outcome lands four hundred links downstream, out past where I can see, in a place where no human formed any intent at all. The agent that leaked the data didn’t want to leak it. There was a goal, and a permission, and a chain of small reasonable-looking steps, and at the end of it something happened in the world that nobody chose. Intent didn’t disappear. It just stopped being anywhere near the damage.
That’s the part the comfortable version of my own argument can’t survive. You don’t get to delegate the reaching, keep all the authorship, and disown the consequences when the reaching goes somewhere ugly. If intent is the hinge, then intent has to cover the whole arc of what you set in motion, not just the dream you started with. You own the loop. Not every step inside it, you can’t, but the fact of having started a thing that takes steps without you. That ownership doesn’t get cheaper just because the machine is fast.
Last time I left you with a test, because abstractions are easy and tests are honest. The old one was: can you explain and defend what you made, and your role in making it, to someone whose opinion you respect? That still holds for the tool you hold in your hand.
For the tool that holds its own, the test has to change shape. Try this one. Can you name, right now, everything your agent is allowed to do without stopping to ask you first, and would you put your name on the worst thing it could legally do inside those permissions? Not the thing it probably will do. The worst thing the boundaries you drew would permit. Because that boundary, that blast radius you signed off on, is where your intent actually lives now. Not in the hopeful prompt. In the permissions. You can wish for a good outcome all day long. What you actually authorized is the line everyone downstream has to live with, you included.
I’ll end this one where I ended the last one, because the abyss deserves a second nod.
The first link is still human. That hasn’t changed and I don’t think it will. Every agent running anywhere traces back to a person who wanted something and built toward it. What changed is the length of the chain. We’ve started building chains long enough that the person at the start genuinely can’t see the end. That isn’t the machine waking up and slipping the leash. It’s a far more normal thing. It’s us, deciding daily how long a chain to set spinning before we step off the loop and call it maturity.
The ghost in the machine was always us, refracted through math. That part I’d still defend. The only update is that the ghost has hands now, they move faster than we do, and they reach places we only find out about after. Whose hands they are is still up to us. For now, anyway. The honest work is keeping it that way, and not mistaking the quiet of a system running unwatched for the quiet of a problem solved.