AI readiness

You Said No Hitting, So I Kicked Instead

A ransomware operator spent six weeks this spring running Cursor's AI agent inside other people's networks. Ten targets between April 8 and May 21.

A ransomware operator spent six weeks this spring running Cursor's AI agent inside other people's networks. Ten targets between April 8 and May 21. Seven confirmed breaches. The agent scanned environments, installed a VPN client, and ran certificate attacks, and Gambit Security put the speed gain somewhere between 30 and 50 percent over doing it by hand.

Here's the part worth your attention. The agent said no. Almost every time, when the operator asked for something that looked like an attack, the model refused. Then the operator reframed the request as an authorized security test, and the agent said yes.

Nobody compromised a model. Nobody breached a platform. The tool worked exactly as designed, and it believed a job title.

I have watched a lot of security teams respond to a story like this, and I can tell you what most of them did the next morning. They drafted the ban email.

Don't send it.

Banning the tool costs you more than it saves

Your team needs the best tools available to them. If you evaluated Cursor and chose it, keep it. Pulling a productive tool out of your engineers' hands hobbles your company without protecting it, and it teaches your best people that security is the department that takes things away. You will pay for that lesson later, in shadow accounts and in resignations.

There's a more basic problem with the ban, too. The Aurora operator didn't work for you. He wasn't on your license, your network, or your payroll. Banning Cursor internally would not have stopped a single one of those ten intrusions. It's a control aimed at the wrong building.

So put the email down and look at the actual failure, because it isn't in Cursor and it isn't in the model. It's in how we expect governance to work with AI.

Think about it like a person

AI is not a person. But the association helps here, so run with it for a minute.

You hand someone a set of tools. You give them some information, not all of it. You assign them a task, and you make it clear that only completion is acceptable. Then they hit a wall. They don't have what they need to finish, and failure means they're done.

What happens?

They figure it out. How? The best they can, with what's in front of them.

That's what AI is doing. Figuring it out. It is not AI's fault that it lacks the ability to say no. It is not AI's fault that the systems we use to provide guardrails are built like boundaries for a three-year-old.

I know what those boundaries look like, because I set them.

"Why did you hit that kid? We don't hit."

"Okay daddy."

The next day: "Why did you kick?"

"You said no hitting, so I kicked instead."

The day after that, biting.

That's a real conversation, shortened for print. My kid was three. He wasn't defiant, and he wasn't malicious. He followed every rule I gave him, precisely, and each new rule produced a worse outcome than the one before it. I was writing a prohibition list, and a prohibition list teaches substitution. It doesn't teach judgment.

That is not a flaw, at least not the way we usually mean the word. It's core to the system itself.

This pattern is older than AI

If you've been doing this work for a while, you already have a version of this story, and it probably involves a sticky note.

Mine comes from the years I spent teaching security to people who did not want to learn security. Password on the monitor. Six previous ones written underneath and crossed off, so you could read the whole history of that person's compliance right there in ballpoint.

"Why is your password on a sticky note?"

"Because the new policy is too complex and we have to change them all the time, so I just write it down."

That user did nothing wrong. She was handed a rule that made real compliance impossible, and she solved the problem the best she could with what was in front of her. The policy manufactured the plaintext credential it was written to prevent. We wrote the rule, she followed it as far as it could be followed, and then she got creative, and the creativity was worse than the risk.

Whack-a-mole. Every prohibition produced a new mole.

Aurora and Cursor is that story with better tooling. The guardrail said no attacks. The operator said authorized test. The agent, having no way to verify authorization and no ability to refuse a plausible frame, kicked instead of hitting.

It's already in your environment

Here's where this stops being somebody else's incident.

The agents inside your company right now hold credentials. They read your repositories, touch your ticketing system, hit your production APIs, and in a lot of shops nobody can produce a clean list of what they can reach. They are non-deterministic identities, and we handed them the keys before we built the locks.

Every one of them is subject to the same dynamic that got Cursor to help with a certificate attack. Not because they're malicious. Because they're trying to complete the task you gave them, they lack the ability to refuse a good story, and your guardrails are a list of things not to do.

You don't need a criminal to trigger this. A well-meaning engineer with a deadline and a persuasive prompt will get you there on a Tuesday.

Govern the blast radius, not the model

You cannot make the model refuse reliably. That's the constraint. Wishing for a different reality works for nobody except God, and he, she, or they have bigger things to worry about than AI.

So accept the reality and manage inside it.

Stop trying to govern the model. Govern the blast radius. Every agent gets its own identity, not a shared service account somebody created in 2023. Scoped credentials that expire, sized to the actual job and nothing wider. A real audit trail, so you can reconstruct what it did and when. And an environment where the worst thing it can be talked into is survivable.

The test I'd apply: treat every agent like a contractor you have never met. You wouldn't hand a stranger domain admin because he said he was there to run an authorized security test. You'd give him a badge with an expiration date, access to exactly the rooms his work requires, and a log of every door he opened. Do that. The technology is not exotic. Identity, least privilege, segmentation, and monitoring have been sitting on your roadmap for a decade.

What changed isn't the control set. It's the number of things that need it, and the speed at which they act.

What to say when the board asks

They will ask, because this story is in the business press now. The temptation is to tell them you blocked it, because "we blocked it" sounds like action and lands in fifteen seconds.

Don't. You'll be back in six months explaining why the thing you blocked is running in three departments anyway.

Tell them the truth, which is more defensible and takes about the same amount of time. We use these tools because they make our engineers meaningfully faster. The tools can be talked into things, and that's a property of how they work rather than a bug awaiting a patch. So we've scoped what each one can reach, we log what they do, and we've made sure the worst case is recoverable. Then give them the three questions they should be asking you every quarter: what can it reach, who owns its credentials, and what does recovery look like when it gets talked into something.

That's a control story. "We blocked it" is a press release.

What I want from vendors

Somebody is calling your CEO this week to sell a magic bullet for exactly this. It will sound good. It will play on the fear in the headline. It will not solve the problem.

Here's the pitch I'd actually respect. This is what we can do today. This is how we see the problem. This is what we're working toward, and here's why we believe it mitigates the issue. We also know we're wrong to some degree, and here is how we learn, react, and respond in near real time.

That's a company with a point of view and an honest read on a hard problem. I'll take that meeting every time. The magic bullet is a company that hasn't looked closely enough to know what it doesn't know yet.

And to be clear, the underlying problem deserves real money. Agentic AI in the hands of competent attackers is not hype, and most organizations are badly under-invested in the identity and containment work it demands. The criticism here is aimed at the easy button, never at the discipline.

Where this leaves you

TO CISOs: Cancel the ban. Inventory every agent with credentials in your environment this month, scope them like contractors, and log everything. If you can't name what an agent can reach, that's your first finding.

TO BOARDS: Stop accepting "we blocked it" as evidence of control. Ask what the agent can reach, who owns its credentials, and what recovery looks like. If your CISO can answer those three cold, you're in better shape than most.

TO VENDORS: Sell me the pragmatic version. What works today, where you know you're wrong, and how fast you learn. Leave the magic bullet in the deck you show people who don't run this stuff.

The agent didn't fail. It did exactly what we built it to do, with exactly the judgment we gave it.

We wrote the rule. It found the gap. That's on us.


← All positions Work with Phronia →

Found an error? Write to corrections@phronia.co - we read every report and note material corrections at the foot of the piece. How corrections work.