Who reviews the security fix the bot wrote for you?

A patch showed up in a repo. It looked fine. It was generated, not written, and the pipeline that applied it had credentials for a great deal more than the patch needed.

That is roughly how the Snowflake Jira compromise Wiz wrote up this week started. Not with a clever exploit. With a plausible-looking change and a permission set nobody had ever read out loud.

The easy reading is "AI wrote bad code, therefore don't let AI write code". I think that reading is wrong, and it lets everyone off the hook.

The model did its job

The model produced a plausible patch. That is what it is for. Plausible patches are also what human contributors submit — most of them, most of the time, are fine.

What failed was structural, and it has nothing to do with machine learning.

A generated suggestion carried the same trust as a reviewed change. The pipeline that applied it held credentials far broader than the change required. Both of those would have been true if the patch had arrived from a stranger on the internet — which, for security purposes, is exactly what it was.

Nobody would hand a first-time contributor deploy keys because their diff looked tidy. We did it here because the diff came from inside the building.

Blast radius is the number nobody reports

Every organisation adopting agentic tooling makes the same quiet trade.

You grant the agent permissions sized to the hardest thing it might ever need to do. Then you invoke it, forty times a day, for the easiest thing it does. The distance between those two is your blast radius, and almost nobody measures it.

(Illustrative. The point is the shape: one identity, one permission set, sized for the worst case and spent on the routine one.)

Open a pull request, merge a pull request, deploy the result — three very different levels of consequence. Most setups collapse all three into a single grant, because splitting them is work and nothing forces the issue until something like this happens.

Here is the version worth arguing about in your own repo:

If the right-hand column is the same for every row in your setup too, you do not have a permission model. You have one key and a lot of doors.

A control nobody can describe is not a control

There is a second requirement that even careful teams skip: the boundary has to be describable by the people relying on it.

Ask your team what your CI agent is permitted to touch. Not by opening the settings page — from memory, in a sentence. If nobody can answer, the permission set is not doing the work you think it is. It is decoration that happens to be enforced.

The question is not whether the AI should have write access. It is the smallest unit of consequence this actor can produce without a human in the loop.

And this is where it stops being a security story and becomes our kind of story. The knowledge of what the automation may do lives in one person's head — usually whoever configured it. When they move teams, the organisation keeps the capability and loses the understanding of its limits. That is the worst of both, and it is a learning failure, not a tooling one.

What to actually do on Monday

Treat AI-authored changes as untrusted input that happens to be well formatted. Review them on the same footing as a contribution from an unknown outside contributor. The fact that the text was produced inside your own tooling is not provenance.

Then do the unglamorous part: write down, in one paragraph a new joiner can read, what each automated actor may do and what it may not. If that paragraph is hard to write, you have just found the problem before it found you.

The mechanics of how these injections actually work are worth twenty minutes if this is new to you:

Wiz's write-up is here, and it is worth reading in full — the timeline is more mundane than you would expect, which is the point.