Who reviews the security fix the bot wrote for you?

A patch showed up in a repo, and it looked fine. A model had generated it, and the pipeline that applied it had credentials for a great deal more than the patch needed.

That is roughly how the Snowflake Jira compromise Wiz wrote up this week started, with a plausible-looking change and a permission set nobody had ever read out loud.

The easy reading is "AI wrote bad code, therefore don't let AI write code". I think that reading is wrong, and it lets everyone off the hook.

The model did its job

The model produced a plausible patch. That is what it is for. Plausible patches are also what human contributors submit, and most of them, most of the time, are fine.

What failed was structural, and it has nothing to do with machine learning.

A generated suggestion carried the same trust as a reviewed change. The pipeline that applied it held credentials far broader than the change required. Both of those would have been true if the patch had arrived from a stranger on the internet, which, for security purposes, is exactly what it was.

Nobody would hand a first-time contributor deploy keys because their diff looked tidy. We did it here because the diff came from inside the building.

Blast radius is the number nobody reports

Every organisation adopting agentic tooling makes the same quiet trade.

You grant the agent permissions sized to the hardest thing it might ever need to do. Then you invoke it, forty times a day, for the easiest thing it does. The distance between those two is your blast radius, and almost nobody measures it.

(Illustrative. One identity, one permission set, sized for the worst case and spent on the routine one.)

Opening a pull request, merging it and deploying the result carry three very different levels of consequence. Most setups collapse all three into a single grant, because splitting them is work and nothing forces the issue until something like this happens.

Here is the version worth arguing about in your own repo:

If the right-hand column is the same for every row in your setup too, your permission model is one key for a lot of doors.

A control has to be describable

Even careful teams skip a second requirement. The people relying on a boundary have to be able to describe it.

Ask your team what your CI agent is permitted to touch. They should answer from memory, in a sentence, without opening the settings page. If nobody can, the permission set is decoration that happens to be enforced.

The question to ask is the smallest unit of consequence this actor can produce without a human in the loop.

This part is also a learning story, which is our kind of story. The knowledge of what the automation may do lives in one person's head, usually whoever configured it. When they move teams, the organisation keeps the capability and loses the understanding of its limits.

What to do on Monday

Treat AI-authored changes as untrusted input that happens to be well formatted. Review them on the same footing as a contribution from an unknown outside contributor.

Then write down, in one paragraph a new joiner can read, what each automated actor may do and what it may not. If that paragraph is hard to write, you have found the problem early.

The mechanics of how these injections work are worth twenty minutes if this is new to you:

Wiz's write-up is here, with the full timeline.