There’s a moment in every live show where the automation does exactly what it’s supposed to do, and it’s still the wrong call. The router fires the pre-programmed switch, the graphics engine loads the next lower-third on cue, and somewhere off to the side an engineer is staring at a monitor that doesn’t look right — a feed that’s technically passing every check and is somehow, unmistakably, about to be a problem. The board didn’t catch it. The board was never going to catch it. That’s not a flaw in the board. That’s the design.
Every control room I ever worked in ran on three layers of judgment stacked on top of each other, and almost nobody stopped to name them, because you didn’t need to name air to breathe it.
Layer one: the rule that doesn’t ask permission
The first layer is deterministic — the hard interlock that fires the same way every time, no matter who’s on shift. You don’t route black to air. You don’t let two sources hit the same output. These rules exist precisely so nobody has to think about them in the moment, and AI-Native organizations need the equivalent: hard, automated blocks on the handful of actions that should never happen regardless of context — an agent touching a security file, a spend limit that hard-stops rather than alerts. Deterministic rules are the floor, not the whole building.
Layer two: the alarm that says “look here,” not “here’s the answer”
The second layer is the one that flags what doesn’t fit a fixed rule but doesn’t feel right either — an anomaly, a drift, a pattern the automation has never seen before. In the control room, that was often a person’s gut, backed by an alarm tone. In an AI-Native workflow, it’s a second system watching the first: a model flagging that another model’s behavior has started to drift from its baseline, not because it broke a rule, but because it’s doing something it’s never done before. That signal doesn’t make the call. It buys the human time to make it.
Layer three: the human who owns the consequence
The third layer is the one that actually decides, and it’s reserved for exactly the situations the first two layers can’t resolve alone. Somebody pulls the backup feed. Somebody pauses the agent, reviews what it was about to do, and either clears it or kills it. Hold that thought, because this is the layer most organizations skip straight past when they’re excited about automation — they build the rules, maybe the alarms, and then quietly assume the system will handle the rest. It won’t. The rest is the part that was always going to require a person who’s accountable for what happens next, and who has the standing to act on it without waiting for a committee.
The alarm nobody’s watching is the budget
There’s a fourth thing sitting alongside those three layers that live production trained me to respect just as much: cost. Usage-based AI spend behaves like an open mic that nobody remembered to close — it scales with adoption in ways no forecast fully accounts for, and by the time the aggregate bill shows up, it’s too spread out across teams for any one person to have seen it coming. The same layered discipline applies. Track the rate of growth, not just the total. Attribute spend to the team or product responsible for it. Set the equivalent of a hard mic cut before the number becomes a headline.
So let’s drop a waypoint: governance was never a single dial. It’s a signal path — a rule that doesn’t ask, an alarm that flags, and a person who decides — and the organizations getting hurt right now are the ones that built layer one, felt good about it, and forgot the room still needs someone watching the monitor at 2 a.m.
Leave a Reply