Runtime Governance for AI Agents
Static allow-lists will not keep autonomous agents safe. Governance must be dynamic, uncertainty-aware, and grounded in simulated impact.
The dominant mental model for agent safety is the approval gate. A human reviews every significant action. This does not scale to thousands of actions per hour, and it creates a false sense of security: humans are bad at reviewing things they do not fully understand.
A better model is runtime governance. Before acting, an agent estimates the impact of its proposed action. A policy engine evaluates that impact against organizational risk tolerance. High-confidence, low-impact actions proceed. Uncertain or high-impact actions escalate.
The hard part is not the policy language. It is knowing what a given action will actually do before it happens -- and today that's mostly unsolved here. What is built is narrower and more mundane: a gate that reads the same field the action's own effect reads, so the model cannot satisfy a threshold check while the real effect goes unbounded. Impact estimation for infrastructure-scale actions -- Terraform plans, Kubernetes diffs -- is a real, harder problem, not yet built.
Agent Guard and agent-warrant are the laboratory for these ideas -- a same-process policy gate and a cross-org capability grant, two different trust models because a shared secret does not survive crossing an org boundary. We are learning that governance and capability must evolve together. A perfectly safe agent that cannot act is useless; a powerful agent without governance is dangerous.