How to govern AI agents

You cannot govern what you cannot list. Start with a sweep: ask each team what agents, copilots and automated assistants they run, check your identity provider and cloud accounts for service identities and API keys named after tools, and look in procurement for AI features inside software you already pay for. The discovery methods are close to those in how to detect shadow AI.

Record each agent once. A useful minimum is: name, purpose in one sentence, business owner, technical owner, where it runs, which models it calls, which tools and systems it can reach, what data it reads and writes, and whether a person is involved before it acts. Give each agent an ID that will not change.

Do not wait for a perfect list. Publish the first version in a week, mark gaps as "unknown", and make the owner field mandatory. An agent with no owner is the first finding of the exercise.

Shared API keys and a human's personal token are the usual starting point and the worst place to end up. If five agents share one key, the log cannot say which acted, and revoking the key stops all five. Give each agent its own non-human identity, and issue credentials with a short life, tied to the task where you can.

Identity vendors now provide this as a feature. Microsoft's documentation for Entra Agent ID, for example, describes agent identities as separate from users, with the same kinds of adaptive access policy, risk detection and lifecycle management as users and workloads, and says all agent authentication and activity is logged for audit. Check the licensing on that page before you plan around it, since it names a separate licence requirement. Whatever you use, the principle is the same: one agent, one identity, one owner.

Link the identity to the inventory ID so a log line can be traced back to a named owner. See the page on non-human identity for more on credentials and rotation.

Turn the inventory row into a profile that a policy engine, a reviewer or an auditor can read. Swfte's Trust Profile uses these fields: identity, owner, risk level, approved models, data classification, data residency, permitted systems, allowed actions, restricted actions, human approval rule, retention, audit and policy set. You can keep the same fields in any format.

Write the "cannot" list first, because it is the part people skip. The example below is the worked example in Swfte's positioning: an AI procurement agent that can read approved supplier information, analyse contracts, compare pricing, prepare purchase recommendations and create draft purchase orders, but cannot access unrelated employee data, approve its own high-value transaction, make payments or modify restricted records. It requires approval for purchases above a threshold, contractual changes and sensitive external communications.

Set the risk level by what the agent can do, not by how clever it is. An agent that can only draft is lower risk than one that can send and pay. The risk level decides the autonomy level in step 5 and how often you review in step 8.

A profile that nobody enforces is paperwork. Guardrails ask "should this be blocked?" Governance asks a wider set: who is acting, allowed to do what, under which policy, with which data, using which model, at what risk and with what oversight, and can we prove it. Make the decision explicit, with six outcomes: allow, deny, warn, filter, escalate, or require human approval.

Put the enforcement point where actions happen: in the tool layer or gateway that sits between the agent and your systems, not in the prompt. A prompt can be argued with. A tool call that fails an authorisation check cannot. Read the permitted systems and allowed actions from the profile, deny anything not listed, and require approval where the profile says so.

The OWASP guidance on excessive agency says the same: implement authorisation in downstream systems rather than relying on the LLM to decide whether an action is allowed, and use human-in-the-loop control for high-impact actions. Keep rules small and testable. Each rule should say who, what, on which resource, under which condition, and what the decision is. Change them through review like code, and record which policy version applied to each action.

Autonomy is a dial, not a switch. Swfte's positioning uses five levels of controlled autonomy: L1 Assist, where the AI recommends; L2 Approve, where a human approves; L3 Supervise, where it acts within limits and is monitored; L4 Autonomous, where it acts independently within strict policy and risk bounds; and L5 Adaptive, where it improves within controlled boundaries. You can use your own scale. What matters is that each agent has a written level.

Start low. A new agent runs at L1 or L2 until it has a record. Write the promotion criteria: for example, a defined number of reviewed runs with no policy violations, an eval pass above an agreed bar, and a named owner's sign-off. Also write the demotion rule: any serious incident drops the agent a level until the cause is fixed.

Match level to risk. A read-only research agent can reasonably sit at L3. An agent that moves money should stay at L2 for the actions that spend. The detail of designing the approval step is in how to set up human approval for AI agents.

Evidence is what turns a claim of control into proof. For every action record: agent identity, the user or trigger that started the run, data accessed, model used, output, tools called, policy applied (with version), the decision, any approval and who gave it, the action taken and the outcome. That is the record Swfte's positioning lists for the procurement example, and it is a sound general minimum.

Store logs where the agent cannot edit them, keep them for a period set by your retention policy, and make them searchable by agent ID, user and time. Redact or tokenise personal data in logs where you can, and keep access to the logs itself to a short list.

Check that the trace is complete by picking a real outcome and walking it backward: what did the agent do, why was it allowed, and who approved it? If any link is missing, fix the logging before you scale the agent.

Decide in advance how an agent is stopped. The mechanisms are plain: revoke its credentials, disable its entry in the tool gateway, pause its schedule, and cut its network route. Make at least two of these work without a code deployment, and make sure the on-call person knows where they are.

Separate the stop levels: pause one agent, pause a class of agents (all agents that can send email, say), pause everything. Write who may trigger each. After a stop, an agent should not restart without its owner confirming and the cause being recorded.

Run a drill. Once a quarter, pick an agent in a test environment, trigger the stop, and time it. Include the steps that people forget: queued jobs and scheduled retries that restart the agent a few minutes later.

Set a review cadence from the risk level: higher-risk agents more often. At each review the owner looks at the sampled runs, policy violations and denials, approvals and overrides, incidents, cost, and whether the profile still matches what the agent does. Update the profile and policy in the same session. Retire agents nobody owns.

If you need to show your approach against a framework, two are widely used. The NIST AI Risk Management Framework, which NIST describes as intended for voluntary use, is organised around four functions: Govern, Map, Measure and Manage. The ISO/IEC 42001:2023 standard specifies requirements for an artificial intelligence management system and was published in December 2023. Map your steps to them: inventory and profiles fit Map; enforcement and logs fit Manage and Measure; ownership, policy and review fit Govern.

Mapping is a convenience for people who read the standard, not a claim that you comply with it. Certification or conformity assessment is a separate process. For the audit side, see how to audit AI systems.