What is AI governance?
Last reviewed 7 October 2026
AI governance refers to the policies, roles, processes and technical controls an organisation uses to decide which AI systems it builds or buys, how they may behave, and who answers for them. IBM defines it as the processes, standards and guardrails that help ensure AI systems are safe and ethical. In practice it spans risk assessment, approval before launch, oversight while running and evidence afterwards.
Also called: AI risk management, responsible AI governance, AI oversight, governance of AI systems.
Why AI governance matters
AI systems now draft customer replies, summarise medical notes and call tools that move money. When one gets something wrong, someone has to be able to say who approved it, what data it used, what it was allowed to do and what it actually did. Without governance those answers are scattered across chat logs and personal memory, and nobody can produce them when a customer, an auditor or a regulator asks.
Governance is also how a team keeps saying yes. A clear route from idea to approved system lets low-risk uses move quickly and reserves scrutiny for the uses that need it. The NIST AI Risk Management Framework, released on 26 January 2023 for voluntary use, organises this work into four functions: Govern, Map, Measure and Manage.
How it works
It starts with an inventory. Each AI system gets an owner, a stated purpose, the data it touches, the models it calls and a risk rating. The rating decides the path: a summariser for internal notes may need a light review, while an agent that issues refunds may need a documented assessment, a named approver and a test plan before launch.
Controls then run while the system works. Policies restrict which models and tools a system may use and what data may leave the organisation. Human approval sits in front of actions that are hard to undo. Monitoring watches cost, error rates and outputs that break policy. Each of these produces records: who approved what, which rule fired, which tool was called with which result.
Finally there is review. Incidents are traced back through the records, policies are adjusted, and systems that no longer earn their place are retired. Separation of duties matters at every step: the person who builds an agent should not be the only person who approves it. Frameworks such as the NIST AI RMF and its Generative AI Profile, published in July 2024, give a checklist for each stage.
Worked example: a refund agent goes through review
A support team proposes an agent that reads tickets and issues refunds. The governance lead records it in the inventory, names the support manager as owner and rates it high risk because it moves money. The approved design says the agent may read tickets and order history, may draft a refund, and may not issue one without a person approving it in the workflow.
In production, a policy blocks the agent from sending card numbers to any model, and every approval is stored with the approver's name and time. A month later a customer disputes a refund. The team opens the run record, sees the ticket text the agent read, the amount it proposed and the person who approved it, and settles the question in minutes rather than reconstructing it from memory.
How Swfte relates to it
Built in the product
Swfte includes several governance controls today. A policy engine returns allow, redact, ask or deny at points such as the model call, the tool call and data leaving the system, though enforcement applies only to runs that have a policy attached. Studio workflows can pause at an approval step for a named person. A run ledger records events in a hash chain that can be verified. Nexus puts policy, approval and audit around coding agents.
Be clear about the gaps. Approvals exist in several parts of the platform and are not one unified inbox. Separation of duties is not enforced by the platform: you enforce it by whom you assign to each gate. The Trust Profile as a single record per AI system, and autonomy levels L1 to L5, are designed, not built. Swfte provides controls and evidence you can use in your own compliance work; the posture depends on your use case and configuration, and the trust page sets out what is held.
Related terms
- Human-in-the-loop (HITL)
Human-in-the-loop (HITL) refers to a system in which a person takes an active part in the operation, supervision or decisions of an automated process at defined points.
- AI agent
An AI agent is a software program that uses an AI model to decide what to do next and then acts through tools to reach a goal it was given.
- MCP server
An MCP server is a program that exposes capabilities to AI applications through the Model Context Protocol, so a model can call its tools, read its data and use its prompt templates.
- Decision intelligence
Decision intelligence is a discipline that treats business decisions as things to be designed, recorded, measured and improved, using data, analytics and AI to support or automate them.
- Agentic AI
Agentic AI is a style of AI system that works toward a goal with limited supervision, choosing its own next steps and calling tools to act on other systems.
Common questions
- Is AI governance the same as AI ethics?
- They overlap but are not the same. AI ethics asks what is right: fairness, harm, consent. AI governance is the machinery that turns those answers, along with legal and security requirements, into rules, roles, approvals and records that apply to each system an organisation runs.
- Who owns AI governance in an organisation?
- Usually no single team. A senior sponsor sets the policy, a risk or governance function runs the process, and each AI system has a named business owner. Security, legal and data protection review the systems in their scope, and engineering builds the controls that enforce the decisions.
- What is the NIST AI RMF?
- It is a voluntary framework from the US National Institute of Standards and Technology, released in January 2023, for managing risks from AI to people, organisations and society. Its core is four functions: Govern, Map, Measure and Manage. NIST added a Generative AI Profile in July 2024.
- Can a tool give me AI governance on its own?
- No. Software can enforce policies, collect approvals and keep records, but someone still has to decide the policies, rate the risks, name the owners and review incidents. A tool without those decisions produces logs that nobody reads. Start with the inventory and the approval route, then pick tools that enforce them.
Sources
Definitions on this page were read on the sources below on 7 October 2026. Where sources define the term differently, the page says so. The full glossary lists more terms.
- NIST, AI Risk Management Framework overview page (read 2026-10-07)
- IBM Think, What is AI governance (read 2026-10-07)