By Dr Kevin Shepherdson, CEO and Founder of Straits Interactive
.jpeg)
If you are a GRC professional, legal adviser, compliance officer, privacy professional, risk leader, senior executive, or board member, you have probably heard this phrase many times in AI governance discussions: “Don’t worry, there is human-in-the-loop.”
It sounds reassuring. It suggests that a human is still involved, still exercising judgment, still able to prevent harm. In board meetings, governance discussions, compliance reviews, and AI project proposals, the phrase is often used as a quick answer to concerns about hallucination, bias, safety, privacy, and accountability.
But the phrase is also increasingly used too casually.
For some organisations, “human-in-the-loop” has become a memorised response. It is the politically correct answer when someone asks, “What controls do we have over the AI?” A project team may say it. A vendor may or a GRC professional may say it. Even a board paper may include it.
I see this regularly in the AI governance courses we run. At SMU Academy, many professionals sign up for the AIGP professional certification course - awarded by the International Association of Privacy Professionals (IAPP). Many are GRC, legal, compliance, risk, privacy, or audit professionals. They are highly experienced in governance, but many are not technical by training. Some may only have a basic understanding of how AI systems actually work, let alone how GenAI systems behave, how agentic workflows operate, or why human oversight can fail in practice.
This is not a criticism. It is the reality of where many organisations are today. The people expected to govern AI are often not the people who built it. Yet they are increasingly expected to assess whether AI systems are safe, explainable, accountable, and properly controlled. That is why phrases such as “human-in-the-loop” need to be unpacked carefully. If GRC professionals use the phrase without understanding what meaningful oversight requires, they may unintentionally approve controls that look reassuring on paper but fail in operation.
The Comforting Illusion of Human Oversight
Human oversight is one of the most important safeguards in AI governance. It is especially important where AI systems influence decisions affecting people, money, safety, rights, reputation, legal obligations, or access to services.
But human oversight can also create a false sense of security.
An organisation may believe it has managed AI risk simply because a person is involved. But if that person is not qualified, not informed, not given enough time, not able to see the evidence, or not empowered to stop the system, then the control may be more symbolic than real.
This is where “human-in-the-loop” can become ‘governance theatre’.
1. It looks like oversight.
2. It sounds responsible.
3. It satisfies the checklist.
But it may not actually reduce risk.
A human who merely clicks “approve” without understanding the AI output is not a safeguard. A manager who reviews hundreds of AI-generated recommendations under time pressure is not exercising meaningful judgment. A domain expert who cannot see how the AI reached its conclusion is being asked to trust, not verify. A reviewer who has no authority to override the AI is not in control.
In such cases, the human is not truly in the loop. The human is decorating the loop.
Why This Matters More in the Age of Gen AI and Agents
This issue is becoming more urgent because AI systems are no longer limited to simple predictions or recommendations.
Generative AI can draft documents, summarise meetings, generate legal arguments, write code, advise customers, screen candidates, classify complaints, analyse contracts, and produce reports. Agentic AI goes even further. AI agents can use tools, call APIs, search the web, update systems, send messages, create tickets, execute code, and trigger workflows.
When AI only produces an answer, the risk is whether a human relies on that answer incorrectly.
When AI acts, the risk becomes much larger. The question should not be: Is the AI output correct? It becomes: What is the AI allowed to do, who is supervising it, and can a human meaningfully intervene before harm occurs?
This is why human oversight must be designed carefully. It cannot be added as an afterthought.
Understanding the Different Oversight Models
Many people use the phrase “human-in-the-loop” to describe all forms of human oversight. But in AI governance, it is useful to distinguish between different oversight models.
The three common models are:
1. Human-in-the-loop
2. Human-on-the-loop
3. Human-over-the-loop
These terms may be new to some readers, so let us break them down simply.
Human-in-the-Loop: Human Approval Before Action
Human-in-the-loop means the AI does not complete the action until a human reviews and approves it. The human is part of the decision process before the outcome is final.
For example:
1. an AI drafts a legal clause, but a lawyer must approve it before it is sent;
2. an AI recommends a candidate shortlist, but a qualified HR manager must review it before interviews are arranged;
3. an AI flags a financial transaction, but a compliance officer must decide whether escalation is required;
4. an AI coding assistant proposes a code change, but a developer must review and approve the merge.
This model is most appropriate for higher-risk situations where the AI output could affect rights, safety, employment, money, legal obligations, regulatory exposure, or customer outcomes.
But even here, the phrase can be misleading. Human-in-the-loop is only meaningful if the human has the expertise, time, evidence, authority, and independence to challenge the AI.
Otherwise, it becomes a rubber-stamp process.
Human-on-the-Loop: Human Monitoring During Operation
Human-on-the-loop means the AI can act within defined boundaries, but a human monitors the system and can intervene if something goes wrong.
The human is not approving every action before it happens. Instead, the human supervises the system while it operates. For example:
1. a customer service chatbot answers routine questions, but human agents monitor escalations and unusual complaints;
2. an AI fraud monitoring system flags patterns automatically, while analysts review alerts and intervene when thresholds are triggered;
3. a logistics agent optimises routes automatically, while an operations manager monitors exceptions;
4. an AI workflow agent updates internal records within limits, while a supervisor reviews anomalies.
This model may be appropriate for medium-risk, bounded, and reversible use cases.
However, it requires strong monitoring, alerts, dashboards, logs, escalation paths, and clearly defined stop conditions. If humans are expected to monitor too many systems without meaningful visibility, human-on-the-loop becomes ineffective.
A human cannot supervise what they cannot see.
Human-over-the-Loop: Governance and System-Level Oversight
Human-over-the-loop is broader. It refers to strategic, governance-level oversight over the AI system as a whole. This is not about approving every individual output or monitoring every transaction. It is about ensuring that the AI system is designed, deployed, reviewed, and managed responsibly.
For example:
1. an AI governance committee reviews high-risk AI use cases before deployment;
2. risk and compliance teams define policies for acceptable AI use;
3. model owners monitor performance, drift, incidents, and user feedback;
4. senior leaders decide whether an AI system should be expanded, restricted, paused, or retired;
5. auditors review whether controls are working as intended.
Human-over-the-loop is essential because not every risk can be managed at the point of output. Some risks arise from design choices, data sources, vendor dependencies, model selection, system permissions, workflow integration, and long-term monitoring.
This is the level where organisations ask: Should this AI system exist in this form at all?
Human Oversight Is Not a Comfort Blanket
Human oversight remains one of the most important safeguards in AI governance. But it cannot be treated as a magic phrase that automatically neutralises AI risk.
Saying “there is human-in-the-loop” is not enough.
The real question is whether the human is qualified, informed, empowered, supported, and positioned at the right point in the workflow. If the human lacks domain expertise, does not understand AI limitations, has no access to evidence, is overwhelmed by review volume, or cannot override the system, then the oversight may be symbolic rather than meaningful.
This is especially important as organisations move from simple GenAI use to AI-powered decision support and agentic AI. When AI systems begin to influence decisions, trigger workflows, call tools, update systems, or act with some degree of autonomy, the human role must be designed with much greater care.
Part I has focused on the illusion of human oversight — why the phrase “human-in-the-loop” is often used too casually, and why the mere presence of a human does not guarantee effective control.
In Part II, we turn to the next question: What kind of human oversight is actually appropriate for different levels of AI risk? Not every AI system requires the same model of oversight. Some systems require human approval before action. Others may only require human monitoring during operation. At the governance level, organisations also need humans overseeing the entire AI system across its lifecycle.
This is where we need to distinguish between human-in-the-loop, human-on-the-loop, and human-over-the-loop and understand when each model is appropriate.
This is Part 1 of a two-part story. For Part 2, please click here.
This article was originally published on 22/7/2026 at the Governance Age.
