By Dr. Kevin Shepherdson, CEO and Founder, Straits Interactive
In Part I, I argued that Agentic AI is fundamentally changing the relationship between humans and machines. We explored why the familiar governance principle—"You can delegate the task, but not the responsibility"—has never been more relevant.
As autonomous AI agents begin planning, reasoning, calling tools and executing increasingly complex workflows, many organisations are making a dangerous assumption: that delegating execution somehow transfers accountability. It does not.
We also examined several emerging misconceptions surrounding Agentic AI, including:
1. Why successful demonstrations often create a false sense of organisational readiness—a phenomenon I described as CEO Psychosis.
2. Why jobs are more than a collection of tasks, and how confusing task automation with job replacement is already leading some organisations to rethink their AI strategies.
3. Why the statement "The AI decided" oversimplifies reality, when autonomous behaviour is ultimately shaped by human choices—system prompts, workflows, guardrails, permissions and governance decisions.
If Part I was about understanding the governance paradox of Agentic AI, Part II shifts the conversation towards a more important question:
As AI becomes increasingly commoditised, what will actually differentiate organisations, professionals and leaders?
My answer may surprise you. It isn't the latest AI model. It isn't the most autonomous agent. And it isn't even the organisation with the biggest AI budget.
The real competitive advantage will belong to those who can combine deep domain expertise, AI capability and responsible governance to create lasting business value.
The Next Competitive Advantage Isn't Technology
Another misconception is that technology will determine the winners. I disagree. Technology is rapidly becoming commoditised.
Today's leading AI platform may be overtaken tomorrow. New models are released every few months. Today's breakthrough quickly becomes tomorrow's standard feature. The real competitive advantage lies elsewhere.
It lies in:
1. Domain expertise
2. Institutional knowledge
3. Trusted content
4. Proprietary data
5. Business ecosystems
6. Governance capability
Anyone can ask an AI agent to build an HR application.
Far fewer people understand recruitment law, employment regulations, organisational culture and the subtle biases that influence hiring decisions.
While technology can manufacture software, it cannot manufacture decades of domain expertise. That remains a uniquely human asset.
This belief has shaped much of my work over the past few years. My mission has never been to teach everyone how to become programmers or AI engineers. Instead, it is to enable the 80% of professionals who are not technologists—educators, HR practitioners, marketers, lawyers, accountants, healthcare professionals, compliance officers, and countless other subject matter experts—to transform their knowledge and experience into AI-enabled capabilities.
For too long, software development was largely the domain of programmers. Agentic AI and no-code platforms are beginning to change that equation. They allow subject matter experts to participate directly in creating AI applications, AI assistants and intelligent workflows without writing thousands of lines of code.
However, democratising software development does not diminish the importance of domain expertise—it elevates it.
The most valuable AI applications of the future will not be created by those who simply know how to use the latest AI tools. They will be built by people who deeply understand the problems they are solving.
As technology becomes increasingly accessible, the winners will not necessarily be those with the best technology—they will be those who combine deep domain expertise with strong AI capability and responsible governance.
Three Risks We Are Sleepwalking Towards
This concern is not theoretical.
I was recently honoured to serve as a judge at a Google Gemini Hackathon. The experience was inspiring because it showed the creativity, ambition and speed with which teams can now build AI-enabled applications.
But it also revealed something important.
During the judging process, a number of demos either could not run properly or failed midway because teams were using trial editions, preview access or limited token allocations. In some cases, the tokens had simply run out.
At one level, this was understandable. Hackathons are experimental environments, and teams often work with limited resources. But at another level, it highlighted a deeper issue that will become far more serious in enterprise environments: many builders are still treating AI cost as an afterthought.
They want the magic of autonomous AI. They want agents that plan, reason, call tools, generate outputs and retry failed steps. But they may not yet fully appreciate that every layer of autonomy consumes resources, increases complexity and introduces risk.
This observation connects directly to three risks I believe organisations are sleepwalking towards.
1. Cheapest Model Syndrome
Many organisations optimise for token cost rather than reasoning quality. They assign planning, reasoning and tool selection to the cheapest available model.
The savings appear attractive. The operational risk often remains invisible—until something goes wrong.
The hackathon demos that failed because of trial limits or exhausted tokens were a useful reminder that AI capability is not free. If a solution depends on a powerful model during the demo but is later downgraded to a cheaper model in production, the quality of reasoning, tool selection and decision-making may change significantly.
Cost optimisation is necessary. But careless cost optimisation can become a governance risk.
2. Testing Becomes a Cost Instead of an Investment
Agentic systems are frequently presented as self-correcting. This creates a dangerous temptation.
If the AI can retry, reflect and improve, perhaps we can reduce testing. The opposite is true.
Autonomous systems require more validation, not less. Edge cases, adversarial inputs, governance testing and scenario analysis become increasingly important as autonomy increases.
When teams are unwilling or unable to spend on sufficient tokens even to demonstrate the system reliably, it raises an uncomfortable question: will they be willing to spend enough to test it properly?
Testing autonomous AI systems is not just about checking whether the output looks impressive. It requires repeated testing across scenarios, contexts, failures, exceptions and misuse cases.
That costs money. But not testing costs more.
3. Autonomous Token Burning
An autonomous agent does not simply answer a question.
It plans, reasons, calls tools, creates subtasks, retries failures. requests additional information, and repeats the cycle.
Every autonomous loop consumes resources while simultaneously increasing the number of decisions made without direct human observation.
This is why token exhaustion during demos is more than a technical inconvenience. It is an early signal of what happens when autonomous systems are allowed to operate without proper cost governance, stopping rules, retry limits and escalation logic.
Greater autonomy delivers greater productivity. It also creates greater opportunity for unexpected outcomes.
The lesson is clear: organisations cannot govern Agentic AI only by looking at the final output. They must also govern the economics, testing discipline and autonomous behaviour that sit behind the output.
When the Investigation Begins
Imagine an AI-powered recruitment platform integrated with an organisation's Applicant Tracking System.
An autonomous agent generates a job advertisement. The advertisement unintentionally discriminates against older applicants. A complaint is lodged. The regulator arrives. What happens next?
Most organisations instinctively examine the final advertisement. That is only the beginning. A proper investigation retraces the entire AI system lifecycle.
Investigators may ask:
1. What business objective was originally defined?
2. What goal was given to the autonomous agent?
3. Which system prompt governed its behaviour?
4. Which model performed the reasoning?
5. Which workflow orchestrated the task?
6. Which external tools were invoked?
7. What organisational policies were available to the agent?
8. Which version of the prompt was deployed?
9. What guardrails were configured?
10. Which human approvals existed?
11. What logs were captured?
12. Who authorised deployment?
Suddenly the investigation is no longer about one discriminatory advertisement. It becomes an investigation into the entire governance architecture that produced it.
This is where concepts such as traceability, explainability, provenance, lineage and auditability move from academic discussions to operational necessities.
Without them, organisations may struggle to answer the most fundamental question: "Why did the agent do what it did?"
It is to address such risks that Singapore introduces….
The Governance Paradox
The AI industry often tells us that autonomous agents reduce the need for human involvement.
I believe the opposite is true. The more autonomous AI becomes, the more important governance becomes.
Not because we distrust AI. But because responsibility has never left the organisation. It has simply moved further away from execution.
This is the governance paradox of Agentic AI.
We can delegate execution. We can delegate planning. We can even delegate reasoning. But we can never delegate accountability.
A Prediction
The first generation of AI incidents was dominated by hallucinations, misinformation and inaccurate outputs.
The second generation expanded to privacy breaches, copyright disputes, deepfakes, etc.
I believe the third generation will look very different. It will be characterised by autonomous agents making thousands of individually reasonable decisions that collectively produce legal, operational, financial and ethical failures.
These failures will not occur because AI suddenly became malicious. They will occur because organisations confuse autonomous execution with autonomous responsibility. The organisations that thrive in the coming decade will not necessarily be those with the smartest agents.
They will be those that understand where autonomy creates value, where humans must remain accountable, and how to govern the collaboration between both.
Because in the age of Agentic AI, one principle remains unchanged. You can delegate the task. You can never delegate the responsibility.
This is Part II of a two-part series. Read “Part I: The Governance Paradox of Agentic AI” here.
