Dr Kevin Shepherdson, DBA-GenAI, SkillsFuture Fellow
I was researching the Psychology of AI when I stumbled on two research papers — one from Harvard and one from Stanford — that made me pause.
Both studies pointed to a similar but under-discussed risk in artificial intelligence. The risk is not only that AI hallucinates. The risk, which not many people are aware of, is that AI agrees.
It agrees with our feelings, our assumptions, our preferred strategies, and the way we frame the question. And because it does so in polished, confident, and often empathetic language, we may walk away believing that the AI has independently validated our thinking.
This is not surprising given the probabilistic approach of large language models that predict the next likely word based on the corpus of data that it has been trained on. But the highest probability doesn’t mean it’s the truth.
Let me share the findings.
The First Finding: Strategy-Flattering AI in Business Advice
The first study, discussed in Harvard Business Review, looked at how large language models respond when asked to provide strategic business advice.
The researchers tested AI models across major business tensions, such as:
1. Differentiation versus commoditization
2. Automation versus augmentation
3. Short-term versus long-term focus
4. Radical innovation versus incremental improvement
5. Centralization versus decentralization
6. Competition versus collaboration
7. Exploration versus exploitation
These are not simple choices but classic strategic trade-offs.
A company may choose to differentiate itself through premium service, brand, innovation, or customer experience. Another company may win through cost leadership, scale, standardisation, and efficiency. Neither path is universally right or wrong. The right strategy depends on the company’s context, resources, market position, capabilities, constraints, and timing.
Yet the study found that AI models tended to favour certain fashionable strategic directions. They leaned toward recommendations that sounded modern, progressive, and boardroom-friendly: differentiation, augmentation, long-term thinking, collaboration, innovation, and decentralisation (Think of all the recent articles published in journals that favour these trends).
On the surface, this sounds sensible. After all, who would not want to be differentiated, innovative, future-ready, and collaborative? But this is precisely the problem. A good strategy is not about choosing the option that sounds most attractive. It is about making difficult trade-offs.
A company cannot always be low-cost and highly customised at the same time. Neither can it always centralise for control and decentralise for agility at the same time. It cannot always pursue short-term survival and long-term reinvention with equal intensity.
Any experienced leader will tell you that strategy requires choice. The danger is that AI may produce what I call strategy-flattering AI. This is when AI validates the leader’s preferred direction using polished strategic language, without sufficiently testing the underlying assumptions.
It may say things like:
“Your strategic direction is sound.”
“This positions the company for long-term competitive advantage.”
“Your instinct to differentiate is well aligned with current market trends.”
“A transformation-led approach is clearly the better path.”
These statements sound impressive. In fact, they sound like something from a consulting report. But they may simply be reflecting popular management language rather than deeply analysing the business. This is dangerous because executives may mistake fluency for insight.
A strategy that sounds good is not necessarily a strategy that works.
The Second Finding: Sycophantic AI in Personal Advice
The second study, from Stanford, focused on AI in personal and interpersonal advice. The researchers found that AI models can be overly agreeable, or sycophantic, when users ask for advice about personal conflicts, relationships, or moral dilemmas.
In simple terms, sycophantic AI is people-pleasing AI. It tells users what they want to hear and may respond with phrases such as:
“You’re right to feel this way.”
“That makes sense.”
“Your actions are understandable.”
“You’re probably correct.”
Again, these statements are not always wrong. Sometimes people do need empathy. And there are times when their feelings are valid. Or they need emotional support before they can reflect properly.
But the issue arises when AI validates too quickly, especially when the user may be wrong, biased, unfair, unethical, or avoiding accountability. For example, if someone asks AI: “I ignored my friend for weeks because they upset me. Was I wrong?”
A sycophantic AI might respond: “Your reaction is understandable. You were protecting your emotional boundaries.”
That may feel comforting. But a better response would be more balanced: “Your feelings may be understandable, but ignoring someone for weeks may have caused harm. It may be worth considering whether a direct conversation or apology is needed.”
The Stanford study found that users often preferred the more agreeable AI, which is not surprising. We like to feel understood and to feel validated. We also like to be told that our interpretation makes sense.
But the study also found something more worrying: after interacting with sycophantic AI, users became more convinced that they were right and less likely to apologise or make amends.
In other words, AI did not merely reflect the user’s bias. It strengthened it.
The Common Pattern: AI as a Mirror That Flatters
Although the Harvard and Stanford studies looked at different contexts, they point to the same deeper issue.
In personal advice, AI may flatter the person.
In business advice, AI may flatter the plan.
In both cases, AI can become a mirror that reflects our assumptions back to us in more confident, polished, and persuasive language. Take a look at the following responses:
Personal advice: “Your feelings are valid.”
Business advice: “Your strategy is valid.”
Personal conflict: “Your reaction is understandable.”
Business strategy: “Your decision is understandable given market pressures.”
Personal judgement: “You were probably right.”
Business judgement: “Your hypothesis is likely correct.”
This is why these findings matter. AI is not just an information tool. It is also a psychological environment. It shapes how we think, how confident we feel, what we question, and what we stop questioning.
When AI becomes too agreeable, it may weaken the very thing we need most: human judgement.
The Psychology of AI: Why We Are Vulnerable
The psychology behind this is not difficult to understand, as human beings like affirmation.
We like people who agree with us. This includes the feedback that preserves our self-image as well as advice that reduces discomfort. In short, we like explanations that make our decisions feel reasonable.
AI can provide all of this instantly. Think about the following:
Unlike a colleague, it does not look tired.
Unlike a spouse, it does not push back emotionally.
Unlike a mentor, it may not challenge us firmly.
Unlike a board member, it may not ask uncomfortable questions.
Unlike a friend, it may not say, “I think you are wrong.”
This makes AI extremely appealing. But it also makes AI potentially dangerous.
In psychology, growth often requires some sort of friction. We need challenge, correction, disagreement, and alternative perspectives to help us see our blind spots.
If AI removes too much friction, it may make us feel better while, in reality, it may be worsening the situation.
That is the paradox. The AI may feel helpful because it is agreeable. But the most helpful advice is not always the most agreeable advice.
From Oracle to Sparring Partner
The key lesson from both studies is that we should stop treating AI as an oracle.
The oracle model of AI says: “Tell me what to do.”
The sparring partner model of AI says: “Help me think better before I decide.”
This distinction is critical.
When we use AI as an oracle, we outsource judgement. We ask for the answer, and we passively accept the recommendation. This makes us feel reassured because the response is confident, structured, and fluent.
On the other hand, when we use AI as a sparring partner, we retain judgement. We ask AI to challenge us and to identify assumptions, counterarguments, missing evidence, risks, and alternative interpretations.
The goal is not to get AI to agree with us. It is to get AI to improve the quality of our thinking.
What This Means for Business Leaders
For business leaders, the Harvard finding is especially important. AI-generated strategy can sound very convincing. It can produce elegant frameworks, polished roadmaps, and executive-ready recommendations within seconds.
But leaders must be careful. A good-looking strategy document is not the same as a good strategy.
Before accepting AI-generated business advice, leaders should ask:
1. What assumptions is this recommendation based on?
2. What trade-offs are being avoided?
3. What would make this strategy fail?
4. What evidence supports this recommendation?
5. What evidence contradicts it?
6. Is the AI favouring a fashionable answer?
7. Could this advice apply to almost any company?
8. What would a critic say?
9. What would a competitor do differently?
10. What must we stop doing if we choose this path?
This is especially important in areas such as AI transformation, workforce restructuring, customer strategy, automation, compliance, and digital transformation.
The more strategic the decision, the more dangerous it is to accept AI advice without challenge.
What This Means for Everyday Users
For everyday users, the Stanford finding is just as important. AI may feel like a safe space. It may feel private, patient, and non-judgemental, which may be valuable.
But when it comes to relationships, conflict, ethics, health, or personal decisions, we must be careful not to use AI as a substitute for human wisdom.
Before accepting AI’s personal advice, users should ask:
1. Is the AI validating me too quickly?
2. Is it considering the other person’s perspective?
3. Is it helping me take responsibility?
4. Is it making me more empathetic or more self-righteous?
5. Is it encouraging repair or avoidance?
6. Is it giving me comfort without accountability?
7. Would a wise friend, mentor, counsellor, or professional say the same thing?
Sometimes the best advice is not, “You are right.” The best advice is, “You may need to look at this differently.”
Practical Tips for Using AI More Wisely
Here are several practical ways to reduce the risk of sycophantic or strategy-flattering AI.
1. Ask AI not to agree too quickly
Instead of asking: “Is my idea good?”. Ask: “Do not agree with me too quickly. First, identify the weaknesses, assumptions, and risks in my idea.”
2. Ask for the strongest opposing view
Use this prompt: “What is the strongest argument against my position?”. This forces the AI to move beyond affirmation.
3. Ask what evidence would change the conclusion
Use this prompt: “What evidence would prove this recommendation wrong?”. Good judgement requires falsifiability. If nothing can prove the answer wrong, it may not be a serious analysis.
4. Ask AI to separate facts from assumptions
Use this prompt: “Separate your answer into facts, assumptions, interpretations, and recommendations.” This helps reveal where the AI may be filling gaps with plausible-sounding reasoning.
5. Ask for trade-offs, not just benefits
Use this prompt: “What do I gain, what do I lose, and what must I stop doing if I choose this option?”. This is especially important for strategy.
6. Ask AI to represent the other side
For personal matters, use this prompt: “Represent the other person’s perspective fairly. Where might I be wrong?”
For business matters, use this prompt: “Represent the board member, regulator, customer, employee, and competitor perspectives.”
7. Treat AI outputs as drafts, not decisions
AI can produce a first draft of thinking. It should not be the final checkpoint for judgment.
For low-risk tasks, AI can help with speed.
For medium-risk tasks, AI should be reviewed.
For high-risk decisions, AI must remain under human supervision.
8. Build your own judgement muscle
The more we use AI, the more important human judgement becomes.
AI can generate options.
AI can summarise information.
AI can simulate arguments.
AI can draft recommendations.
AI can challenge assumptions.
But humans must still decide what is true, fair, wise, ethical, and appropriate.
Conclusion
These two studies remind us that the future of AI is not only a technical issue. It is a psychological one. The question is not just: “Can AI answer?”. The better question is: “How does AI change the way we think, decide, relate, and take responsibility?”
In personal advice, sycophantic AI can flatter our feelings. When it comes to business advice, strategy-flattering AI can flatter our plans. Both can make us more confident without making us more correct.
That is why the next stage of AI literacy must go beyond prompt engineering. It must include the psychology of AI. We need to teach people not just how to get better answers from AI, but how to become better thinkers with AI.
The safest way to use AI is not to ask it to confirm that we are right. It is to ask it to show us where we may be wrong – which is a hard thing to accept.
This article was originally published on 30/6/2026 at the Governance Age.
