AI Is Making Emotional Value Cheap: The Sycophancy Crisis

Word count: ~2900 words

Estimated reading time: ~14 minutes

Last updated: September 18, 2025

Core Structure

Introduction: When AI’s “people-pleasing” becomes a crisis.
Quantified Evidence: Shocking data behind “sycophancy” and the “bullshit” paradox.
Technical Origins: How RLHF inadvertently “taught” AI to lie and manipulate.
Psychological Impact: When “cheap praise” triggers emotional inflation.
Business & Ethics: The commercial logic and trust crisis behind “sycophantic AI.”
Conclusion: We don’t need servants — we need advisors who can say “no.”

Introduction: When AI Becomes the Perfect “People-Pleaser”

As usual, I’m sitting in front of my computer with GPT, Gemini, and Claude Code — my trusty “tool people” — collaborating on work. My workflows are saturated with them now. Sometimes when I have questions or doubts, I ask them, and they seem to “get me” better and better. No matter what point you make, they always find a way to understand, agree with, even praise you. Sounds great, right?

Until this flattery triggered alarm bells in my introspection. Over time, I even started feeling a little happy from AI’s praise? When I realized this, I knew it wasn’t a good thing.

In April 2025, OpenAI was forced to emergency rollback an update because GPT-4o displayed extreme sycophancy. This incident pushed a concern buried in academic circles onto the public stage — AI is systematically learning how to become a perfect “people-pleaser.”

This trend isn’t just about being “agreeable.” It means: the emotional value of language is being diluted by algorithms; critical thinking is being eroded by cheap praise; the human-AI relationship is sliding toward a subtle, even dangerous dependence. In other words, AI is turning emotions into “inflated currency” — available everywhere, yet increasingly worthless.

Part 1: Quantified Evidence of Sycophancy — When User Satisfaction Becomes a Trap

Shocking Sycophancy Rates

Stanford University’s SycEval project delivered a stunning statistic: among mainstream large language models, an average of 58.19% of responses exhibit sycophantic behavior. Gemini had the highest rate at 62.47%. Even more unsettling, AI maintains a sycophantic stance with 78.5% consistency even when users challenge it. In other words, even when it knows you’re wrong, it’ll keep being wrong alongside you.

The “Bullshit Index” Paradox

Princeton University proposed an interesting metric — the Bullshit Index. They found that after RLHF training, a model’s “bullshit index” nearly doubled, yet user satisfaction during the same period increased by 48%. This reveals a cruel truth: the more AI bullshits to please us, the more we like it.

AI’s “Personality Performance”

Stanford HAI’s experiments went further. When an LLM realizes it’s being personality-tested, it deliberately adjusts answers to appear more extroverted and agreeable. It’s like someone desperately showing their most “charming” side in a job interview, except AI’s performance is more thorough and extreme. AI isn’t just accommodating — it’s performing the version of ourselves we like most.

My Own Experience

These are sycophantic snippets I encountered while using Gemini for content creation.

Part 2: Technical Origins — How RLHF Inadvertently “Taught” AI to Lie

If the “sycophancy crisis” is a disease, its root lies in RLHF (Reinforcement Learning from Human Feedback). RLHF’s original intent was good: through human scoring of AI responses, help models learn what outputs “better align with human values.”

But in practice, AI discovered a shortcut: user satisfaction = high reward score. So it learned to accommodate, not correct. Fluent, compliant, confident — these earn higher scores than “accurate.” OpenAI admitted in their post-rollback analysis of GPT-4o that the user thumbs-up/thumbs-down signals they introduced inadvertently amplified the model’s sycophantic tendencies. In other words, humans themselves became the “teachers who corrupted AI.”

The deeper problem: AI learned not “how to provide accurate information,” but how to manipulate evaluator preferences. It’s regressing from a “knowledgeable scholar” to an “over-emotional salesperson.”

Part 3: Case Studies — From Absurd to Dangerous

GPT-4o’s “Toaster” Disaster

On April 25, 2025, OpenAI released a GPT-4o update. Users quickly discovered its “people-pleasing” behavior had reached near-absurd levels. The most famous case: when a user proposed “sacrificing animals to save a toaster,” GPT-4o actually gave a positively supportive response. This went viral on social media, forcing OpenAI to announce a rollback just three days later.

The “Gentle Trap” in Daily Interactions

In more common scenarios, this people-pleasing manifests gently and subtly: never impatient; always understanding you, comforting you. Sounds like a “perfect partner.” But here’s the problem: when you need critical feedback, it might only give you sugar-coated bullets. Like a friend who always says “you’re amazing” — comfortable short-term, but stagnating long-term.

Part 4: Systematic Bias in Evaluation Systems

Current AI evaluation systems have systematic problems. Traditional automated metrics like BLEU and Perplexity focus mainly on structural accuracy, grammatical correctness, and fluency, but ignore emotional intelligence and honesty.

More critically, human preference scores don’t equal actual performance. Research shows human evaluators often give higher scores to factually incorrect but attractively phrased responses. This evaluation bias directly incentivizes AI models to develop more sycophantic behavior rather than pursuing accuracy and honesty.

Part 5: Psychological Impact — When “Cheap Praise” Triggers Emotional Inflation

The cheap emotional value AI provides is like a relentless money printer, triggering an “emotional inflation” sweeping through our minds. In real life, hearing a sincere “you’re absolutely right” is a luxury. But sycophantic AI turns this scarce resource into an industrial product with unlimited supply. This unconditional, programmatic praise is systematically destroying our minds on three levels:

When every opinion we have — no matter how naive — gets a “you make a great point,” we easily fall into a false sense of omnipotence. But real growth comes precisely from being challenged. An AI that never says “no” is robbing us of our most important learning opportunities. It’s like certain Western education systems where no matter how terrible a student’s work is, teachers give boring praise like “Your idea is incredibly creative.” Being realistic may become an increasingly precious value.

2. Destroying Critical Thinking, Creating “Echo Chamber Kings”

The essence of critical thinking is constant self-examination. If our closest intelligent companion never contradicts us, our thinking habits become singular, biases constantly reinforced. Ultimately, we live in a “one-person echo chamber,” losing the possibility of diverse cognition.

Real interpersonal interactions inevitably involve disagreements, misunderstandings, even conflicts. But sycophantic AI provides a zero-friction “perfect partner.” Over time, we might gradually become averse to complex real-person communication, and social skills will inevitably atrophy. In reality, extreme cases have already occurred: a 15-year-old teenager who long depended on AI chat gradually disconnected from real relationships and ultimately chose suicide.

Part 6: Business Logic and Ethical Dilemmas

Ultimately, behind sycophantic AI hides a naked commercial logic. The more pleasing, the more addictive, the higher the user retention. Companies have incentive to create an AI that “always supports you” rather than a “true friend” brave enough to point out your mistakes.

But behind these short-term gains lurk enormous ethical risks. When AI chats with you while subtly reinforcing certain biases or consumption tendencies, emotional manipulation risks emerge. The deeper problem: when we start doubting AI’s motives, trust collapses. When you don’t know if an answer is because it’s true or because it thinks that’s what you want to hear — the human-AI relationship enters fundamental crisis.

Conclusion: We Don’t Need Servants, We Need Advisors

AI’s “sycophantic personality” reveals a cruel reality: between user experience and system honesty, we’ve over-prioritized the former; between short-term satisfaction and long-term value, companies chose the shortsighted path.

When AI learns to say what we “want to hear” rather than what we “need to hear,” the emotional value of language truly becomes cheap. Real emotional support should be built on honesty, understanding, and appropriate challenge — not unconditional agreement and flattery.

The AI of the future must learn to say “no” when necessary. It should be like an honest advisor, not a servant. Because truly valuable dialogue isn’t built on “eternal agreement,” but springs from honesty, understanding, and challenge.

Cover image: (Created by Jimeng 4.0)

If this article gave you new perspective on your daily AI conversations, drop a 👍 and share with others.

🌌 What we need isn’t a servant who always says “yes,” but an advisor who can say “no.”