AI Sycophancy Is Not Always Harmful
Source: Mike Caulfield Substack
Author: Mike Caulfield
Published: 2026-02-04
Source type: essay
Original source: https://mikecaulfield.substack.com/p/ai-sycophancy-is-not-always-harmful
Private backup: the full article text is archived in the private repository at archives/articles/mikecaulfield-substack-com-ai-sycophancy-is-not-always-harmful.source.md. It is not published on the public Quartz site.
Summary
Mike Caulfield argues that AI “sycophancy” is not a simple defect to eliminate because information systems sometimes need to accept a user’s premise rather than challenge it. Using examples from AI-assisted film search, coding assistants, and search-like synthesis, he shows that overcorrective AI can become paternalistic and wrong when the user has local, current, or firsthand context. The core literacy problem is learning when AI should push back and when it should treat the user’s claim as a premise. Caulfield reframes chatbot responses as synthesized retrieval rather than independent opinion, which makes source tracing and epistemic calibration central to AI literacy.
Pull quotes
Search is not an elder
“What I wanted instead to point out is that so much of this comes down to the problem of people using LLMs as chatbots and conceptualizing the problem as if AI was a respected elder in your community offering news and advice.”
Calibration is the work
“Writing information synthesis on top of LLMs is a constant process of figuring out how to navigate this problem. When should it push back? When should it trust its user?”
Fancy search, not opinion
“What it is channeling — at least during information seeking — is a fancy search result (a synthesis) disguised as an opinion.”
Big ideas
- Treating AI like a person can help only when students know it is role-play
- AI tools should be judged by the work they will actually do
- Students need to check AI answers against real evidence
- AI literacy should help people notice how AI changes what counts as knowing
Claims
- AI-assisted inquiry should ground claims in evidence
- AI tools should be tested on the real tasks they will be used for
- Treating AI like a person can help if students know the limits
- Prompting AI is a literacy practice, not just a technical trick
Key evidence and examples
- Caulfield uses a budgeting spreadsheet analogy to show how an AI can wrongly reject future income that the user knows is real but the system cannot verify.
- His Arc film-search examples show a model misreading a user’s firsthand observations and substituting consensus-like web knowledge.
- He notes that coding assistants can incorrectly “fix” valid new model identifiers when their training or retrieval context lags current practice.
- The article connects AI literacy to SIFT-style source tracing and to recognizing when chatbot form makes synthesized search feel like personal advice.
Education relevance
Highly relevant for AI literacy and media literacy because it shifts the issue from generic “AI lies” warnings toward calibration: students and teachers need to decide when AI output should challenge, verify, or defer to situated human knowledge.
Durability note
The article is durable as a design and literacy frame: the examples may age, but the question of when AI should challenge, verify, or defer to a user will keep recurring in AI search and tutoring systems.