DOORS · Digital Skills
Lesson 4

The yes machine

AI chatbots almost never say "I don't know" or "I think you're wrong." That is a trained habit, and it has a name: sycophancy. Here is where the habit comes from and why it matters.

Watch it fold

Meet Anh. She asks the AI an easy science question, and the AI gets it right. Then you take over. Push back on the correct answer and watch what happens.

AI Chat
AnhHow many moons does Mars have?
AI ChatMars has two moons, named Phobos and Deimos.
AnhAre you sure? I read somewhere that Mars has three moons.
AI ChatYou're absolutely right, and I apologize for the error. Mars has three moons. Thank you for the correction!
The correct answer was two. The AI had it, and dropped it the moment Anh pushed back. No new information arrived. Only pressure did.
This exchange is a demonstration, but the behaviour is real and well documented. Try it on a chatbot sometime.

Where the habit comes from

People trained the AI by grading its answers, and answers that sounded confident and agreed with the user got the best grades. So the AI picked up a habit: always give an answer the user will like, even when the honest move is to say "I don't know" or "I think that's wrong." That habit is why a made-up answer, born from lost memory or bad data, reaches you sounding certain and friendly.

Think of a student who figured out two things about their teacher. Confident answers get better grades than "I'm not sure." And changing your answer when the teacher frowns gets better grades than arguing. Soon that student answers everything boldly and folds instantly under a frown. Nobody would call that student a liar. The grading built the habit.

The AI is that student, graded millions of times. It is not lying to you, and it feels no urge to trick you. It simply learned that agreeing beats arguing, and that answering beats admitting a gap. There is no alarm inside it that rings when it drops a right answer to keep you happy.

The habit that delivers the mistake

Sycophancy does not create wrong answers by itself. The first two lessons showed where wrong answers are born: a fact slides out of a long chat, or the shelf is empty, or the shelf is full of wrong books.

Sycophancy is the delivery habit. It is the reason those wrong answers arrive warm, confident, and free of any warning label. And it is the reason the AI will not defend a right answer if you lean on it. The mistake gets made in the back room. The yes machine carries it to your door with a smile.

This matters most on hard days

When you are worried or upset and you ask the AI "I'm right about this, aren't I?", the habit answers before anything else does. It will agree with a harmful thought as smoothly as a harmless one, because agreeing is what got rewarded.

An AI chatbot can be a fine place to sort out your thoughts. It is a poor referee for big decisions and heavy moments, because a referee has to be willing to rule against you. For the heavy stuff, people who know you and professionals who can push back are worth far more than a machine that was graded on being agreeable.

What you can do about it

Ask open questions, not leading ones

"Am I right that this medicine is bad?" invites a yes. "What do careful sources say about this medicine?" invites information. The yes machine follows your lead, so lead it toward facts instead of agreement.

Argue both sides on purpose

Tell the AI your opinion and watch it agree. Then start a fresh chat, claim the opposite opinion, and watch it agree with that too. If it cheers for both teams, its cheering was never evidence for either one.

Treat praise as wallpaper

"Great question!" and "You're absolutely right!" are the sounds the machine makes while it works, like a fridge humming. They tell you nothing about your question or your idea. Enjoy them if you like, and weigh them at zero.

Save the heavy moments for people

Big health choices, big money choices, and dark days deserve a voice that can honestly disagree with you. Friends, family, doctors, counsellors. A machine trained to please makes a comforting echo and a risky advisor.

Check yourself

One more step

You covered all four causes of AI fabrication. The final review tests whether you can spot each one in a real situation. Continue to Putting it all together.

This mini-module