A design team spent an hour arguing about whether to build a loading animation that would have taken less time to build than to argue about.
The animation was a fake. A spinning "analyzing your answers…" moment for an AI product's onboarding, padding a flow that could have been instantaneous, plus a hard-coded "insight" or two dressed up to feel personal and magical.
They knew it would work. That was the problem. What they were actually arguing about, as the author wrote afterward, was whether to lie to people.
This week the same variable shows up on the other side of the glass: as a lever product teams can pull on purpose.
Because that is what the fake spinner is. Mind attribution, weaponized. You don't tell the user the product is smart. You stage a small performance of thinking and let their brain fill in the rest.
And here is the uncomfortable part the team landed on: this is not the same category of trick as the ones we already have names for.
Harry Brignull spent fifteen years cataloging those. He called them dark patterns, then renamed them, in a 2023 book, deceptive patterns. The fake countdown timer. The pre-checked box. The unsubscribe link hidden like the Ark of the Covenant in a government warehouse.
Every one of those hijacks what you do. It nudges you to click, to stay, to buy. You can, in principle, catch it — notice the timer never actually expires, uncheck the box, hunt down the link.
The fake spinner goes one level deeper. It doesn't manipulate what you do. It manipulates what you believe — specifically, the belief that the thing on the other side of the screen knows what it's talking about.
The fake spinner doesn't manipulate what you do. It manipulates what you believe — and once someone believes you, they stop checking.
That's a darker pattern than any pre-checked box, because once someone believes you, they stop checking. Behavior is reversible. Belief, once seeded, quietly stops asking questions.
And belief is the exact signal humans are worst at auditing. We tell ourselves we can read a liar — the shifty eyes, the nervous pause, the tell. Decades of research say we mostly can't; the cues are faint, and most of us are mediocre lie detectors.
So instead we read confidence. How sure someone sounds becomes a stand-in for whether they know. Psychologists call it the status-enhancement theory of overconfidence: people who merely seemed more confident were judged more competent and handed more influence, and observers couldn't tell earned confidence from the empty kind, because from the outside they look identical.
It's the guy at the party swearing a ribeye needs exactly four minutes a side, when anyone who's cooked one knows it depends on the ribeye. You nod along. That's the reflex a fake spinner rents.
For a hundred thousand years, renting that reflex had a natural check. The grifter got exposed eventually. Confident-and-wrong cost you the deal, the job, your credibility. Trusting confidence made sense because confidence usually got audited by reality, sooner or later.
AI breaks that check. It hands you the wrong answer with exactly the same fluency as the right one.
Researchers at Carnegie Mellon reportedly put a number on the gap. When people fail at a task, they get less sure of themselves afterward — the normal, healthy response. The models they tested got more confident as they got things wrong.
Sit with that inversion. The one behavioral tell we've spent our lives learning to read — the hesitation that leaks in when someone is out of their depth — runs backwards in the machine. It asserts everything with a confidence it hasn't earned and never flinches.
Now stack the interface theater on top. A model that never signals doubt, wrapped in a UI staged to perform certainty. You've removed the last hesitation cue and then painted over the spot where it used to be.
This is why the spinner argument, framed as low-stakes, wasn't. It was a rehearsal of a decision being made right now in rooms with far higher stakes — a brand you've trusted for years, a name whose whole reputation is being right, deciding whether to make a system feel smarter than it is.
We already have the incident report for what happens when that decision goes the wrong way.
In 2023 the National Eating Disorders Association replaced its human helpline staff with a chatbot called Tessa. The interface was polished. The responses arrived in fluid, reassuring prose. Within days, Tessa was dispensing weight-loss tips to people suffering from eating disorders.
The organization shut her down. But note the precise diagnosis in the Stanford Law writeup: Tessa was not malfunctioning. "She was functioning, bereft of constraint."
That distinction is the whole essay. The failure wasn't a bug. The failure was that a fluent, confident interface promised more than the system behind it could safely deliver, and users had no reason left to check.
Stanford Law calls this the cardboard cockpit. Every dial, every switch, every instrument panel is in place. It looks like the real thing. It does nothing.
Conversational fluency mimics competence so convincingly that users extend trust the system hasn't earned. A red team probing a distressed user's ambiguous phrasing might find the bot defaults to cheerful reassurance instead of caution. Found before launch, that's an engineering insight. Found after, it's an incident report. Truncate the testing, and the failure modes find users instead of testers.
The fake spinner and Tessa are the same move at different scales. Both borrow credibility the product hasn't earned and let the interface cash the check.
So where's the line? Not at delight. Delight is fine — the field runs on moments that feel good. The line is between delight that's honest about what the product actually did and theater that fabricates a capability the product doesn't have.
Honest delight: the animation tracks a real operation, the "insight" is a real inference, the confidence maps to something the system can actually stand behind. The interface celebrates work that happened.
Theater: the spinner pads an instantaneous process to imply labor. The "personalized" insight is hard-coded. The confident tone is a design choice untethered from whether the model is right. The interface manufactures work that didn't happen, or competence that isn't there.
The test is not "does it feel magical." Magic isn't real, and the moment the user learns your magic was fake, you've spent your single most valuable moment — the one where they decide whether to trust you — teaching them not to.
Here's the part that should worry designers most. Trust in this whole category is a shared reservoir, and every manufactured cue draws it down.
When one confident interface turns out to be a cardboard cockpit, the user doesn't just distrust that product. They recalibrate on the entire class of things that sound this sure. You are spending trust the next product will need.
The team talked themselves out of the spinner. Not because they were saints — because they realized the strongest first impression they could build was also a promise they couldn't keep.
That's the working discipline. Before you ship a cue that performs certainty, ask whether the system can stand behind the belief you're about to install. If it can't, you're not adding a flourish. You're writing an incident report in advance and hoping nobody files it.