The Shallows, Confirmed: Multi-Study Evidence That LLM Reliance Produces Weaker Understanding
Over 10,000 people sat down to learn something—how to grow a vegetable garden, how to lead a healthier life—and the ones who asked ChatGPT came away with advice that was, in the researchers' own words, "sparser, less original, and ultimately less likely to be adopted."
There. That is the sentence. It will be screenshotted, quoted in keynote decks, and appended to a thousand posts about how the machines are hollowing us out.
And it is exactly the sentence I want to talk you out of reading that way.
Last week in these pages we made the cultural case—the argument that deep reading is a discipline being crowded out by faster, shallower habits. This week the argument gets a spine of data. Shiri Melumad and Jin Ho Yun ran seven experiments, n = 10,462, published in PNAS Nexus this October, and the finding held with unusual stubbornness.
But the interesting thing about their study is not that it confirms the shallows. It is where the confirmation refuses to land.
The obvious reading is the alarmist one: LLMs make you dumber, so use them less. The study does not say that. It says something narrower and, once you sit with it, considerably more uncomfortable.
Start with the control the researchers built to protect themselves against the cheapest objection. The worry is always that ChatGPT users simply see less information—that Google surfaces more facts, more angles.
So Melumad and Yun fed both groups identical sets of facts through simulated interfaces. Same information, different delivery. The gap survived. Knowledge from the synthesis was shallower than knowledge assembled from links, even when the raw material was word-for-word the same.
That already tells you the problem is not access. It is what you do—or don't do—with what you have.
Then comes the detail that should end the whole "just add sources" conversation, the one everyone reaches for when they want to keep the convenience and lose the guilt. In one experiment (the "ChatGPT with links" condition), the researchers reportedly gave participants a version of the chatbot that offered real-time web links to original sources, right there beside the synthesis.
Reportedly, nothing meaningful changed. The knowledge stayed shallow.
And here is the number that does the real work: of the roughly 800 participants handed those links, only about a quarter reportedly clicked even one.
Sit with that ratio. Three out of four people, offered the exact door back into the effortful process the researchers had identified as the source of deeper learning, declined to open it.
This is why the "bolt sources onto the summary" fix is not a fix. It treats the missing ingredient as information, when the missing ingredient is an act.
The act is synthesis. In traditional search you navigate links, discard some, reconcile others, and build—clumsily, effortfully—a representation that is yours because you assembled it. The LLM performs that assembly and hands you the finished object. What it removes is not the facts. It is the doing.
The LLM performs that assembly and hands you the finished object. What it removes is not the facts. It is the doing.
The word the researchers use for the doing is friction. And their own links experiment reportedly suggests that friction is not packaging you can restore later. Once the synthesis exists, the motivation to reconstruct it collapses. Who reads four sources to rebuild an answer already sitting in front of them, coherent and complete?
Nobody. Or one in four of us, on a good day.
This is where Daniel Oppenheimer, the Carnegie Mellon psychologist, sharpens the finding into something the headline misses. He would reframe it. It is not quite right, he says, to claim that people who synthesize for themselves gain deeper understanding than people handed a synthesis.
More accurate, in his phrasing: "LLMs reduce motivation for people to do their own thinking."
That is the less comforting version, and I think the correct one. The pessimistic story—LLMs erode our capacity to think—lets us off the hook, because capacity sounds like something being done to us. The motivation story implicates us directly.
The tool does not disable the faculty. It removes the reason to exercise it. And we, reliably, take the exit. One in four click; three in four don't.
For a concrete portrait of what gets lost, leave the lab and read the coder.
A software engineer went six months without ChatGPT or Copilot—an experiment in his own dependence. Before, AI had become "a reflex": need a parser, ask; strange exception, ask; rename a method, apparently also ask.
Nothing looked wrong. Pull requests merged. The code worked, sometimes cleaner than what he'd have written.
Then this: "A few days later, I often could not explain why a certain solution was built that way. I remembered the task, the final code, and maybe the prompt. The reasoning in between was missing."
The reasoning in between was missing. That is the shallows made local and precise—not ignorance of the answer, but absence of the path to it.
When he wrote a retry mechanism by hand, he had to decide what "robust" actually meant. Should every error be retried? What happens when the request body can't be replayed? Can two retry loops multiply the load on a sick server?
The code took longer. "But the understanding stayed." Every line had a reason, because he had been the one forced to supply it.
That is friction doing its designed work. The forty-minute task that took eighty was not inefficiency. The extra forty minutes were the learning.
Now hold that against the direction of travel. Schools are integrating AI by default. Workplaces measure velocity—tasks closed, tickets cleared, drafts shipped. Every one of those metrics rewards the removal of exactly the friction the research identifies as the mechanism of durable understanding.
We are optimizing, at institutional scale, for the single variable the evidence flags as corrosive. Faster, smoother, less effort per output—which is to say, sparser, less original, less retained.
The efficiency is real. That is what makes it dangerous. The coder's PRs merged; the ChatGPT users produced advice faster and felt fine about it. The cost is invisible at the moment of use and shows up days later, when you cannot explain your own conclusion.
So what is the honest response? Not abstinence. Oppenheimer is right that abandoning a genuinely useful tool because it "will universally lead to shallower learning" is its own error. The tool's effect depends on how you use it—and the finding is precisely that people "don't naturally use it as well as they might."
Naturally being the operative word. Left alone, we click one link in four and pocket the synthesis.
The response, then, is to design against our own defaults. Melumad calls it building "healthy frictions"—guardrails and speed bumps that push a user past the easy synthesized answer and back toward the sources. Not information added, but effort re-required.
The coder found the same thing by hand, the crude way: he banned the tool for six months to feel where the reasoning had gone. That is friction imposed by rule because it would not arrive by choice.
Both are admissions of the same fact. The convenience the study measures is not a feature we can enjoy while quietly keeping the depth. The convenience is the removal of the depth. They are the same motion.
Which is the genuinely damning thing the seven experiments prove—not that the machine thinks for us, but that when it offers to, we let it. Three out of four times. Even with the door held open.
The friction was never the obstacle to learning. The friction was the learning. And we are, with great enthusiasm and considerable investment, engineering it away.