AI Evals Graduate From 'It' Skill to Core PM Discipline — But the Practice Gap Persists
Fabricated quotes dropped from 12.5% to 1.7%. Ungrounded claims from 8.4% to 3.2%. Status errors to zero.
Those numbers come from a fake houseplant app. According to the ProductTalk posts, a synthetic product called Sprout was built — it takes a customer interview transcript and writes a short story about the person's first month, then labels whether they're still using the app, stopped, or undecided. Then the whole test harness was published in a repo, run on both a sloppy baseline prompt and a tightened variant, showing the error rates fall for anyone who wants to watch.
That is the entire mystery of AI evals, made embarrassingly concrete.
And yet. The skill an OpenAI CPO reportedly called "the most important thing a product manager can learn to do" — the skill reportedly demanded in 32% of 654 PM job descriptions Dexity analyzed, riding alongside the 85% that mention AI/ML — is one the ProductTalk guide's author still can't get teams to define.
"I still meet product teams who only have a vague idea of what evals are," the ProductTalk guide reads.
Sit with that contradiction for a second. A third of the market prices evals as a hard requirement. A top-lab executive ranks it above every other PM skill. There is a public repo with a worked example you can run without an API key. And teams still can't tell you what an eval is.
The gap is not about awareness. Nobody in product has failed to hear the word "evals" this year. The gap is operational, and it lives at exactly one step — the one nobody wants to do.
The gap is not about awareness. Nobody in product has failed to hear the word 'evals' this year. The gap is operational, and it lives at exactly one step — the one nobody wants to do.
Here is what an eval actually is, stripped of mystique. With traditional software, you write a function that adds two numbers, and you unit-test it: send it 2 and 3, assert it returns 5. It's deterministic. Same input, same output, forever.
LLMs break that. Send the same input twice and you might get 5 the first time and 4 the second. And most of the tasks we hand them — summarize this, judge that, write a story — don't have one right answer anyway.
So you stop asking "did it get it right" and start asking "how often does it get it right." An eval measures a rate. That's the whole conceptual shift.
Which sounds simple, and is. The concept was never the bottleneck.
The bottleneck is the thing the harness quietly forces you to do first: establish a baseline. Before you can improve anything, you run every input through your prompt, save every output, run every eval against every output, and tally the results. Then you read the failures.
Not the summary table. The failures. On Sprout, that means sitting with the specific ways the model lied: two separate sentences stitched into one fake quote, "within days" when the participant said "within like a week," a bookmark feature that appears in the story but never in the transcript.
It means confronting the transcript where the participant's sister is the one paying for the annual plan, and checking whether the model attributed that subscription to the participant. It means the participant who opens the app every single morning but has already decided to cancel — and noticing the baseline prompt labels her status wrong.
This is the unglamorous first step. It is looking, by hand, at what your AI actually does before you build anything to automate the looking.
And it is precisely the step the industry is racing to skip.
Read the room in product right now and you see a tidy progression: 2023 was prompt engineering, 2024 context engineering, 2025 harness engineering, 2026 — per the framing cited in Aakash Gupta's newsletter — loop engineering. The energy is all downstream. Everyone wants the loop: an agent that fetches its own inputs, does the work, passes a gate, writes an artifact, and feeds itself back in.
Loops are genuinely powerful. That is not the argument here.
The argument is about ordering. A loop automates improvement. A baseline measures failure. If you build the loop before you've measured a single failure rate, you have not built a quality engine — you've built a faster way to ship regressions, with no instrument to tell you it's happening.
Folkman himself reportedly has the math for why this is dangerous. Ship twice as fast with AI at the same quality rate, and your customers experience twice as many bugs. Speed without measurement doesn't hold quality flat; it multiplies the defects. A loop with no baseline is that dynamic wearing a lab coat.
The reason teams skip the baseline is not laziness. It's that the baseline is the one part of the discipline that offers no dopamine.
The loop feels like leverage — you type a prompt, three prototype variants appear in the background, the machine works while you sleep. The baseline feels like homework. You read fifteen transcripts. You hand-label 29 claims to calibrate your judge, because the judge is an LLM too and you have to check its work. You count errors one at a time.
There's no demo in that. There's no Series B headline in "I read my outputs." But it is the only step that turns a score into a decision.
Notice what the Sprout variant actually proves. The tightened prompt didn't just push the fabricated-quote number down — it made every error rate go down, and the harness proved it with a side-by-side comparison. That's the check that matters: if a change fixes one error but makes another worse, it is not an improvement. You cannot know that without a baseline to compare against. The baseline is what makes "better" a fact instead of a feeling.
And here is the part that should reframe the whole thing for PMs worried about their role. When the eval is wrong — when the product passes and users still hate it — that's not on the model or the engineers. The eval is the PM's artifact. It encodes your understanding of what "good" means. Get the encoding wrong and you ship a 0.9 score attached to a product that fails in production.
That accountability only exists if you did the looking. You cannot write a scoring function that catches the failures that matter for your product until you've seen those failures in your own outputs. The baseline is where PM judgment gets built into the machine. Skip it and you're automating someone's guess.
This connects to a divide worth naming plainly. Elite orgs stand up serious eval infrastructure; a solo founder rents a rack of agents. The eval harness in the ProductTalk repo is the accessible version of that same capability — a small repo, one command per step, written for novice coders, with a template you can hand to a model and ask it to write the next eval module for you.
The tooling is no longer the moat. It's boringly reproducible now. The ProductTalk repo proved that: the discipline fits in a public repo with a made-up plant app.
What separates a real org capability from AI theater is not the harness. It's whether you do the unglamorous step in the right order — measure first, then loop. Everyone will have the loops. Only some teams will have looked at their failures before automating on top of them.
That ordering is the whole discipline. It is also the one thing you can't buy, can't generate, and can't skip.
Read your outputs. Count the errors. Then, and only then, build the machine that goes fast.