Last week, this column asked how a product should surface model uncertainty. The obvious answer — show a number — is the one the industry is quietly abandoning. This week, the evidence points somewhere harder: what makes people trust a system isn't how confident it looks. It's whether they can see who is accountable and where the machine stops.
The confidence score was the great hope of explainable AI. Put a percentage next to the output and let the user calibrate. It doesn't work. A study on miscalibrated AI confidence found that when a system was confidently wrong, only about a quarter of participants could detect the mismatch between the signal and reality. Misuse of the AI's output rose to over 41 percent, against roughly 28 percent under properly calibrated conditions. The number on screen doesn't make someone a better judge of the number. It gives them something to defer to.
That finding reframes the whole problem. "Classified with confidence 0.87" is a property of the model, not a property of the decision. It isn't actionable. "Marked high priority because it mentions a billing failure affecting multiple accounts, review if it's limited to one account" is actionable, and it exposes nothing about how the model got there.
A confidence score is a property of the model, not a property of the decision. It isn't actionable.
So the contract changes. If a system cannot show the human accountable for a claim, and cannot mark the boundary where automation stops, it should be forced to slow down, ask, and expose the decision. That is a harder build than a progress bar. It is also the only version that survives contact with high stakes.
The regulatory version of this already exists. In August 2026, the transparency provisions of the EU's 2024 AI Act took effect, with disclosure requirements for certain AI-generated public-interest text. Crucially, the law exempts content that "has undergone human review or editorial control" and for which a person or organization holds editorial responsibility. Accountability is the exemption. Not a score. Not a badge. A named human who stands behind the output.
NN/G's editorial process is the working example. AI helps rewrite, fit rigid character counts, adapt content across channels, and critique arguments — it once flagged a contradiction in an editor's own draft. But AI never certifies a claim is correct. When it flags something, a human goes back to the source and checks. Delegating the check to the tool would dissolve the exemption.
Now the friction side. A fintech team shipped autonomous overnight portfolio rebalancing. The trades were precise. Within two weeks, over 40 percent of users disabled the feature entirely. They didn't leave because the algorithm failed. They left because the interface removed their sense of oversight. Irreversible decisions with no visible feedback feel like dispossession, not assistance.
The fix isn't more alerts. Ambient dashboards over-seasoned with decorative indicators confuse high-stakes decisions. The fix is calibration: match intervention intensity to task criticality. Low-stakes interactions can run implicitly. High-stakes choices demand explicit, confirmation-driven flows. A confirmation step that requires manual verification isn't a defect. It's warm friction — the pause that lets someone reflect before finalizing something they can't undo.
Status is a separate concern from explanation, and both need naming. "Generating" isn't the same as "searching." A system blocked on an external call should say so. Partial results should be labeled partial. Draft, suggested, verified, completed — not a generic spinner. For anything multi-step, users otherwise can't tell "still working" from "silently failed."
Uncertainty needs language, not a percentage. Verified. Likely. Needs review. Unknown. Each paired with a reason and a way to resolve it where one exists. "Likely match, the invoice number is close, but the vendor name doesn't align. Review before approving" gives someone something to check.
Then there's the case where the friction contract is the entire ethical question. An AI that helps an employee make sense of a hard situation at work gets sold to HR, not to the employee. The 2024 Trust No Bot study found health, money, relationships, and work surfacing in places nobody expected, including translation requests and code edits. The person types the sentence they would never put in an email. Whose sentence is that? The contract says the buyer. The person says the opposite. The product lives in the distance.
Monitoring tools rarely use the word surveillance. They watch proxies — hours outside the workday, idle time, response speed, message volume, dips in activity — then translate them into categories like overload, flight risk, disengagement. ActivTrak promises to identify who is overutilized, at risk of burnout, or at risk of disengagement. Teramind sells total visibility over active and idle time and lists burnout prevention among its uses.
Some of these systems split into two dashboards. The personal one speaks self-care and focus time. The manager's translates the same behaviors into overload or flight risk. The employee doesn't know which data is captured, how it's cross-referenced, how long it's kept, who sees the result, or how to contest a wrong reading. The likely outcome is compulsory self-surveillance.
The design method has no step that asks whether the objective should exist. Someone in product or HR decided, before the first screen, that the problem was detecting burnout risk. That arrives as scope, and craft is superb at making a given objective clear and frictionless. Impeccable craft and a bad hierarchy of values coexist without contradiction.
This is where disclosure and friction stop being UX tactics and become the load-bearing wall. Friction is the only place the person being observed gets a turn.
The market runs the same direction. In Checkr's 2025 Great Untrust report, 88 percent of Americans said it's harder than a year ago to tell what's real online, and 60 percent had backed out of a purchase, booking, or date because something felt off. Adobe found more than 80 percent of consumers call control over personal data the single most important factor in trust. Not brand purpose. Control.
Nielsen's first heuristic in 1994 was visibility of system status, and his note was blunt: predictable interactions create trust in the product as well as the brand. What changed is the surface area. In his world, visibility meant a progress bar. In ours, it means explaining what an AI drew on, how a recommendation was made, and what your data is used for.
Which is why the design principles are being rewritten. Simplicity becomes understandability. Automation becomes explanation. Personalization becomes consent. Delight becomes consistency over time. None of those is a confidence score. All of them are a contract.
A trust score measures compliance with the system that measures trust scores. That's the trap. The disclosure-and-friction contract is the exit: name the human, mark the boundary, and when neither can be shown, slow the system down until someone can.