The number that should stop you is not the score. It's the invoice.
On a seven-task benchmark of genuinely hard agent work — multilingual podcast distribution, multi-source data reports, domain literature reviews, codebase architecture docs — a single Claude Sonnet 4.6 run scored 69.1 and cost $18.36. Under PilotDeck 2.0's tiered routing, with Sonnet 4.6 doing the planning and MiniMax-M2.7 doing the execution, the same suite scored 70.6 at $3.15.
Higher score. Roughly one-sixth the cost. Call it 83%, reportedly, according to the developers' own benchmarking.
Higher score at one-sixth the cost is the headline. The unanswered question is which tasks paid for it.
That is reportedly the open-source OS-layer version of a lever Perplexity already pulled in production. This week the argument gets a reference implementation anyone can install, and a quality caveat.
Start with why routing works at all.
PilotDeck's Smart Routing detects task difficulty, then splits the work. Complex planning and decision-making stay on a flagship. Execution, polishing, and formatting get demoted to a lightweight model. In a social-media operations test, that meant Opus 4.5 at the planning checkpoints and Sonnet 4.5 on the sub-agent: $2.83 versus $12.58 with routing off. About 70% saved, reportedly.
The single-agent baseline tells the same story from the other side. MiniMax-M2.7 alone cost $1.90 and scored 37.1 — reportedly unusable, per the framing in the source. Cheap models don't fail because they're cheap. They fail because nothing is deciding what they're allowed to touch.
That's the real claim. Not "small models are good enough." Rather: most tokens in a long-horizon task are not reasoning tokens, and routing is the mechanism that stops paying frontier prices for them.
Now the part that deserves more scrutiny than it's getting.
Perplexity reportedly reported a 68% cut in cost per task across six agent benchmarks with its Fast Search preset, versus the default — "comparable quality," in their words. Then, in the same disclosure, they said Fast Search spends less compute on ranking, and that on internal tests of long-tail and broad-coverage queries it scores 0.24 points lower on relevance and about 3 percentage points lower on answer availability.
Both statements are true. They are also not the same claim. "Comparable quality" is what shows up in the launch tweet. The 0.24 and the 3 points reportedly show up in a separate post, per the evidence.
This is the pattern to watch as routing becomes standard infrastructure. The aggregate number is real; the distribution underneath it is where the cost gets paid, and it is paid unevenly. Long-tail queries degrade first. Broad-coverage queries degrade next. The tasks your benchmark averages away are exactly the ones your users notice.
PilotDeck's benchmark is reportedly stronger than most — seven named complex tasks with published scores and costs, not a vibes-based efficiency claim. But the same blind spot applies. A composite score of 70.6 versus 69.1 tells you the router won on average. It does not tell you which of the seven tasks the sub-agent got wrong, whether the main model caught it, or what fraction of runs needed a re-query or a reconciliation pass.
One reply to Perplexity's announcement reportedly put it precisely: a median that makes an agent re-query a weak result isn't actually cheaper. That objection applies verbatim to every tiered-routing system now shipping.
Here is where PilotDeck is genuinely ahead — and where the contrast gets uncomfortable.
Its memory layer is reportedly white-box. Generation, extraction, storage, and retrieval are visible end-to-end. If the agent misremembers a parameter, you can open the panel, find the offending entry, edit or delete it, and pin what matters. Dream Mode reportedly consolidates scattered memories during idle windows, and if consolidation drops a critical detail, one-click rollback restores the prior state.
That is auditability as a product feature. You can inspect what the system knows and correct it.
The routing layer has no equivalent. There is no panel showing you, per run, which model handled which subtask, what each step cost, what latency it added, and whether the cheap model's output survived review or got silently discarded upstream.
Memory is inspectable. Routing — the thing actually moving the money — is not.
This is the open question, and it lands squarely here: will vendors publish per-run cost, latency, and model mix? PilotDeck publishes benchmark-level totals. That is a start, and more than most. It is not an audit trail.
Why this matters beyond one open-source release: routing is about to be everywhere, because the economics leave no choice. If every subtask runs on a frontier model, long-horizon agents stay a high-value, low-frequency luxury. If routing works, background execution becomes viable — and PilotDeck's Always-on mode, which reportedly keeps running after you close the window and lands deliverables as local files, only makes sense in a world where tokens are cheap enough to burn unattended.
So the practical frontier isn't cheaper models. It's auditable routing: per-run cost, per-run latency, per-run model mix. The teams that instrument this will know which tasks their router should never have demoted. The teams that don't will discover it from users, later, at the worst possible moment.
PilotDeck's white-box memory is the right template. Apply it to the router.
Until someone does, treat every routing cost claim — including this one — as a floor on savings and an unknown on what got quietly traded away. The 83% is reportedly real. The question is what it cost, and right now the invoice doesn't itemize.