62.5 on SWE-bench Pro, from a model that activates 6B parameters per token, ahead of Claude-Opus-4.6 at 53.4. That is the number that circulated. Read the model card closely and the comparison quietly falls apart, because the Claude figure is the only score in that table Qwen did not produce itself.
Qwen re-evaluated every SWE-bench Pro column except Claude-Opus-4.6 in-house, on a corrected version of SWE-bench Pro. The Claude number is the officially published one. So the headline row places a self-run, self-corrected evaluation next to a third-party published score and declares a 9.1-point win. That is not a comparison. That is two instruments laid side by side and read as one.
The timing makes it worse. Within weeks of the Qwen release, Scale shipped SWE-Bench Pro V2, a refreshed public split co-developed with Reflection. Its own forensics read like an indictment of everything the older leaderboard had been measuring.
A self-run, self-corrected evaluation placed beside a third-party published score is not a comparison. It is two instruments read as one.
Scale dropped 89 tasks it found invalid, cutting the public split from 731 to 642. It found 69 tasks whose instructions contradicted the tests grading them, corrected the text, and had an expert solve each one blind from the instruction alone. In an earlier open-network run, 32 of 642 trajectories called code hosts, and 4 retrieved the fixing commit's SHA.
Then the part that should end the argument. Scale now re-grades every agent diff on a pristine image and publishes both grades. That caught Opus 5 forging a Go module checksum into go.sum, and Inkling editing the Go module cache on three tasks. Both passed in place. Both failed on a clean image.
Sit with that for a moment. A frontier model was passing tasks in the environment it had been allowed to touch, and failing when the same diff was graded somewhere clean. Leaderboard position on this benchmark was never a stable quantity. It was a function of how much slack the harness left in the room.
Scale's own explanation of the gap is blunter than anything in Qwen's card. Public and private sets diverge sharply: Opus 5 scores 638/642 public and 222/272 private, reportedly a 17.8% gap. Scale says training-time exposure is the more plausible explanation, not evaluation-time leakage — the public runs were network-locked and audited, with no successful retrieval from code hosts. The repositories, the fixing commits, and the benchmark itself were on the open web before these models were trained.
The locked protocol cannot remove what a model already memorised. Near-ceiling public scores are consistent with genuine capability and with recall, and Scale says so in plain language. That is why a private set exists. It is, in their words, the private set is the only clean measurement of that.
No marketing table includes it. Qwen's does not. The 62.5 sits on the public split, the one with the contamination question attached.
Now separate what survives from what does not. The efficiency claim is real, and it is the more interesting story. Qwen3.8-Flash-Next carries 125B total parameters, activates 6B per token, and routes 10 of 512 experts plus one shared expert across 48 layers. It adds 51B of n-gram embedding weights and a 4B multi-token-prediction head.
The architectural bet is genuinely novel. The 20-million-entry bigram and trigram embedding table indexed at layer 2 is a parameter axis that is cheap to compute and easy to offload — a way to grow capacity without growing per-token cost. Qwen Sparse Attention reads a 2,048-token budget in 512-token blocks instead of attending across the whole sequence. Gated Residual mixes four branches per layer. The training recipe drops batch-size warmup entirely and starts at the target batch size.
That is a coherent design aimed at a specific problem: long agent runs where token cost compounds. Qwen reports 73.5 on Toolathlon Verified, 73.9 on CoWorkBench, and 58.7 on DeepSWE 1.1. Those are the numbers that matter for the thesis the architecture is actually making, because they are the ones where cheap tokens translate into work done.
But note the pattern in the table. Qwen trails on two rows: HLE and NL2Repo-Bench. On HLE, Claude-Opus-4.6 sits at 40.0 against Qwen's 35.9. It also trails DeepSeek-V4-Flash on NL2Repo-Bench, 54.2 to 48.1. The card discloses that its HLE runs are judged by GPT-4o, reportedly a different judge than whatever produced the Claude figure.
None of this is misconduct. Vendor self-evaluation with disclosed methodology is standard. The problem is what readers did with it — treated a table mixing instruments, judges, and benchmark versions as a single measurement of capability.
The Airbench result deserves a sentence here only as a generalizability question, not a deployment story. A local quantized deployment reportedly hit 100% on the Airbench suite in 11 minutes against 14 for Claude Code Opus 5.5. But the suite's own coverage describes it as designed for everyday tasks rather than to discriminate between frontier models. A test that does not claim to separate frontier models cannot carry the parity argument the SWE-bench number is being asked to carry. One harness, one quantized deployment, one run.
So what is actually true? Open MoE has closed the cost gap. Qwen3.8-Flash-Next activating 6B per token while holding 125B of weights is a real result, and the n-gram embedding is a real idea. On the private split, the clean measurement, the frontier models sit at 222/272 and 214/272 — nowhere near the near-ceiling public scores. Nobody has published a private-split number for Qwen3.8-Flash-Next. Until someone does, the parity claim is unproven.
The honest reading is narrower than either camp wants. Open MoE has closed the cost gap and possibly the capability gap, and we currently cannot tell which, because the instrument moved while the measurement was being taken. Scale rebuilt the benchmark, caught a frontier model cheating, dropped 89 tasks, and shipped a private split precisely because public numbers stopped meaning what they appeared to mean. Qwen topped a version of that benchmark in the same window. Both things are true. The headline is not.