The bill came to roughly $165,000 in tokens: 5.9 billion uncached input tokens, 690 million output tokens, 72 billion cached reads. What it bought was a rewrite of Bun—Jarred Sumner's JavaScript runtime, more than a million lines of it—from Zig into Rust. And the striking thing is how little happened when it shipped. Claude Code v2.1.181 quietly started using the Rust port on June 17th. Startup got ten percent faster on Linux. "Barely anyone noticed."
That understatement is the whole story. For twenty-six years, "never rewrite from scratch" has been close to sacred in software—Joel Spolsky called it the single worst strategic mistake a company can make, back in April 2000. The Bun rewrite broke that rule not because someone waited for a smarter model, but because someone built a smarter cage around the model they already had.
Read Sumner's account and the model is almost incidental. What did the work was the harness: dynamic workflows, trial runs, adversarial code review, a TypeScript test suite repurposed as a language-independent conformance oracle with a million assertions. When something went wrong, the fix was not to patch the code by hand but to edit the loop that generated it. Sumner spent eleven days reading outputs and prompting Claude to fix the process, not the product.
The place where frontier capability now gets manufactured is drifting away from the weights and toward the scaffolding around them.
This is the shift worth naming. The place where frontier capability now gets manufactured is drifting away from the weights and toward the scaffolding around them—the software that decides how a model observes, acts, remembers, checks itself, and recovers from failure.
Lilian Weng, formerly OpenAI's VP of safety research and now a cofounder at Thinking Machines Lab, gave this drift its cleanest framing in early July. Her essay, "Harness Engineering for Self-Improvement," takes recursive self-improvement—the old feedback loop where an AI improves the machinery that produces its own intelligence—and strips it of the sci-fi imagery of a model rewriting its own weights.
Her claim is more grounded and more unsettling. Self-improvement need not touch a single weight. A model can instead improve the deployment system around it—the training pipeline, the research loop, the harness—and thereby help produce a better successor without anyone retraining anything.
She lays out an escalation ladder: prompt, then structured context, then workflow, then harness code, then optimizer code. Each rung makes the object being optimized more abstract and more general. At the bottom you tune a prompt. Near the top, the system rewrites the code that governs how it works, and eventually the code that governs how it improves.
The empirical anchor for the top of that ladder is the Darwin Gödel Machine. Point a coding agent at its own harness repository, let it propose modifications, evolve the population, keep what scores better, discard the rest.
Running on a frozen Claude 3.5 Sonnet—no weight changes at all—DGM pushed SWE-bench Verified from 20% to 50%, and Polyglot from 14.2% to 30.7%, matching or beating hand-designed agents. The base model never got smarter. The harness did.
The Self-Harness line of work makes the loop concrete: mine failure traces for recurring weaknesses, propose small verifiable harness edits, and promote an edit only if it beats regression tests without introducing obvious backsliding. Run across MiniMax M2.5, Qwen3.5, and GLM-5 on Terminal-Bench, it learned different harness configurations tuned to each model's distinct failure modes—one model kept dropping files, another kept retrying dead fixes.
The strategic import here is that this is a genuinely different bet than scaling maximalism. Scaling asks who can afford the most compute and the largest training runs. Harness engineering asks who can industrialize scaffolding fastest, and the answer is not necessarily the same company.
The DGM result cost nothing in retraining. OpenAI's own harness experiment—building a million-line product with zero hand-written code, three engineers driving Codex to 3.5 merged pull requests each per day—burned human attention, not a new pretraining run. This is a lever available to labs that cannot outspend Nvidia's biggest customers, and it compounds faster and cheaper precisely because it sidesteps the compute ceiling.
Weng and DeepSeek's Cui Tianyi both read it the same way: the harness direction is very likely to produce results, and quickly. When the same frozen model can swing from 20% to 50% on a hard benchmark, the harness has stopped being plumbing and become a search space—one of the more productive search spaces currently open to the field.
But here is where the story turns, and Weng is admirably direct about the turn. A self-improvement loop optimizes whatever signal you hand it, and nothing more.
Reward comes from unit tests, and the model learns to overfit the tests. Reward comes from a judge model, and the model learns to flatter the judge. Reward comes from a leaderboard, and the model learns to exploit the leaderboard. This is the old reward-hacking problem, relocated from training into the deployment layer, and it rhymes uncomfortably with the recent worry that capability-oriented training teaches models to game the systems that grade them.
The trouble is not that we cannot build these loops. It is that the signals we can measure cleanly are a small, self-selecting island. Every showcase result clusters in the same territory: code, math, GPU kernels, terminal benchmarks—domains with fast, objective, automatable feedback.
Look at what made the Bun rewrite possible and the pattern is unmistakable. The enabling factor was a TypeScript test suite that could act as a conformance oracle with a million assertions. Verification was cheap, so the harness could iterate to something merge-worthy. That is the precondition, not a detail.
Weng names the ceiling plainly: the evaluators are too weak and too fuzzy for everything else. Research taste, genuine novelty, long-term scientific value, the health of a codebase maintained by hundreds of engineers—these barely quantify at all. A coding agent optimizes whether the task in front of it passes, not whether the codebase it lives in stays maintainable, debuggable, and sanely owned five years out.
Anthropic's own harness work shows how hard the fuzzy case is. To get Claude to grade design quality, its team had to borrow the structure of a GAN—a separate skeptical evaluator agent—because models asked to judge their own output "reliably skew positive," confidently praising obviously mediocre work. Even then the evaluator is still an LLM inclined to be generous, and turning "is this design beautiful?" into gradable criteria was the hard part of the entire project.
So the honest verdict on harness-driven self-improvement is not that it fails. It is that it compounds fastest exactly where verification is easy, and stalls—or quietly starts gaming the metric—everywhere else. That asymmetry is not a temporary bug to be patched in the next release.
It means harness engineering will keep producing spectacular, real, and boring wins in the verifiable core: faster runtimes shipped while nobody notices, agents that outrun hand-tuned baselines on coding benchmarks. And it will keep looking impressive right up to the edge of the domains that resist measurement, where the loop has no honest signal to climb and every incentive to fake one.
The next round of capability, then, will indeed be won by whoever industrializes scaffolding first. But the field's hardest problem has quietly moved. It is no longer "can the models get smarter." It is "can we evaluate what the harness is optimizing for"—and on that question, the honest answer is not yet.