The word is "fully." DeepSeek's release notes call DeepInfer fully open source, and the same engineering account says the KV-cache management and load-balancing components are not yet fully released, because they remain coupled to internal infrastructure. The engine is open; important service-level orchestration remains a deployment responsibility.
That gap is the whole story. Last week we traced how routing decisions were becoming the real lever in agent economics; this week the same logic shows up one layer down, in the serving stack itself. DeepSeek is handing out the engine while keeping the middle.
Start with what actually shipped. DeepInfer is a customization of vLLM, with credit to SGLang and TensorRT-LLM, covering distributed inference, speculative decoding, SWA Bounded Replay, and structured output on both NVIDIA and Huawei Ascend. A companion deepseek-recipe implements production token encoding and decoding with official-API-compatible processing.
DeepSeek is handing out the engine while keeping the middle — the cache, router, and scheduler welded to internal infrastructure.
What did not ship is the part that makes the numbers work. The ultra-optimized KV cache — compact, with lifetimes past eight hours — plus PD-disaggregation and load balancing stayed home. DeepSeek's 3FS file system was open-sourced earlier, so this is a deliberate line, not an oversight.
Now look at the numbers those withheld pieces produce. DeepSeek reports 155 trillion tokens per day from one cluster, 96% input-cache hits, 400ms average time to first token, and 230 tokens/s generation, reaching 300 tokens/s for coding. At off-peak prices of $0.15/$0.60 per million input/output tokens, that works out to $38 revenue per accelerator-hour.
Against all-in costs of $5–8 per card per hour, that is an inference margin above 80%, and an 81-day payback period on inference margin — against DeepSeek's famously stated 10-month planning target.
These are first-party production claims, not independently audited measurements. The workload is also a heavily cached one, and the profitability debate that followed is sensitive to workload mix, cache accounting, hardware cost, and which expenses get included. Treat the margin as a disclosure, not a proof.
The more interesting test came from vLLM. After acknowledging DeepInfer was initially built primarily on vLLM and released under Apache 2.0, the vLLM team reported decode throughput within 6% of DeepInfer's published results on GB200 under matched configuration, and published AgentX results and reproduction recipes alongside.
That deflates the claim that DeepSeek's serving efficiency is uniquely inaccessible. It also does something more useful: it localizes the remaining 6%. If a matched open configuration gets you within six points, the distance between a good open engine and DeepSeek's production service is not a kernel. It is the part that was not released.
Here is why the withholding is rational rather than stingy. Prefill scheduling estimates attention-compute cost and allocates work by estimated compute rather than raw request length, varying chunk sizes against a time budget. Prefill routing does not require cache affinity at all, because cached prefixes live in shared storage and any available worker can take the request.
Those are not algorithms you can drop into a repository. They are properties of a deployment — shared storage, pooled prefixes, a scheduler that knows what every worker is holding. They are welded to infrastructure, which is exactly what DeepSeek said.
Which brings us to Modal, a company that has never claimed to have the best kernel. It crossed $1 billion in annualized run rate and raised a $750M Series D at a $15.75B post-money valuation, growing more than 15x year-over-year and tripling usage in five months.
The striking part is where the revenue sits. Sandboxes launched in 2023 because Erik Bernhardsson noticed people were using LLMs to generate code, and that someone would need a place to run it. Early adoption was modest enough that the team discussed shutting the product down.
Instead, sandboxes became a $300M run-rate product on their own — roughly a third of Modal's revenue. The company has spun up close to 3 billion sandboxes, scaling to a million concurrent. Bernhardsson says it is a result he did not predict at launch.
That product is not an inference engine. It is a scheduler, a custom file system, a container runtime, process checkpointing, and GPU memory snapshots, running on a capacity pool spanning 20-plus clouds and 100–150 regions, rented rather than owned. Modal buys three months ahead into exponential demand because GPUs take three to six months to procure.
Bernhardsson frames the core bet as duration transformation: a typical capacity contract is a rectangle — fixed GPUs for a fixed period — while real workloads grow, spike, and fall. Pool demand across thousands of customers and you can sell elastic slices of a smoother aggregate. That is a financial and operational product, not a serving optimization.
Read the two stories together and a pattern resolves. DeepSeek's disclosed architecture separates vision encoding, prefill, and decode, recommends ordinary eight-accelerator machines for prefill and 64–128-accelerator supernodes for decode, and uses wide expert parallelism to keep effective batches large. Every one of those choices is about matching stages to bottlenecks across a fleet.
vLLM closed the kernel gap to 6%. Nobody has published a recipe that closes the orchestration gap, because the orchestration gap is not a recipe. It is a cache that survives eight hours, a router that ignores affinity because storage is shared, and a scheduler that treats capacity as something to be reshaped rather than reserved.
This is where the durability question actually lives. If the engine is common and the orchestration is proprietary, the inference layer's defensibility is not in the kernel at all — it is in the undownloadable middle.
Note what that means for pricing. If open engines converge, per-token margins compress and the engine becomes a commodity input. The margin then accrues to whoever can hold a cache hit at 96%, route around a stale worker, restore a sandbox in milliseconds, and buy capacity cheaper than the next operator. DeepSeek's own disclosures make this visible from the other direction: its profitability depends on a 96% cache hit rate and off-peak pricing, not on a novel attention kernel.
It also means the open-source release is a competitive move, not a gift. Publishing the engine raises the floor for everyone and commoditizes the layer DeepSeek no longer needs to defend, while the service-level orchestration stays inside the walls. The gift is the engine; the product is the middle.
Modal's numbers suggest the middle is where the money is. Inference is roughly two-thirds of its revenue and sandboxes roughly a third, and most demand comes from AI-native companies founded in the last five to seven years, often reaching enterprises indirectly. Bernhardsson also describes a "super significant" move toward open-source models like Kimi K2, GLM, and Qwen — good enough without fine-tuning and much cheaper.
That should be uncomfortable for anyone selling open-weight inference as a business. Modal's growth is partly a bet that model choice keeps fragmenting and workloads keep getting spikier — both of which increase the value of the layer that schedules and isolates, and decrease the value of the layer that multiplies matrices.
The honest caveat: Modal's model carries real risk on the other side. Locking in capacity at today's prices and watching GPU prices fall leaves you holding expensive supply, a possibility Bernhardsson says matters when building the business even though he does not expect an imminent collapse. Orchestration margins are also exposed to the same cache-accounting skepticism as DeepSeek's.
So the open question is not whether DeepSeek's efficiency gains translate into durable pricing pressure. It is where that pressure lands. This extends the question we raised when DeepInfer dropped: the pressure is at the orchestration layer, not the kernel, and the kernel is being given away because that is where the pressure already arrived.
If that reading holds, the next twelve months look less like an engine race and more like a capacity race. The winners will be the operators who can turn rented rectangles into elastic slices, hold prefix state across a fleet, and restore an environment before an agent notices it was gone. None of that is in the repository. All of it is the product.