Read two numbers side by side and they refuse to reconcile.
Morgan Stanley says a Vera Rubin NVL72 rack costs $7.8 million, with roughly $2 million of that in memory. A separate teardown circulating in March put the HBM4 alone at $10 to $16 million per rack — more than the entire Morgan Stanley bill.
They can't both be right, and the gap is not a rounding error. It's the whole story.
The reconciliation is this: the $2 million figure counts memory the way an accountant does — as procurement line items. The $16 million figure counts it the way a physicist does — as the thing the workload actually needs and cannot get. The distance between those two numbers is the space where a new architecture is being built.
The distance between an accountant's $2 million and a physicist's $16 million is the space where a new architecture is being built.
Start with what didn't change. The Vera Rubin rack keeps the same Oberon chassis, the same 72 GPUs and 36 CPUs as the Grace Blackwell generation before it. Nvidia doubled the price — from $3.5–4 million to $7.8 million — without adding a single accelerator or expanding the box.
So where did the roughly $3.9 million go? Almost everywhere except the GPU.
The GPU's share of the bill of materials fell from about 63% to 51%. Individual Rubin dies do cost more — around $55,000 versus $35,000 for Blackwell — but that scaling accounts for only ~$1.5 million of the increase.
The memory line tells the real story. Memory content jumped 435%, from roughly $374,000 to about $2 million per rack, now 25–30% of total cost. And a storage line that was effectively zero on Blackwell materialized at roughly $1 million — a full 13% of the rack.
In absolute dollars, the memory increase exceeds the processor increase. That has never happened before in enterprise computing.
Read as a spreadsheet, this looks like a cost problem Nvidia is passing through. Read as a blueprint, it looks like a deliberate bet: Nvidia spent its entire cost increase on everything-but-the-GPU because that is where dollars now buy usable throughput.
To see why, stop looking at the rack and look at the workload.
GPT-class context windows went from 4,096 tokens in 2022 — about one long article — to roughly 1 million tokens in 2025, a PDF of several hundred to a thousand pages. That is not a feature bump. It is a change in the physical quantity of state a rack must hold.
Long context generates a KV cache: the intermediate attention state a model reuses across every token it emits. Run a thousand concurrent users at million-token contexts and that cache balloons past 100 terabytes.
The rack holds 20.7 TB of HBM4. The math does not close. The most expensive memory ever manufactured cannot physically contain what the workload produces.
This is the collision behind the two irreconcilable numbers. HBM is 20% of memory capacity and the overwhelming majority of memory cost. It is also fixed-scarce — HBM4 requires 3–4x more silicon per gigabyte than DDR5, and only three vendors can make it at volume.
You cannot buy your way out of the KV cache problem with more HBM. There isn't enough, and even if there were, packing 100 TB of it beside every rack would be economically absurd.
So the flat "buy HBM and be done" model shatters into a hierarchy. Each tier trades latency for capacity for dollars, and each is a place a designer can actually move a variable.
At the top sits HBM4 — 288 GB per GPU, the compute workhorse, non-negotiable and price-fixed by scarcity. Below it, LPDDR5X: the Vera CPUs carry 1.5 TB each via SOCAMM2 modules, pushing system memory from 17 TB on Blackwell to 54 TB. That LPDDR expansion is a large part of why the memory line tripled.
Below that, CXL pools — DDR5 attached over a switched fabric, KV-cache overflow at roughly 1% of the HBM price per gigabyte. Then GPU-Direct flash, where GPUs read NAND at fine granularity through Nvidia's storage stack. Then, at the base, dedicated flash appliances.
That bottom tier is where the ~$1 million storage line comes from, and it has a physical form.
AIC's F2032-01-G6 is a 2U box holding 32 PCIe Gen6 E3 SSDs — up to 8 petabytes of QLC in a single chassis. Its I/O fabric is built around Broadcom's 144-lane PEX90144 Gen6 switch, and it is driven not by a host CPU but by up to two of Nvidia's BlueField-4 DPUs acting as storage controllers.
This is a just-a-bunch-of-flash appliance built for one job: serving as a KV cache tier inside Nvidia's Context Memory Storage platform, the caching standard designed to slot into Vera Rubin racks. The point is proximity — keep the key-value state as close to the inferencing GPUs as the physics permits, so a million-token context doesn't force a round-trip to network storage on every step.
Which reframes the JBOF box entirely. It is not "storage" in the old sense of checkpoints and cold data. It is the overflow floor of the memory pyramid — the place the KV cache lands when HBM is full, which for agentic workloads is always.
Now the architecture Nvidia bought becomes legible. It did not spend $3.9 million to make the GPU faster. It spent it to make sure the GPU is never starved — on LPDDR5X capacity, on PCB layer counts up 233% for the fabric that moves data between tiers, on the cooling and power those tiers demand.
And here is the consequence that should reorganize how buyers think.
HBM is fixed. Its price, its scarcity, its share of the bill — a designer cannot move any of it. That constraint, the subject of every conversation about memory-vendor leverage, is precisely what makes the rest of the pyramid matter.
Because if the top tier is immovable, the only levers left are the tiers below it: how much LPDDR5X you provision, how deep your CXL pool runs, how many JBOF bays you hang off each rack, where in the hierarchy each slice of KV cache lives.
Those decisions determine whether a rack's dollars buy inference throughput or stranded silicon. Under-provision the cache tiers and the GPUs — 51% of your bill — sit idle waiting on data that lives in the wrong place. Over-provision and you've spent a million dollars of flash serving a workload that never needed it.
This is why the swing variables migrated downward. The GPU count is fixed at 72. The HBM is fixed at 20.7 TB. The chassis is the same Oberon it was last generation.
The only things a cloud operator can actually tune — the SOCAMM2 module, the CXL switch, the 32-bay flash appliance behind a BlueField-4 — are the things that used to be rounding errors.
Morgan Stanley's accountants see a $2 million memory bill. The teardown that priced HBM at $16 million was measuring the same rack against what its workload demands.
The gap between those figures is not an error to be corrected. It is the design space — the tiers of cheaper, slower, movable memory that now stand between a rack full of expensive silicon and a rack that actually runs.
Cloud margin used to be won at the accelerator. Now it's won in the basement of the memory pyramid, one JBOF bay at a time.