For most of the AI boom, progress has been narrated in the language of raw compute: exaflops, petaflops, ever-bigger numbers. Nvidia's next platform, Vera Rubin, is an argument that the industry has been measuring the wrong thing. When a rack of GPUs tries to run a trillion-parameter model, the chips are rarely starved of arithmetic. They sit idle waiting for data to arrive from memory. That bottleneck has a name, the memory wall, and Vera Rubin is built to knock it down. Production shipments begin this fall across all eight confirmed cloud partners, from AWS, Google Cloud, Microsoft Azure and Oracle to specialists like CoreWeave and Lambda.
The engineering choices follow directly from that thesis. Each Rubin GPU carries 288 gigabytes of new HBM4 memory delivering 22 terabytes per second of bandwidth, close to triple the previous Blackwell generation. A faster NVLink interconnect doubles the bandwidth between chips, and a custom Vera CPU links to the GPU over a connection that erases the old boundary separating processor and accelerator memory. The practical payoff, Nvidia says, is that the flagship NVL72 rack can train large mixture-of-experts models with roughly a quarter of the GPUs a Blackwell system would need, while serving inference at one-tenth the cost per million tokens. Jensen Huang calls it "token factory economics," the idea that every data center is now a factory whose output is measured in tokens per watt.
There is an irony buried in the bill of materials. By solving a memory problem with a great deal more memory, Nvidia has made memory the single most expensive thing in the box. A Morgan Stanley estimate puts a single NVL72 rack at about $7.8 million, nearly double the Blackwell equivalent, and the HBM4 and related chips alone account for roughly $2 million of that, around a quarter of the total. For the first time, the memory supply chain, not the GPU, is the dominant cost driver in an AI server. That is the same squeeze, viewed from the top of the market, that is now pushing up the price of laptops and phones as data centers hoover up the world's memory.
The launch lands in a jittery moment for the company that has defined the boom. Nvidia stock just posted its worst week in more than a year, sliding below $200 as investors began to question whether the industry's vast AI spending will pay off and whether rivals are closing in. AMD's competing Helios systems promise comparable inference performance with more memory per chip; Google keeps expanding its in-house TPUs; and customers from OpenAI to ByteDance are designing their own inference silicon. The bull case, as one analyst put it, "is built on tightness," and tightness can cut both ways.
Still, the breadth of Vera Rubin's launch is its own kind of moat. Coordinating all four major clouds and four specialist providers into a single half-year deployment window is something no competitor has matched, and it rests less on raw benchmarks than on Nvidia's CUDA software, its sprawling supply chain and its relationships with every hyperscaler. The catch for everyone else is patience: with TSMC's advanced capacity finite and HBM4 yields still maturing, most enterprise teams will not get their hands on Vera Rubin until 2027. The memory wall is coming down, but only for those at the very front of the queue.