A product manager may never touch a chip, yet this year's hardware news shows up in their products as pricing tiers, rate limits and the speed at which text streams onto a screen. Three things happened in 2026 that are worth understanding: the newest general-purpose platform started shipping, hardware built only for decode went mainstream, and the largest labs began buying capacity in gigawatts.
Every figure below is from the companies' own announcements, filings or documentation, checked on 20 November 2026. Performance multiples in vendor announcements are company claims.
The new platform shipped
NVIDIA's results for the quarter ended 25 October 2026, released on 17 November, reported data-center revenue of $107.6 billion, up 110% from a year earlier, and said the company "began shipping NVIDIA Vera Rubin NVL72", a rack of 72 Rubin GPUs and 36 Vera CPUs. The competition shipped too. AWS made Trainium3 UltraServers generally available in December 2025, with 144 GB of HBM3e per chip. Google's Ironwood TPU, which it called "the first designed specifically for inference", carries 192 GiB of HBM per chip. AMD's November filing said Anthropic, Meta, Microsoft, OpenAI and Oracle Cloud Infrastructure were running workloads on its first Helios rack systems.
For products, a new platform arrives as provider capacity, not on announcement day. Watch for its effect in rate cards and tiers rather than in chip specifications.
Hardware built only for decode
The bigger shift is specialisation. An LLM request has two phases. Prefill reads the prompt and is limited by arithmetic. Decode writes the answer one token at a time and is limited by how fast weights can be read from memory. AI accelerators for PMs walks through the arithmetic.
Through 2025 and 2026, vendors built for one phase at a time. NVIDIA announced Rubin CPX for long-context prefill, using cheaper GDDR7 memory. Its March 2026 platform paired Rubin GPUs with a Groq 3 LPX rack of 256 LPUs holding 128 GB of on-chip SRAM to "boost decode", after Groq licensed its inference technology to NVIDIA in December 2025. OpenAI agreed in January 2026 to deploy 750 MW of Cerebras wafer-scale systems, which it described as "a dedicated low-latency inference solution".
Read diagram description
Compute capacity OpenAI announced with three hardware partners. Broadcom: 10 gigawatts of OpenAI-designed accelerators, deploying from the second half of 2026 to the end of 2029 under a term sheet. AMD: 6 gigawatts of GPUs, the first gigawatt from the second half of 2026. Cerebras: 750 megawatts of wafer-scale systems for low-latency inference, multi-year from 2026.
Capacity measured in power rather than chips says something about the constraint. Building enough inference now depends on electricity, cooling and supply as much as on silicon design.
What reaches your product
Speed becomes a tier. Decode-specialised hardware serves fewer users per chip at higher speed, and providers price it that way. Anthropic lists fast mode for Claude Fable 5.2 at $16 and $80 per million tokens, double its standard rate, and OpenAI renamed its priority tier to Fast mode in July 2026. Pay for speed only where a person waits for the complete output, such as coding agents running many calls; streamed text that people read as it arrives gains little. The arithmetic is worked in which products should pay for a fast tier.
Waiting becomes a discount. On 17 November OpenAI added, in beta, two-hour and eight-hour background windows priced 30% and 40% below standard. Work that can wait fills capacity between peaks. Classify every workload as interactive or deferrable before pricing it.
Long prompts carry a latency cost. Prefill grows with prompt length, and on shared serving pools a long prompt can slow other users' streams, which is one reason providers price long contexts higher and why serving systems increasingly split prefill from decode. Prompt caching removes repeated prefill work, cutting both cost and time to first token.
Capacity is still a roadmap dependency. Shipping platforms do not mean unlimited supply. Rate limits, regional availability and provisioned-throughput commitments remain the levers providers use to ration capacity. Plan launch-day capacity as a dependency with an owner.
The question to bring to your next planning meeting
Which of our AI workloads have a person waiting for the full output, and which could run in the background? That one classification decides whether to buy speed, buy discount or buy neither, and it will stay useful through the next hardware cycle too.
For the self-hosting side of the same economics, see GPU economics for PMs and mixture of experts.
Questions about this topic
Prefill processes the whole prompt at once and is limited mainly by arithmetic, so it sets time to first token. Decode generates one token at a time and must re-read the model's weights for each step, so it is limited by memory bandwidth and sets how fast text streams.