OpenAI's upcoming Jalapeño chip looks like it'll be an inference beast

OpenAI showcased its Jalapeño custom AI inference accelerator at Hot Chips, targeting higher throughput and lower latency versus leading Nvidia GPU systems, with reported peak throughput gains of 1.5x–1.9x and end-to-end latency improvements of 1.7x–3.6x across models tested. Each Jalapeño rack uses 128 accelerators delivering 1.7 exaFLOPS of 4-bit compute, 27.5 TB of HBM4, and just under 2 petabytes/sec of memory bandwidth, with OpenAI claiming 2.1x–4.1x faster ultra-low-latency inference. Jalapeño is expected to trickle later this year and reach volume production in 2027 in a rack-scale design, likely affecting AI infrastructure demand sentiment but not yet moving markets broadly.

Analysis

The market should treat this less as an imminent displacement of GPU demand and more as evidence that the inference layer is becoming a custom-silicon battleground. That is structurally positive for AVGO because its AI story increasingly includes “design-win optionality” rather than just networking; however, the actual revenue inflection is back-half 2026 to 2027, so the stock can rerate on expectations well before dollars hit. The nearer-term risk is that investors overestimate how quickly OpenAI’s internal silicon can absorb meaningful load, leaving the incumbents’ training spend intact for longer than bears expect.

For NVDA and AMD, the immediate hit is probably sentiment rather than fundamentals: this reinforces that hyperscalers and frontier-model operators want inference economics optimized for their own workloads, but it does not eliminate the need for general-purpose GPUs in training, experimentation, and fast model iteration. The more important second-order effect is pricing power — if inference is increasingly custom, the premium attached to the highest-end GPU platform may compress at the margin in 6-18 months, especially for workloads that are stable enough to be ASIC-ified.

Contrarian view: consensus may be too binary. The real winner may be the company that owns the software/tooling and the supply-chain orchestration around custom AI racks, not the chip itself. If OpenAI’s chip rollout slips, or if its inference mix remains small versus training, the whole bear case on NVDA/AMD should fade quickly; watch for 2026 capex commentary and any evidence that custom silicon is still confined to a narrow subset of requests.

Key Decisions for Investors

• Long AVGO vs short NVDA on a 3-6 month horizon: express the view that custom AI silicon share gains will rerate Broadcom before they materially impair Nvidia; target a modest pair spread, with the thesis falsified if NVDA prints accelerating inference-related demand or AVGO gives no update on additional AI ASIC wins.
• Do not short AMD here; instead, use any post-event weakness to accumulate on a 6-12 month horizon, because AMD’s risk/reward still depends more on GPU share gains and ROCm adoption than on OpenAI-specific custom silicon.
• For tactical positioning, buy AVGO on dips only after management confirms broader customer replication beyond a single flagship design win; if the narrative stays one-customer-specific, trim after a 10-15% re-rating.
• Set an alert on NVDA/AMD commentary into next earnings: if management frames inference pricing or customer custom-silicon adoption as accelerating, reduce long exposure across the AI semi complex; if not, treat the move as noise and keep the structural long.