Nvidia's new chips accelerate large AI system training, data shows

New data from MLCommons indicates Nvidia's Blackwell chips significantly improve AI training efficiency, requiring fewer chips to train large language models like Meta's Llama 3.1 405B. The Blackwell chips are reportedly over twice as fast as the previous Hopper generation on a per-chip basis, with 2,496 Blackwell chips completing a training test faster than three times as many Hopper chips. CoreWeave's Chetan Kapoor noted a trend towards smaller, specialized chip subsystems for AI training, enabling faster training times for complex models.

Analysis

Newly released data from MLCommons indicates a significant advancement in AI training efficiency with Nvidia's (NASDAQ:NVDA) latest Blackwell chips. These chips are reported to be over twice as fast on a per-chip basis compared to the previous Hopper generation, demonstrated by 2,496 Blackwell chips completing a training test on Meta Platforms' (NASDAQ:META) Llama 3.1 405B model in 27 minutes, a task that required more than three times the number of Hopper chips. This performance in training large language models, which can involve trillions of parameters, underscores Nvidia's continued leadership in high-performance AI hardware, as only Nvidia and its partners submitted data for this specific large model training benchmark. The findings are particularly relevant as the number of chips needed for system training remains a key competitive factor, despite growing market focus on AI inference. Furthermore, an industry trend highlighted by CoreWeave suggests a shift towards assembling smaller, specialized groups of chips into subsystems for distinct AI training tasks, potentially accelerating the training of multi-trillion parameter models and influencing future infrastructure design. While Advanced Micro Devices (NASDAQ:AMD) was also mentioned in the MLCommons release, specific comparative data for AMD on the Llama 3.1 405B model was not detailed in this context.

Key Decisions for Investors

• Investors should consider the sustained technological lead demonstrated by Nvidia's Blackwell chips, which significantly enhances AI training efficiency and reinforces its competitive position in the high-performance AI hardware market.
• Monitor developments regarding the industry shift towards smaller, specialized chip subsystems for AI training, as this could influence demand dynamics and infrastructure build-out strategies for AI datacenters.