Google's Latest AI Chip Puts the Focus on Inference

Google is launching its seventh-generation Ironwood TPUs and new Arm-based Axion virtual machines for Google Cloud, targeting significant cost reductions and performance enhancements for AI inference and agentic AI workloads. Ironwood offers substantial performance improvements, positioning Google to capture demand in the emerging 'age of inference' where AI model deployment is paramount. This strategic investment in custom silicon, highlighted by Anthropic's expanded TPU utilization, aims to bolster Google Cloud's competitive standing against rivals like AWS and Azure, building on its recent 34% Q3 revenue growth to $15.2 billion.

Analysis

Google has announced the availability of its seventh-generation Ironwood Tensor Processing Units (TPUs) and new Arm-based Axion virtual machine instances for Google Cloud customers. The Ironwood TPUs deliver a significant 10X peak performance improvement over TPU v5p and over 4X better performance per chip compared to TPU v6e for both training and inference workloads. These advancements are designed to lower costs and enhance efficiency for critical AI inference and agentic AI workloads, marking Google's custom silicon as its most powerful and energy-efficient to date.

This strategic product launch positions Google to capitalize on the emerging "age of inference," where the industry's focus is shifting towards high-volume AI model deployment rather than solely on training. Google anticipates near-exponential growth in compute demand as AI adoption accelerates, and the efficiency of its new TPUs was a key factor in Anthropic's expanded commitment to utilize 1 million Google TPUs for its operations.

Google Cloud aims to leverage these custom silicon innovations to strengthen its competitive standing against rivals like AWS and Azure. The segment demonstrated robust performance in Q3, with revenue growing 34% year-over-year to $15.2 billion and achieving an operating income of $3.6 billion, representing a 24% margin. This growth rate surpassed AWS's 20% increase, indicating Google's potential to gain market share in the rapidly expanding AI infrastructure market.

Key Decisions for Investors

• Investors should consider the long-term implications of Google's aggressive investment in custom AI silicon and its strategic focus on the 'age of inference' as a potential differentiator in the competitive cloud computing market.
• Monitor Google Cloud's continued revenue growth and operating margin expansion, as the adoption of Ironwood TPUs and Axion VMs could accelerate its market share gains against AWS and Azure.
• Evaluate the broader market shift towards AI inference workloads and the efficiency demands of large AI model deployments, as companies like Google providing optimized infrastructure are poised for significant growth.