Google uses speculative decoding to speed up Gemma 4 by 3x

Google DeepMind says its new multi-token prediction drafter models can deliver up to a 3x speedup in inference throughput for open-weight Gemma 4 models without degrading output quality or reasoning. The update improves one of the key cost drivers in AI workloads: inference efficiency. The news is positive for Google’s open-source AI stack, but likely modest in near-term market impact.

Analysis

This is less a product story than a margin-structure reset for the AI stack. If open weights can legitimately deliver ~3x more tokens per dollar on the same hardware, the competitive edge shifts away from raw model quality and toward who can compress inference cost fastest; that is structurally favorable for hyperscalers with proprietary silicon and for distributors of open models that can monetize usage at lower price points. The second-order winner is likely enterprise adoption: cheaper inference reduces the threshold for always-on copilots, agent workflows, and high-volume batch use cases that were previously uneconomic.

For GOOGL, the immediate benefit is strategic, not just financial. Lower inference cost on Gemma improves Google’s ability to seed the open ecosystem, preserve developer mindshare, and create a de facto benchmark for efficient deployment, which can later funnel workloads into GCP, Vertex, and TPU-based serving. The risk is that this compresses differentiation for rival open-model hosts and inference middleware vendors, because if the underlying model becomes materially cheaper to run, pricing power migrates up the stack to cloud and compute providers rather than model wrappers.

The market may be underestimating how quickly this changes buyer behavior over the next 6-18 months. Once teams can show a credible 2-3x throughput gain without quality loss, procurement cycles usually accelerate because CFOs can justify broader rollout on unit economics alone. The main reversal risk is that gains prove hardware- or workload-specific, or that competing open models neutralize the advantage with their own efficiency releases, which would turn this into a short-lived optics event rather than a durable platform edge.

Contrarian take: the headline is mildly positive for Google, but the bigger implication may be bearish for AI infrastructure names whose valuation assumes ever-rising inference intensity. If more models become cheaper to serve, the long-duration thesis for some inference-only beneficiaries weakens, while the beneficiaries with control over distribution, cloud, and custom silicon gain leverage. This argues for favoring platform incumbents over pure-play model-adjacent or middleware exposures until the elasticity of demand becomes visible in usage data.

Key Decisions for Investors

• Long GOOGL vs. basket short of inference-layer beneficiaries (select AI middleware / model-hosting names) over the next 3-6 months; thesis is margin compression at the application layer and share capture by the platform owner.
• Add GOOGL on any post-announcement drift rather than strength chase; the setup is a gradual monetization story, with the real upside likely appearing in 2H25 usage metrics, not immediately in revenue.
• Initiate a relative-value long GOOGL / short GPU-semiconductor beta pair for 3-6 months; if inference efficiency keeps improving, incremental spend may slow even as model usage rises, compressing the “tokens = more chips” narrative.
• Avoid chasing pure-play open-model hosting names for now; wait for evidence that throughput gains are translating into durable paid workload expansion rather than one-time enthusiasm.
• If you want convexity, use call spreads on GOOGL 6-12 months out; the risk/reward improves if cheaper inference materially expands enterprise adoption, but downside is limited if this remains a developer-only feature.