NVIDIA TensorRT Boosts Stable Diffusion 3.5 Performance on NVIDIA GeForce RTX and RTX PRO GPUs

NVIDIA and Stability AI have collaborated to optimize Stable Diffusion 3.5, reducing VRAM consumption by 40% through FP8 quantization via NVIDIA's TensorRT, enabling broader accessibility across RTX GPUs; performance has also been doubled with the NVIDIA TensorRT software development kit (SDK). Furthermore, NVIDIA has released TensorRT for RTX as a standalone SDK, featuring just-in-time (JIT) on-device engine building and a significantly smaller package size, facilitating seamless AI deployment to RTX AI PCs and allowing developers to create generic TensorRT engines optimized on-device in seconds.

Analysis

NVIDIA's collaboration with Stability AI to optimize the Stable Diffusion 3.5 (SD3.5) model underscores its commitment to enhancing AI accessibility and performance on its GPU architecture. The application of FP8 quantization via NVIDIA TensorRT has significantly reduced VRAM consumption for SD3.5 Large by 40% to 11GB, a critical development as AI models grow in complexity and memory demand; this reduction effectively allows five GeForce RTX 50 Series GPUs to run the model from memory compared to just one previously. Concurrently, TensorRT optimizations have doubled performance for SD3.5 Large and Medium models, with SD3.5 Large achieving a 2.3x performance boost compared to BF16 PyTorch implementations. Furthermore, the release of TensorRT for RTX as a standalone SDK, featuring just-in-time (JIT) on-device engine building and an 8x smaller package size, simplifies AI deployment for developers targeting over 100 million RTX AI PCs and integrates with Microsoft's new Windows ML framework. These advancements, including support for FP8 on GeForce RTX 40 Series and Ada Lovelace RTX PRO GPUs, and upcoming FP4 support on Blackwell GPUs, reinforce NVIDIA's dominant position in the AI hardware and software ecosystem by making its platforms more efficient and developer-friendly for demanding generative AI workloads. The planned release of SD3.5 as an NVIDIA NIM microservice in July further aims to streamline model deployment for creators.

Key Decisions for Investors

• Investors should view these developments as reinforcing NVIDIA's technological leadership and expanding market opportunity in AI, particularly within the growing AI PC segment and on-device AI processing.
• Consider the positive implications for NVIDIA's GPU sales and software ecosystem adoption, as reduced VRAM requirements and increased performance make advanced AI models accessible to a broader user base on NVIDIA hardware.
• Monitor the uptake of the new TensorRT for RTX SDK by developers and the subsequent impact on the attractiveness of the RTX platform for AI applications, which could further solidify NVIDIA's competitive moat.