Skip to content

BRK441 Breakout Tue 17 Nov 16:30 PST

Inside Azure Maia: Purpose-built for AI inference at scale

Vikram Vemulapalli, Andrew Wall

Summary

From the recording

Microsoft said Maia 200 is live in data centres running production workloads for its first-party services, delivering about 40 to 70% more tokens per megawatt. Maia 300, announced a few weeks earlier, uses a multi-chiplet design with 8 HBM stacks and 18 TB/s of HBM bandwidth. Racks move from 48 accelerators to 96, giving 5x aggregate rack-level HBM bandwidth, and up to 12,000 Maia 300s can connect. Microsoft expects 7x rack-scale token throughput versus Maia 200. Maia 300 is expected next year. Caveat: the baseline for the 7x figure is unclear; customer access, pricing and regions were not stated.

Session description

Microsoft's catalogue description

Building world-class AI infrastructure requires more than a great chip. Learn how Azure Maia combines custom silicon, systems, networking, and software into a unified platform optimized for AI inference and see how end-to-end co-design translates into better customer outcomes.

← All sessions