TL;DR: Microsoft has announced Maia 300, the third generation of its in-house AI chip, and says it will become the primary system for running AI in Microsoft products starting in 2027. Microsoft claims Maia 300 delivers 40% more tokens per megawatt than leading GPU-based systems and up to seven times the rack-level throughput of Maia 200. Those numbers come from Microsoft and can’t yet be independently tested, but the direction is clear: Microsoft wants to depend far less on Nvidia for AI inference.
What Microsoft announced
In a post on the official Microsoft blog on October 12, cloud chief Scott Guthrie said the company will scale its Azure Maia family to “multi-gigawatt deployments starting in 2027, making Maia the primary inference system for Microsoft products and services.” The announcement coincided with the OCP Global Summit, according to StorageReview.
Maia 300 follows Maia 200, which Microsoft unveiled in January and which is already running production workloads including Microsoft Copilot, GitHub Copilot and Microsoft Foundry, as The Verge reported.
Maia 300 by the numbers
| Spec or claim | Maia 300 (per Microsoft and published coverage) |
|---|---|
| Purpose | Inference: running trained AI models, not training them |
| Design | Multi-chiplet, with separate compute, memory-controller and I/O chiplets |
| Process | Compute chiplets on TSMC’s N2P node |
| Memory per accelerator | 288GB of HBM4 across eight stacks, about 18TB/s of bandwidth |
| Power per accelerator | 950W TDP (reported by StorageReview) |
| Rack configuration | 96 accelerators in a 167kW rack, 27.6TB of HBM total |
| Efficiency claim | 40% more tokens per megawatt than leading GPU-based systems |
| Throughput claim | Up to 7x the rack-scale token throughput of Maia 200 |
| Availability | Production deployments begin in 2027 |
Why inference is the battleground
Training a frontier AI model is a huge one-off job. Inference, actually answering billions of prompts from users and AI agents, is the ongoing cost that keeps growing as products like Copilot spread. Modern workloads also look different from a couple of years ago. Long conversations, large documents and multi-step agents generate long contexts and many output tokens, which puts enormous pressure on memory bandwidth rather than raw arithmetic.
That is exactly where Maia 300 is aimed. Microsoft’s architecture deep dive describes the chip as optimized for “long-context, reasoning-intensive, and decode-heavy inference.” StorageReview’s analysis of the published figures points out that per-accelerator memory bandwidth rose about 2.5 times compared with Maia 200, while FP4 compute rose a more modest 37%. That trade-off makes sense for a chip whose main job is feeding data to the processor fast enough during token generation.
The architecture changes in brief
- Chiplets instead of one big die, which Microsoft says improves efficiency, scalability and cost.
- Locality-aware memory, so software can place data close to the compute that uses it.
- New narrow-precision number formats with hardware conversion, reducing the bytes moved for each token.
- Bigger scale-up domains, with 96 accelerators per rack and a “superpod” of up to 12,288 accelerators working as one system.
How it builds on Maia 200
Microsoft says Maia 200 already delivers up to 1.7 times more tokens per watt for built-in AI features used by hundreds of millions of people, according to its EMEA newsroom summary. Maia 300 doubles the number of accelerators per rack and, per Microsoft’s architecture post, offers more than 2.5 times the performance per watt of its predecessor. The quick cadence, two generations announced in one year, shows how seriously Microsoft is taking custom silicon.
What it means for Nvidia
Microsoft is one of Nvidia’s largest customers, and the Verge framed the announcement as making Microsoft less reliant on Nvidia’s GPUs. That does not mean Nvidia is being cut out. StorageReview notes that Nvidia and AMD hardware remain in Microsoft’s training fleet, and Maia is focused on inference.
Still, inference is where the volume is heading. Every major cloud provider now has its own custom AI silicon: Google with its TPUs, Amazon with Trainium and Inferentia, and Microsoft with Maia. Each in-house chip that handles a share of inference is work that does not go to a merchant GPU. Microsoft AI CEO Mustafa Suleyman told The Verge that “at that scale we need a platform that we own, and that we co-design with our models.”
Reasons for skepticism
The headline efficiency claim deserves caution. As StorageReview pointed out, Microsoft has not said which GPU, which model or which serving setup it compared against for the 40% figure. And because Maia runs only inside Microsoft’s own data centers and is not offered as a rentable Azure instance, outside researchers can’t benchmark it. Claims like “up to 7x” also describe best-case, full-rack configurations rather than every workload.
There is also timing. Production deployments are planned for 2027, so Maia 300 will be competing against whatever Nvidia, AMD and other chipmakers ship by then, not against today’s hardware.
What users might notice
Most people will never see a Maia chip, but they may feel the effects. If Microsoft can serve more AI responses per megawatt, it can scale features in Copilot, Teams, GitHub and its Dragon Copilot healthcare product with lower cost and potentially faster responses. Microsoft specifically cites those services as beneficiaries. Whether savings show up as lower prices or simply more generous usage limits is a business decision Microsoft has not discussed.
The bottom line
Maia 300 is Microsoft’s clearest statement yet that it wants to own the hardware behind its AI products. The design is aggressively focused on memory bandwidth for long-context and agentic workloads, and the company plans to make it the default inference platform from 2027. The performance claims are bold but unverified, so the real test will come when it runs at scale. More detail is expected at Microsoft’s Ignite conference.
The Teck Hustlers Editorial Team covers technology, business, finance and gaming news for readers who want the story behind the headline. Every article is researched, written and edited by our in-house team, checked against company announcements, filings and reputable reporting, and updated when new facts emerge. We focus on what each development means for founders, professionals, investors and everyday users. Have a tip or a correction? Reach us through our contact page.
