How Microsoft’s new chip for AI could disrupt big tech — especially Nvidia, AMD and Intel
/During a Microsoft Ignite event this morning the company took the wraps off one of the more important reveals in the chip market in a long time. Microsoft now has both a custom-designed AI processing chip and a custom Arm-based CPU to add its growing stable of products to help it vertically integrate its services and solutions.
The Arm-based processor that will be used in various places of the Azure cloud compute infrastructure, called the Azure Cobalt 100, has 128 cores and is built on the Arm Neoverse N2 platform. These CPUs provide as much as 40% better performance per core compared to the previous Arm-based CPUs.
The star of the show is the Azure Maia 100 AI accelerator chip. This chip is built for both training and inference (creating the models and then utilizing them for work) and has been collaborated on with OpenAI, the AI giant behind ChatGPT. The Maia 100 isn’t a traditional GPU that you see from Nvidia or AMD, but rather a custom design meant specifically for AI processing, which in theory can give the chip an advantage in performance or efficiency, or both. It is built on a TSMC 5nm manufacturing process that isn’t the most advanced, and thus, also isn’t the most expensive.
Microsoft’s Maia 100 will be targeted for use on workloads like Bing, the various Copilot branded features, ChatGPT, and other generative AI systems. Microsoft did mention it being specifically tailored for LLMs (large language models, the type of AI that powers ChatGPT) but didn’t say if that means it has deficiency in another areas of AI compute. As with nearly all custom designed chips, the specific targeting to a particular workload can allow for big advantages on that workload, but at the expense of more general computing.
Custom AI accelerators and CPUs mean Microsoft can now fully vertically integrated its data center and fully own its future if it so chooses. By owning the silicon, the servers, the software, and the services that run on top of it, it has control over every step between it and the customer. This is the strategy that Apple has taken on consumer products, owning the silicon design to the final sale to the end user, and everything in between. The result is best in class user experiences and true market differentiation.
There are other deep dives on the tech of these new chips and the server infrastructure the company built for them, but I want to look at how these announcements could impact other chip companies both competing directly, and implications of the MFST custom silicon path generally.
AI Chip Competitors
The obvious first name that comes up on impact to business with the release of the Azure Maia 100 chip is Nvidia. No other company has benefited from the rise in AI and the need for compute power for AI than the green giant. Nvidia had more than $10B in revenue in its data center group in Q2 that encompasses the sale of its AI chips like Hopper and Grace, a 141% increase over the previous quarter. Nvidia is far and away the market share leader in GPUs for AI training and inference, in the enterprise and cloud. And Microsoft was one of, if not THE, largest buyer of Nvidia hardware.
So while it seems very likely that Nvidia will face a short term hit for orders from Microsoft for its GPUs, there is more than enough demand in the market from others to make up for it. Nvidia has been selling out its chips months in advance, with many reports stating that 2024 was already sold out. Any chip availability because of Maia will quickly be consumed by others.
And it’s worth noting that Nvidia’s CEO Jensen Huang was on stage during the Microsoft event this week, clearly an indicator of the important of the company to the future of Microsoft’s AI strategy even with these custom chips released.
Next in line to the GPU and AI chip throne is AMD. They are a distant second place, but Microsoft did announce today official adoption of the company’s MI300X chips for some new Azure could instances. CEO Lisa Su has been bullish on the quick ramp in revenue for this chip to $1B – and it seems Microsoft might be the ones to drive most of it.
Microsoft and AMD have a great working relationship. The two have collaborated on custom silicon for the Xbox gaming consoles and previous generation of Surface PCs, and Azure is a big consumer of AMD’s EPYC data center CPUs. That partnership likely leads to things like the MI300X AI chip integration and could help AMD find a bigger footprint in Microsoft’s AI strategy even with the creation of the custom Maia 100 AI accelerator.
Intel has been struggling to catch up in relevancy with the AI hardware race. It’s attempt to build a GPU to counter Nvidia and AMD had only small levels of success in the data center space, with the Intel Data Center GPU coming in late to the party. For companies like Microsoft, but also Google and Amazon and those big infrastructure players, going to a third or even fourth source for GPUs seems like a stretch. These large CSPs (cloud server providers) now have their own custom silicon, Nvidia’s GPUs with its 10-year history and software scale, AMD GPUs as the credible “next man up” as the third option. I think it will be a struggle for Intel to find significant design wins with its GPUs or even its Gaudi AI accelerators.
The market for AI hardware is growing significantly, so even though we might see Nvidia lose some percentage of market share in 2024 and 2025, with much of that going to AMD and custom silicon options like the Azure Maia 100, I don’t expect any regression in the revenue for the AI powerhouse.
Secondary implications of Microsoft’s direction
There are other interesting implications to Microsoft’s continuation down the custom silicon path, both with the Maia 100 AI accelerator and Cobalt 100 Arm-based CPU. First and foremost, as I wrote about last week, Arm continues to prove to the market that it will continue to grow and expand the footprint of its architecture, in comparison to the x86 legacy designs from Intel and AMD. Nvidia and now Microsoft have now bet on their own custom silicon CPUs to help run power efficient data centers and diverse workloads.
For a company like Qualcomm that is partnering with Microsoft on its Snapdragon X-series of processors for the upcoming AI PC launch this could be a mixed blessing. On the one hand it validates the usage of the Arm-architecture for broader adoption in the technology space. But it also brings up the question around verticalization of the consumer market. If Microsoft is willing to start down that path with the data center, might it decide to build its own consumer-facing CPU at some point to displace the current players, or will they continue to let the silicon diversification strategy to play out?
A final consideration worth mentioning around these new custom chips is around production. TSMC is the producer of basically all the top processors and GPUs from Nvidia, AMD, Qualcomm, Apple, and now Microsoft’s custom silicon chips (along with those from Amazon, Google, etc.). Allocation of the that limited resource has been a big driver in who wins the AI race – whoever has the rights to the most silicon wafers has the chips to sell into the market.
If Microsoft struggles to compete with the orders from Nvidia and Apple, will it be able to scale? Or, is this an opportunity for the foundry services portion of Intel to really benefit from this market even if its AI product portfolio struggles? It seems like that if Intel can get its foundry house in order sooner than later, there is ample opportunity.