Microsoft’s next-gen [there was a first gen?] proprietary AI accelerator (NPU) is falling behind schedule, according to various sites citing The Information. Called Maia or Braga, the in-house design would reduce the company’s reliance on Nvidia and could be more cost- and power-efficient.
Whereas Google has deployed several generations of its TPU AI accelerator and Amazon has made a few Trainium versions, Meta and Microsoft haven’t achieved the same technical success. Meanwhile, merchant-market leader Nvidia is churning out new GPUs and investing in software, where it already has a huge ecosystem. A two-quarter delay isn’t unusual for a new chip but is significant in the context of the rapidly developing data-center AI business.
Although AI accelerators replicate much of their logic, they nonetheless entail leading-edge technology. As numerous startups have argued, a GPU-based architecture has shortcomings. Nvidia’s architectural refinement and software ecosystem, however, atone for the original sin of its GPU basis. Moreover, the company has leading-edge interconnect and system design, critical technology for scaling AI acceleration.
We have found that domain-specific processors that hardware teams architect independently frustrate developers, possessing performance gotchas that a coder would’ve flagged and other quirks. Conversely, given the chance to develop a processor, software/algorithm developers will create something architecturally logical but ignorant of real-world constraints.
Without details, it’s impossible for us to blame Microsoft’s delay on these issues, but the company’s Hot Chips 2024 presentation left us underwhelmed. The recent reports attribute the delays to staffing problems and design changes. The company doesn’t need to run faster but instead ensure it’s going in the right direction if it wants to catch the Red Queen—er, Green King.