Reuters is reporting OpenAI will tape out its first AI processor (NPU) later this year. Pilot deployments will focus on inference, but the NPU can also perform training. Broadcom is aiding OpenAI with chip development, and TSMC will fab it in a 3 nm process. Richard Ho helms the project, coming to OpenAI in 2023 after a brief stint at Lightmatter (photonic interposers) and an extended time on Google’s TPU (NPU) team.
The TPU connection significantly boosts our confidence in the project. The search giant was the first hyperscaler to develop a proprietary NPU and was subsequently joined by Amazon with their Trainium and Inferentia projects, Alibaba (HanGuang), Meta (MTIA), Microsoft (Maia), and Tesla (Dojo). Having run production workloads on multiple TPU generations, Google is the most successful of these companies.
Owing to the challenge of developing a monster NPU, the necessary software, and the systems that use it, it’s unclear whether some of these proprietary projects will ever be deployed at scale. In addition to make versus buy, companies could consider buying one of the abundant AI-chip startups—something Meta is reportedly considering. Microsoft Maia’s future is particularly cloudy. The software behemoth has been OpenAI’s data-center partner and nominally developed Maia to run its models. Microsoft remains committed to building data centers supporting OpenAI but isn’t part of Stargate, a plan consistent with OpenAI CEO Sam Altman’s previously expressed ambitions.
Bottom Line
The motivations to make an NPU are clear:
- Racing to build ever bigger data centers to achieve the AI breakthrough that will transform the world, OpenAI and its competitors are hampered by the high acquisition cost and limited availability of Nvidia GPUs.
- Nvidia GPUs aren’t the ideal AI-processing architecture, delivering less throughput and requiring more precious power than a purpose-built NPU.
However, the Nvidia solution is here now and known to work. It has proven hard to replace, particularly for training. Thus, OpenAI’s homegrown chip is a side effort that may lead to future opportunities and possibly help the company negotiate better terms from Nvidia. Selling all it can make, Nvidia is hardly threatened for now.