OpenAI is not deploying inference services on Google TPUs, according to Reuters. The news service previously reported that the ChatGPT developer had agreed to use Google Cloud. Subsequently various free-to-read sites parroted The Information, saying OpenAI would employ the TPU.
Running OpenAI services on the TPU would reduce its reliance on Nvidia GPUs hosted by partner Microsoft and companies such as CoreWeave and Oracle. Google, meanwhile, would monetize its AI-hardware investment.
Nvidia has uniquely benefited financially from the rise of OpenAI and its competitors, which depend on Blackwell and its other GPUs, paying dearly for them. Undoubtedly, OpenAI is keen to mitigate supplier risk, reduce costs, and access more AI-acceleration capacity than Nvidia can supply.
Inference (model execution) is easier to port to a new architecture than training and development, which in almost all cases depends on Nvidia’s Cuda ecosystem. If OpenAI started training on the TPU (as Apple is said to have done and as Google itself does), that would be a man-bites-dog story. Nonetheless, OpenAI would entail costs to adopt the TPU for inference, and it’s unclear if Google, which offers the competing Gemini large language models, would be willing to offer OpenAI TPU access.
Like many of its competitors, OpenAI is developing its own AI accelerator (NPU), which—if successful—would yield even more independence than diversifying by using the TPU. CEO Sam Altman has also expressed interest in AMD’s forthcoming MI400 GPU. These various efforts might not pan out, but OpenAI is opening up options.