Having first disclosed the Grayskull AI accelerator four years ago, Tenstorrent started selling development boards on March 4. The Tenstorrent Grayskull e75 is a 75 watt PCIe Gen 4 card. Selling for $600, the e75 has a single 1 GHz NPU chip that integrates 96 Tensix cores and 96 MB of SRAM. The board also contains 8 GB of standard LPDDR4 DRAM. Requiring 200 W, the e150 card costs $800 and raises the clock rate to 1.2 GHz and increases the number of cores to 120. On-chip memory increases proportionally to 120 MB but the off-chip DRAM stays at 8 GB.
Notables
- Architecture—The Tenstorrent Grayskull architecture differs from other data-center AI accelerators (GPUs/NPUs). The arrayed Tensix cores contain several CPUs that feed a computing unit comprising vector and matrix engines. This structured granular approach should increase math-unit utilization, thereby improving performance per watt. Each Tensix also has 1 MB of SRAM, yielding a healthy on-chip memory total. Unlike other NPUs with big internal memories, Grayskull can interface to external memory. Unlike competing offerings with off-chip interfaces, Grayskull uses standard DRAM instead of costly HBM.
- Software is an Achilles heel for NPU and other processor challengers and a strength of the incumbents against which they compete. Tenstorrent offers two software flows. TT-Buda maps models based on standard AI frameworks like PyTorch and TensorFlow to the Tenstorrent hardware, and TT-Metalium gives developers direct hardware access and allows them to create libraries used by higher-level frameworks. Enabled by the Grayskull architecture, Metalium stands out for providing a more computer-like programming model and could appeal to customers with the resources to code at a low level.
- Power—From the beginning, Tenstorrent cited power efficiency as a differentiator, but we must see the new cards’ MLPerf results to evaluate efficiency. However, 75 W is relatively low and fits within standard PCIe and OCP power envelopes, suggesting a design like the e75 could be a good server add-in board where it would run inference workloads. If the company is serious about training, it needs to show off big OCP boards that can replace the Nvidia-based ones in service now as AMD did at the MI300 launch.
- Business model—In addition to Grayskull chips and boards, Tenstorrent also licenses its high-performance RISC-V CPU and its Tensix cores and is codeveloping chiplets with partners. We understand the desire to generate revenue and startups often change direction, but they usually benefit from focusing on solving a single problem.
- Capitalization—Tenstorrent closed a $100 million up round in August, led by Hyundai and Samsung. Strategic investors and a pile of cash should propel Tenstorrent for a few years.
- People—Noted engineer and CPU architect Jim Keller heads Tenstorrent, having joined the company as CTO at the end of 2020. Tenstorrent’s founder Ljubisa Bajic and engineering VP Drago Ignjatovic left in 2023 to start another AI chip company, Taalas. Brain drain isn’t a good sign but could reflect defectors’ desire to focus specifically on AI processing. Jim Keller’s name alone is enough for onlookers to be optimistic about the company.
- Timing—It’s been four years since the original Grayskull was revealed. With dev boards just becoming available now, it will be at least a year before large-scale deployments. Realistically, those aren’t going to happen with the current design.
Competition
Nvidia has gotten more entrenched, layered on more software, refined its architecture, introduced new products, and generated billions in profit in the four years since Tenstorrent unveiled Grayskull and first touted its advantages over GPU-derived architectures. Although Tenstorrent has recently secured additional funding, it’s hard to see them as a threat to Nvidia. The small impact that Intel’s non-GPU Gaudi architecture has had despite that company’s best efforts shows how tough it is to challenge an incumbent, even in a market’s early stages.
Customers
Customers with the resources and motivation to deviate from the Nvidia norm, particularly those that can benefit from Metalium, will find Tenstorrent Grayskull an unusual (if not unique) value. The new dev boards will enable them to try it out with little capital cost. The bulk of the data-center market, however, has proven satisfied with Nvidia and finds moving away from GPU-derived architectures a step too far.
Bottom Line
Tenstorrent is an intriguing but unfocused company. Its nominal core business has taken years to field a product—a product that’s appealing to only a niche. However, if that niche includes even a single major cloud service provider or an automaker, the company could quickly get a massive purchase order or be acquired.