At Hot Chips 2026, companies unveiled new processors and discussed related topics, such as memory and networking. Large companies dominated the agenda, and several had multiple speaking slots. Below are observations from the conference, covering the majority of the program.
Quick links:
- Memory issues
- IBM Arm-compatible Z series
- Arm AGI CPU
- Fujitsu Monaka
- Intel Diamond Rapids
- Intel Wildcat Lake
- Intel Crescent Island
- SambaNova SN50
- Cerebras CS-4
- Meta MTIA
- Google TPU v8
- Microsoft Maia
- OpenAI Jalapeno
How Are Memory Vendors Tackling HBM Thermal and Density Challenges?
High-bandwidth memory (HBM) scaling faces thermal bottlenecks above 16-die stack heights, alongside die-density limits from through-silicon vias (TSVs). We expect vendors to address these issues by stacking DRAM directly onto processing units to maximize memory bandwidth and reduce I/O power.
Memory was an important subtext of many presentations and was the theme of the Day 0 morning session. Delivered by the three largest memory vendors, presentations in that session focused on HBM. The biggest impediment to increasing the capacity of HBM devices is heat dissipation, and stack heights beyond 16 are particularly challenging. A secondary issue is per-die memory density; through-silicon vias (TSVs) occupy area that would otherwise contain memory cells.
In terms of total area in an HBM-enabled XPU, memory occupies more silicon real estate than logic. Custom HBM, which replaces the base logic and I/O die in an HBM stack with one incorporating functions otherwise located in an XPU (such as the memory controllers), takes advantage of excess space in the base die. For memory vendors, it also offers a way to differentiate their products, that is, to try to raise prices. A subsequent development would be to eliminate the base die and place the memory stack on top of the XPU. Both of these approaches, however, increase the heat generated at the stack base.
Nonetheless, we expect stacking memory on XPUs to be more common. It enables dividing a memory chip into smaller arrays with separate interfaces to the logic above or below, greatly increasing memory bandwidth. At Hot Chips, D-Matrix discussed how it places DRAM atop its accelerator and associated techniques to reduce I/O power, improve reliability, and fully utilize the DRAM capacity. Cerebras revealed that its roadmap calls for stacked DRAM in the generation following its next wafer-scale engine.
How Are CPU Architectures Shifting?
Server CPU architectures at Hot Chips 2026 emphasized specialized workloads, native Arm compatibility, and chiplet-based designs. IBM introduced Arm binary execution on Z-series mainframes, while Fujitsu, Arm, and Intel showcased high-density, power-optimized server processors.
IBM Adds Arm Compatibility to Mainframe Microprocessors
Day One opened with IBM disclosing that its next Z-series mainframe CPU would natively run Arm binaries, a move that the company had telegraphed in an overlooked April press release. We don’t expect this to attract new customers to IBM mainframes, but it will allow them to consolidate general mainframe-adjacent workloads on IBM Z machines. Underscoring the snubbing of the IBM-backed Power Architecture, the speaker stated that the company considered no alternatives to implementing Arm compatibility. Although Arm has wider adoption, Canonical, the company behind the Linux distribution Ubuntu, reported in its talk that Arm and Power have a similar universe of tested Linux packages. By contrast, RISC-V has fewer than 10% of the number of tested packages, largely because Canonical has too few RVA23-compliant computers.
Arm vCPU AGI Aims at AI Agents
Other Arm-themed talks came from Arm, Nvidia, and Fujitsu. Arm presented the AGI processor we previously covered. Despite its name and positioning as purpose-built for agentic AI, it appears to be a multicore processor for consolidating workloads and vCPU cloud offerings. At 100 ns, DRAM latency is low for a recent CPU, but the AGI doesn’t support throughput-doubling MRDIMMs like AMD Venice (Epyc) and the forthcoming Intel Diamond Rapids (Xeon).
Fujitsu Monaka Reduces Power
We also previously covered the Nvidia Vera, and the company’s talk sheds no additional light on the chip beyond the company’s past disclosures. Fujitsu, however, unveiled its new Monaka processor, its first disclosed CPU since the A64FX presented at Hot Chips in 2018. Whereas that microprocessor targeted supercomputing, Monaka is for data centers. For that reason, unlike the A64FX, the new design has no HBM, implements 256-bit instead of 512-bit vector data paths, and has no disclosed high-speed interchip interconnect. However, Fujitsu stated it’s developing a next-generation Monaka featuring Nvidia’s NVLink interface for an upcoming HPC system.
Monaka implements a narrower microarchitecture than other CPUs. For example, it provides only four scalar integer units, two vector (SVE2) units, and two load/store units. Moreover, even the fastest-clocking 144-core model runs at only 2.9 GHz. By contrast, the AMD Epyc Venice is available in a 168-core model running at a similar base speed but capable of boosting to 3.7 GHz while operating in the same 500 W power envelope. Fujitsu withheld performance estimates for scalar integer code, but we expect Monaka to trail its peers.
To conserve power, Fujitsu implemented unusual caching techniques. Monaka caches floating-point registers to avoid lighting up the large register file, expecting matrix-multiplication routines to hit the cache. Similarly, the chip has a unit to serve common predicate patterns instead of reading predicates from their associated register file. Most significantly, Monaka operates at a 30% below standard voltage to reduce dynamic power by 50%.
Fujitsu constructs Monaka from chiplets, mixing process technologies and stacking dice, as Figure 1 shows. Fabricated in a TSMC N2 process, each compute die sits on an N5-based SRAM attached using hybrid bonding. The SRAM die also integrates voltage regulators placed below high-power function units, such as those for fused multiply-add (FMA) operations, for better efficiency and performance. The SRAM die connects through a silicon interposer to a common I/O die, also implemented in a TSMC N5 process.
Intel Adopts AMD-Style Chiplets for Diamond Rapids
Intel is constructing its next-generation Xeon (Diamond Rapids) similarly, the company revealed. Previously, the company divided Xeon into chiplets as Arm has done with the AGI processor, placing computing cores, caches, and memory interfaces on the same die. Diamond hybrid-bonds computing and cache dice, interfacing the latter to a pair of “fabric hubs.” These I/O dice (IODs) also include accelerators such as the QAT encryption engine. Whereas Intel connects each computing die to both fabric hubs, AMD connects its computing dice only to a single IOD, necessitating an IOD-IOD hop to reach some DRAM.
Diamond scales to 256 cores and 1.28 GB of cache, finally matching Epyc’s integration levels. Intel withheld performance and microarchitecture details, but it disclosed Diamond will be the first chip supporting APX instructions. These extensions raise the number of architecture registers to 32 (typical of RISC instruction sets), add predicated instructions similar to those in the 64-bit Arm ISA, and can suppress flags (akin to RISC-V’s flagless architecture). We expect AMD to adopt these extensions. Suggestive of a design that has been in progress for several years, Diamond implements Intel’s AMX matrix-math extensions but not the newer ACE instructions, which support low-precision block formats such as FP8.
Intel fabricates Diamond using its new 18A-P process, which improves power, performance, and likely yield compared with 18A. The company has yet to disclose specific models, performance figures, or an availability date. We expect production to ramp about two quarters after AMD Venice.
Intel Panther Begets Budget-Priced Wildcat
Intel also presented the Core (non-Ultra) Series 3 (Wildcat Lake) processor derived from Panther Lake, detailing cost and power optimizations for low-cost laptops. Beyond trimming features, Intel substituted a conventional interposer for a silicon substrate to connect chiplets. The latter led the company to employ UCIe die-to-die interconnect, which entailed integrating larger transceivers, partially offsetting savings from using the interposer. Further decreasing cost, dice with processing defects affecting a CPU, GPU, or NPU core can be sold as feature-reduced models. Unfortunately for Intel, skyrocketing memory prices render a low-cost CPU moot.
How Do Custom Datacenter NPUs Compare with Modern GPUs?
Datacenter NPUs from Meta, Microsoft, Google, and OpenAI compete directly with Nvidia GPUs with novel architectures focused on efficient data movement when executing large language models.
Memory Capacity Distinguishes Intel Crescent Island GPU
Hot Chips organizers also granted Intel a slot to present the upcoming Crescent Island GPU. Although positioned for computing use, it’s for add-in-card use instead of rack-scale designs. The design adapts the GPU from the Panther Lake PC processor, adding support for formats such as FP4 for AI and improving FP64 for HPC. Intel enlarged the XMX units to better handle AI and HPC workloads and doubled register-file size. Crescent interfaces to LPDDR5X memory instead of HBM to contain cost.
As an add-in board, Crescent will compete with AMD Instinct MI350P cards and Nvidia cards based on desktop or older data-center GPUs. The Intel chip has the advantage of supporting more memory, up to 480 GB, and requiring only 350 W.
SambaNova SN50 Seeks to Complement GPUs
Intel highlighted the SambaNova SN50 alongside the new GPU, indicating potential development of a reference design that allocates LLM prefill and decode functions to Crescent and the SN50, respectively. Although Intel and CEO Lip-Bu Tan have invested in SambaNova, the startup has had little market traction. At the same time, Intel is working on a GPU to compete with the Nvidia Rubin and its successors. The company withheld Crescent’s performance, production schedule, and even its manufacturing process. Therefore, we believe Intel may scupper Crescent products.
For its part, SambaNova presented its SN50 chip, focusing on how its dataflow architecture can keep HBM interfaces near 80% utilization and compute units near 40%. The result is that they’re more efficient, particularly at high token rates on large models when performing an LLM’s decode stage. A heterogeneous, disaggregated design employing four Nvidia H200 GPUs for prefill and 16 SN50s for decode achieved 763 tokens per second, a high rate for an economical hardware setup.
Cerebras CS-4 Goes Backpacking
AMD and Nvidia presented their most recent data-center GPUs but revealed little beyond previous disclosures. However, Cerebras, Meta, Microsoft, and OpenAI all revealed details of their NPUs. Cerebras discussed its new CS-4 system. Its WSE-3 Turbo doubles the clock rate of the CS-3’s wafer-scale engine, WSE-3, and updates the system design. The new design supports three wafers instead of one, multiplying raw performance sixfold in combination with the clock boost. The system also has low-latency wafer-to-wafer links and a new wafer I/O module.
Cerebras also promised an accelerated new-product cadence, showing a roadmap to C6-5 next year followed by CS-6. The latter will stack a DRAM wafer on the computing wafer, which will help the system support large models and context sizes. We speculate that these forthcoming systems will use the same rack as CS-4 with a new wafer-scale engine.
Meta MTIA 400 Takes on LLMs
Meta presented the MTIA 400, its fourth-generation NPU. Whereas the original MTIA and the MTIA 200 achieved limited deployment and focused on deep-learning recommendation-model (DLRM) inference, the MTIA 300 is now being deployed and also supports DLRM training. The MTIA 400 adds generative AI (LLM) capabilities, as we previously discussed. It quadruples peak matrix-multiplication throughput, supports microscaling (e.g., FP4) formats, and provides 1.2 TB/s scale-up and 100 GB/s PCIe-based scale-out networking. New DMA and collective engines facilitate scaling. At 12 PFLOPS per chip, peak FP4 throughput is much less than that of Nvidia Rubin (35 PFLOPS for training). Meta, however, rates the MTIA 400 at 667 W, less than a third of what Rubin requires.
At a high level, the computing architecture is similar to the MTIA 200, comprising an array of processing elements (PEs). Each PE integrates two RISC-V cores and various offload engines and a local memory. New messaging elements (MEs) based on RISC-V cores and accelerator units execute collective operations, transferring data, converting between types, and interacting with network interfaces. The MEs obviate collective engines in the PE or the Ethernet switches in the scale-up network, and they can offload some functions that a host CPU would otherwise handle.
Google TPU v8i Adds Collective Engine
Similar to the MTIA ME, a collective acceleration engine is a new feature in the Google TPU v8i. In its presentation, the company stated that this engine eliminates storing data in HBM, thus consuming vital memory bandwidth, and reduces latency associated with collective operations. Google also disclosed its design team employed AI tools to develop the v8 TPUs, reducing power and area by 5% or better, depending on the function.
Microsoft Maia 200, a Maia 100 Mulligan
Microsoft presented its second-generation NPU, the Maia 200. It’s another design based on a grid of computing elements, each including tensor, vector, and control units plus local memory. Whereas a MTIA 400 tray comprises four chips linked to a PCIe switch, which also connects to a host and an Ethernet interface, a Maia tray also has four NPUs but fully interconnects them using Ethernet. Other on-NPU Ethernet interfaces connect to switches to join NPUs into a 6,144-unit cluster. Much of Microsoft’s talk addressed software-controlled data movement. Employing software to control the placement, transfer, and computation of data to maximize performance, however, is common to all GPUs/NPUs. Our concern with the Maia 100 was that it seemed more like an R&D experiment than a product that will roll out en masse, and the Maia 200 doesn’t exhibit any capabilities that raise our expectations.
OpenAI Delivers Jalapeno Benchmarks
OpenAI had the most anticipated talk of the conference, presenting its Jalapeno NPU. Reiterating that it required only nine months to progress from initial RTL to tape out (freezing RTL about halfway through), OpenAI discussed how it used AI tools in development, enhancing an approach presenter Richard Ho has crafted over the past 10 years. In particular, the tools assisted with scheduling and orchestrating data movement and computation. Regarding software development, AI models also assisted with optimizing kernels and produced code faster than that written by experts.
OpenAI disclosed few architectural details. However, the company indicated that it divides its design into 64 logical slices, associating each with an HBM slice. This provides a high-bandwidth, low-latency localized region. A collective network connects slices within Jalapeno, delivering greater bandwidth and lower latency than the chip’s network-on-chip, which is more flexible and provides connectivity to scale-up network interfaces. As with competing designs, the goal is to minimize the time computing units stall waiting for data.
Significantly, OpenAI presented benchmarks comparing Jalapeno with Nvidia Grace-Blackwell. Running GPT-OSS 120B, DeepSeek R1, and Kimi K2.5 1T, the new chip decodes significantly faster, as Figure 2 shows. The absence of end-to-end throughput data is suspicious, but OpenAI showed Jalapeno and has much lower end-to-end latency. Moreover, Jalapeno is significantly more power efficient. Assuming the quickly designed chip functions reliably, it threatens Nvidia’s position at OpenAI, its largest customer. Meanwhile, OpenAI will soon tape out Jalapeno’s successor.
Bottom Line
Established companies dominated Hot Chips 2026, with several occupying multiple speaking slots, displacing startups on the agenda as much as they keep them at bay in the market. The strongest challenge to the incumbents is coming from hyperscalers and AI laboratories, which are hugely profitable and well funded, respectively. Jalapeno, if it or its successors prevail, will mark the unusual turning point in which chip design is revolutionized not by a semiconductor company but by a customer of those companies.