Nvidia is mounting a charm offensive for its Vera microprocessor, touting its sizable strengths in key areas and retconning it into a framework of how data-center processors have changed. If this framework is valid, the company is at least one generation ahead of other server-processor suppliers. At the same time, Nvidia claims that competing products (implicitly, yet specifically, AMD Epyc) have structural shortcomings that limit their performance in the changed market.
At stake is a swathe of the data-center CPU market, the future size of which depends on AI adoption for productive activities, particularly those executed through agentic workflows. The largest chunks of the server-processor market have been the enterprise and cloud segments. The latter has driven revenue growth, particularly for processors with ever-increasing core counts. However, Nvidia argues that larger core counts have raised per-socket computational throughput at the expense of single-thread performance owing to factors such as memory-bandwidth contention. Single-thread performance, however, is critical to a new era characterized by agentic workflows. In this era, tasks run in parallel, and the slowest one gates forward progress. By this logic, Vera is poised to capture AI-driven market growth owing to its high-performance microarchitecture and its unusually high-throughput memory, chip-to-chip, and core-to-core interconnects.
Vera Brings SMT to Data-Center Arm CPUs
Designed primarily to host GPUs and suited to other data-center applications, Vera is an Arm-compatible processor based on Nvidia’s internally designed Olympus core. Unlike Arm’s high-performance CPUs but typical of x86 designs, Olympus supports simultaneous multithreading (SMT). Nvidia foresees agentic-AI workloads comprising primary and supplementary threads, with the latter performing management and other secondary tasks. Other workloads, such as code compilation, benefit from SMT because it improves execution-unit utilization when one thread blocks on a DRAM access or stalls while recovering from a mispredicted branch.
Like most other high-end CPUs, Olympus implements a wide microarchitecture, as Figure 1 shows. Its 10 decoders match the Arm Neoverse V3’s design. As with Arm’s CPUs, Olympus fetches instructions and converts them to macro-operations (mops). Near the interface between the front and back ends, Olympus may crack mops into two micro-operations (µops) for the execution pipelines to process.
Olympus’s execution-pipeline mix is similar to that of the Arm Lumex C1-Ultra, a client-targeted CPU that could be the basis of a future Neoverse model. The Nvidia microarchitecture has an additional branch unit and a different configuration of load/store address-generation units (AGUs): four dedicated to loads and two to stores compared with the C1-Ultra’s four dual-function AGUs. The additional branch unit should speed up control-flow code and ensure adequate resources for simultaneous threads. The added load/store capacity aligns with Vera’s primary role of intermediating data transfers between its memory and an attached GPU.
Olympus’s Vector Throughput Tops That of Arm’s Server Chip
Complementing the integer units, each Olympus core has six 128-bit SVE2 pipelines, which support eight-bit floating-point (FP8) data. By comparison, Arm configures its AGI server processor with only two vector units per CPU, although the Lumex C1-Ultra also has six. Trading flexibility for throughput, the Zen 5 CPU in AMD Epyc has fewer but wider vector pipelines. Of its four AVX-512 pipes, two only support addition.
Vera integrates 88 Olympus cores. Each has a 2 MB L2 cache and shares a 164 MB L3 cache. That’s about the same total cache as AMD Epyc, which has smaller L2 and bigger L3 caches for models with similar core counts. By contrast, Grace had half the L2 cache and 114 MB of L3 cache shared among 72 cores.
Massive LPDDR5X, NVLink, and On-Chip Bandwidth Distinguish Vera
However, Vera’s interfaces and on-chip interconnect set it apart, not its CPU microarchitecture. Nvidia developed Grace to complement its GPUs by managing data transfers to DRAM, pre-/postprocessing data, and performing other host functions. Vera addresses the same requirements for a new GPU generation, upgrading DRAM and I/O throughput.
Like Grace, but unlike other server processors, Vera employs LPDDR5X instead of DDR5 DRAM, increasing DRAM throughput in a power-efficient manner. Peak throughput more than doubles from Grace’s 500 GB/s to 1.2 TB/s. That’s also twice the bandwidth of the top-end AMD Epyc, which has 12 DDR5 channels. Generous memory throughput aligns with Vera’s primary role alongside GPUs. It also enables the microprocessor to rip through memory-bound workloads, such as the Stream benchmark, the ClickHouse database, or Fast Fourier Transforms.
Exchanging data with a GPU also requires a fast chip-to-chip interface. Vera includes an NVLink C2C interface that provides 1.8 TB/s of I/O bandwidth, twice Grace’s throughput. Just as the interface can link Vera to a GPU, as the Vera Rubin module exemplifies, it can connect two Veras in a dual-socket configuration. To connect to external NICs and other peripherals, Vera has 88 PCIe Gen 6 lanes, which also support CXL 3.1 for memory expansion.
Moving data between memory and an NVLink-attached GPU while traversing the CPU complex for processing requires prodigious on-chip bandwidth. Vera’s mesh interconnect delivers 3.4 TB/s of bisection bandwidth. It’s a slight upgrade over Grace’s 3.2 TB/s, indicating bus widths are unchanged, but clock rates are slightly faster.
Comparison with Epyc is hampered by Epyc comprising many compute dice and a central I/O die. Within a 3.0 GHz Epyc compute die, cores can transfer data to each other at 384 GB/s, based on our back-of-the-envelope calculations and assuming half are reading and half are writing. However, that bandwidth is shared among relatively few (e.g., eight) cores. The I/O die’s bisection bandwidth is about 563 GB/s. Even in a favorable 64-core configuration based on 16 four-core compute dice, that’s a fraction of Vera’s capability.
Setting aside the GPU hosting case and considering general workloads, however, the CPU-to-CPU bandwidth is only relevant to unusual applications that spend significant time coordinating many parallel threads. More commonly, memory-bound workloads will be constrained by CPU-DRAM bandwidth limitations.
Like Grace, Vera is a monolithic design. This limits core count but is consistent with Nvidia’s goal of providing more DRAM bandwidth per core than alternative processors, such as Epyc. Nvidia also highlights potential latency and throughput limitations imposed by a chiplet approach like Epyc’s. It’s unlikely that the added latency affects performance in most applications. AMD could mitigate Epyc’s throughput deficiency, such as by redesigning the processor with a faster chiplet interconnect and by employing LPDDR5X instead of DDR5 as Nvidia has done.
For now, Nvidia has withheld Vera’s clock rate; we speculate that it runs at about 3.6 GHz. The chip supports customer-selectable 250 W and 450 W TDP configurations, and we expect operating frequency to vary roughly in proportion. In addition to Vera Rubin modules and systems, Nvidia has designed single- and dual-socket Vera modules for general computing and a Vera CPU rack with 256 CPUs. The rack comprises 32 trays, each containing four dual-Vera modules. Manufacturers and cloud providers plan to have Vera systems available in autumn.
Bottom Line
With Vera, Nvidia has produced a powerful general-purpose processor. On applications benefiting from its leading DRAM bandwidth per core, it will trounce competing chips. At the same time, it’s unproven on a wide variety of applications. Code-compilation performance is of particular interest, owing to the applicability of AI to software development. If it can dominate on workloads like that, then Vera will indeed be a new CPU for a new era.