As data-center AI has come to dominate the market for computing accelerators such as GPUs, new vendors are targeting the previous main application of accelerators, high-performance computing (HPC), introducing unusual technologies. NextSilicon is one of these companies, supplying national labs and other HPC customers with its Maverick-2 processor based on a data-flow architecture.
The HPC market spans industrial and government customers performing tasks such as simulating molecular dynamics, fluids, climate, and mechanical structures. Computationally intense and typically parallelizable, many HPC workloads map onto a GPU architecture, giving rise to data-center GPUs to offload computing from CPUs. These same GPUs became the foundation of AI processing, but a sharp divide stands between AI computing and HPC: precision. Neural networks are probabilistic, allowing their processing to involve low-precision values. Four-bit values are common and 1.58-bit and less precise models are research topics. By contrast, HPC requires high precision, and 64-bit floating-point data (FP64) is the norm. To better address the lucrative AI market, recent GPUs have deemphasized FP64 to provide more die area for low-precision circuitry, creating an opportunity for NextSilicon.
Data-Flow Architectures Depart from the von Neuman Paradigm
Rather than adapt a CPU or GPU, NextSilicon has developed a data-flow architecture. In such designs, program instructions map to a function-unit array. An array element executes one operation and passes its results to another element that executes the subsequent operation. Chaining elements like this forms an execution pipeline, and multiple chains can run in parallel. It’s an inefficient approach for branchy code. However, it exploits the parallelism of HPC kernels—code loops operating on data.
Moreover, a data-flow architecture can reduce power and speed up execution by eliminating the writeback for intermediate results. Because the data-flow approach unrolls kernels, mapping operations into processing elements, it eliminates the complex fetch, decode, branch-prediction, and out-of-order circuitry (including large register files) of a high-performance CPU. A data-flow architecture can use the silicon area that a CPU would allocate to those functions—and which accounts for the majority of CPU area, excluding caches—for processing elements. A data-flow architecture, therefore, provides several times the raw execution throughput of a CPU. Even compared with a GPU, NextSilicon claims advantages; Maverick-2’s peak throughput can be 10 times that of a GPU while requiring 60% less power.
The strengths of the data-flow technique have led some NPUs, such as the Google TPU, to implement it to specifically handle the tensor/matrix operations that dominate neural networks. Less specialized use of the architecture is rare, however. SambaNova’s NPUs, for example, employ an array of computing and memory units to handle more than just tensor operations. For low-cost, general-purpose processing, Ubitium has developed a RISC-V data-flow processor. Whereas SambaNova’s design is a coarse-grain reconfigurable architecture (CGRA), Ubitium takes a fine-grained approach, employing many simple processing elements.
NextSilicon Maverick-2 Architecture Overview
Based on a grid of basic processing elements, NextSilicon’s approach is similar to Ubitium’s. Exemplifying the simplicity of NextSilicon’s elements, the Maverick-2 processor does not have any dividers. Instead, software maps division instructions into a sequence of elements, eliminating die area that hardware dividers would use for the relatively uncommon operation.
This mapping software is essential to the data-flow approach. Programming any new architecture is an adoption barrier, one heightened by the unusualness of the data-flow approach. Running in real time, code analogous to Transmeta’s code morphing software (CMS) translates unmodified applications authored in C++, Python, Fortran, and even Cuda. The translation software identifies kernels suitable for data-flow execution, mapping those kernels to the data-flow hardware. Code that doesn’t map well runs on conventional RISC-V cores.
When a kernel completes, the translator reallocates the associated processing elements to the next kernel. The next time the kernel executes, the translator can run again, factoring in information gathered from the prior execution. For comparison, the Transmeta CMS also generated progressively better optimized conversion each time a code path ran.
Maverick-2 Form-Factor Options
NextSilicon offers Maverick-2 as a PCIe Gen 5 board, either as a conventional add-in card or an open accelerator module (OAM). The company rates the card at 400 W, which includes the Maverick-2 processor and its in-package HBM3E memory totaling 96 GB. In addition to the data-flow array and RISC-V cores, the 5 nm Maverick-2 has a generous 128 MB of on-chip SRAM. The OAM doubles these specs, featuring a chip with two processor dice (and, therefore, twice the SRAM) and twice as much high-bandwidth memory, as Figure 1 shows. Consequently, power nearly doubles to 750 W. Both the card and the OAM operate the processor at 1.5 GHz. Independent benchmarks aren’t yet available, but NextSilicon reports Maverick-2 meets or greatly exceeds the performance of an unspecified GPU, depending on the workload.
Sandia National Laboratories is NextSilicon’s lead customer, employing dual-die Maverick-2 modules in the Spectra supercomputer. Spectra is Sandia’s second Vanguard project, which explores advanced architectures for national security applications. Spectra integrates 64 nodes, each having two Maverick-2 OAMs. Other customers are assembling similar machines.
Bottom Line
Already shipping Maverick-2, NextSilicon is past the “can they make it” stage that characterizes startups. Comprehensive benchmarks, particularly if they come from an independent organization such as Sandia, will help NextSilicon secure additional buyers. Employing brilliant researchers, national labs and academic institutions may prove more willing to adopt the startup’s unusual architecture. Commercial users will likely need to be assured of software compatibility, requiring proof that NextSilicon’s translation software can handle their applications. All customers will also require information about which workloads run well on Maverick-2 and which aren’t suited to the architecture. Fortunately for NextSilicon, the HPC industry is more open to innovation than conventional computer customers, and Maverick-2 is entering production as GPU alternatives are in demand.