AMD has acquired Taalas, an AI-acceleration startup that hardwires models to achieve eight times the token rate of its nearest competitor while requiring a fraction of the power. Located in Toronto and founded by Tenstorrent cofounder Ljubisa Bajic, Taalas has demonstrated a chip implementing the Llama 3.1 8B model. AMD states that the company will integrate the startup’s technology in systems with Instinct GPUs.
For the AI market, AMD has competitive GPUs such as the MI455X. Although GPUs are programmable, they theoretically lack the performance and efficiency of specialized NPUs. Still software-programmed, an NPU can’t match the performance and efficiency of an ASIC. An ASIC, however, is costly to design and inflexible. To address those drawbacks, Taalas has developed technologies to reduce customization costs and provide a degree of adaptability.
Taalas Accelerates AI Differently
Taalas hardwires neural‑network logic using predefined tiles and mask‑programmed ROM, enabling far higher token throughput than general‑purpose GPUs like the AMD Instinct MI455X. This approach trades flexibility for speed while retaining limited adaptability through layer maps and codebooks.
- Predefined logic tiles—Taalas likely bases its designs on predefined logic tiles, such as multiplier arrays. Directly implementing logic common to neural-network execution is more efficient than employing a sea-of-gates to conjure functions such as multipliers from transistor or gate arrays by mask-programming a few metal and via layers.
- ROM—foremost, Taalas employs mask-programmed ROMs to store configuration data and model weights. Because a Taalas design holds all weights in the chip, it doesn’t spend time loading them during model execution—a key factor behind the company’s fast token rates.
- Density—ROM is a dense memory type, and Taalas further increases density through techniques such as multilevel storage (e.g., storing two or four bits per memory cell) or associating multiple bit lines with a single memory cell.
- Codebooks—to conserve area and provide post-manufacture configurability, the on-chip ROMs do not necessarily directly store weights. Instead, they store indices into lookup tables (codebooks).
- For example, instead of storing 1,000 16-bit weights, the ROM stores 1,000 8-bit values. These eight bits index into a codebook holding 256 16-bit entries. That is, instead of holding 1,000 arbitrary 16-bit values, a Taalas chip restricts the model to using only 16-bit values from a 256-item menu.
- Flexibility—codebooks are an unusual form of quantization and possibly outmoded by the prevalence of 4-bit floating-point weights. However, because Taalas stores them in registers/RAM loaded at boot or runtime, they add flexibility to a Taalas chip, enabling it to execute post-trained or fine-tuned versions of the model for which it was originally programmed.
- Layer maps—to map the structure of an AI model into hardware, a Taalas base design integrates mechanisms like switches that connect the predefined logic tiles. A layer map loaded at run or boot time sets the position of the switches. Thus, a model isn’t fully baked into a Taalas chip at manufacture time.
Thus, Taalas technology does not produce a completely rigid chip only capable of executing a single version of a model. Through layer maps and codebooks, postproduction tweaking is possible. A Taalas design could support derivatives of a pretrained model, such as the multiple point releases that follow an annual all-new model from a major lab.
A one-year lifecycle for a chip sounds wasteful, but because it largely hard-codes a model, a single Taalas chip can replace multiple GPUs. Thus, the Taalas approach may be more economical. Note also that mask programming is quick—requiring less than three months—compared with the multiple years required to develop a new GPU/NPU generation.
AMD‑Taalas AI-Acceleration Options
AMD has disclosed no firm plans for Taalas. It could merely absorb the Taalas staff. Alternatively, to allow AMD to offer high‑token‑rate inference, AMD could offer the startup’s technology as a GPU alternative, a complementary accelerator, or part of a hybrid FPGA design. That is, AMD’s options for the startup and its tech include:
- Aqui-hire—AMD scraps Taalas technology and redirects the acquired team to other projects.
- Alternative—AMD offers Taalas-based chips as an alternative to Instinct GPUs such as the MI455X.
- GPU Complement—customers hard-code a portion of a large-language model (LLM) in a Taalas chip and run other functions on GPUs.
- For example, GPUs can handle the compute-bound prefill phase, where AI labs require GPUs’ flexibility to implement innovations in KV-cache compression or other functions. A Taalas chip, which stores all weights locally, can perform the decode phase, where a GPU can get bogged down loading weights.
- A system comprising both GPUs and Taalas chips may not be as fast as one based solely on Taalas technology, but it will be much faster than a GPU-only design. Nvidia is pairing Groq technology with its GPUs in a similar manner. Other complementary approaches are possible, too.
- FPGA Hybrid—Taalas-FPGA designs could retain the former’s performance while enhancing adaptability. The design could use FPGA-based logic to patch in model updates, such as executing new activation functions or computing delta matrices to implement fine tuning.
Bottom Line
Because AI models quickly evolve, the gains from wiring them into hardware aren’t worth the obsolescence risk. Taalas has developed technologies that predominantly hardwire models while providing limited flexibility. For embedded or other rarely updated systems that require low power, low cost, or high-performance AI acceleration, this may be enough. For general computing cases, however, a customer endorsement would help overcome skepticism about the Taalas approach. It’s possible that customers are interested but require the backing of a major company, such as AMD, before committing to it. It’s also possible that AMD seeks to match the Nvidia-Groq high-token-rate offering but hasn’t determined how customers will accept Taalas technology as part of an AMD-based system.