AI model compression IP

Purpose-built AI model compression IP, combining software compression with hardware decompression for static model data and hardware compression and decompression for dynamic data, all at extremely low latency.

What it does

AI inference is memory bound. During LLM decode the bottleneck is memory bandwidth, not compute, so the amount of data moved per cycle sets the token rate. ZeroAI removes the waste in that data in real time, freeing both capacity and bandwidth from the memory you already have. It integrates into AI accelerators and memory controllers, for example the digital part of an LPDDR memory controller, and the same technology applies to HBM, GDDR, LPDDR, and DDR.

Static and dynamic data

• Static model data: software compression with hardware decompression.
• Dynamic data such as weights, activations, and KV cache: hardware compression and decompression.
• Lossless by default, so AI model retraining is never required.

The algorithm advantage

The patented ZeroPoint algorithm achieves higher compression than state-of-the-art algorithms while operating at much smaller block sizes (as small as 64 bytes), which is what makes very low, deterministic decompression latency possible on the inference path.

Key specifications

Detailed area, latency, and per-model compression figures are configuration dependent.

• ~50%  more usable memory capacity for higher precision and larger context windows
• ~33% higher memory bandwidth, increasing tokens per second
• ~15% lower TCO across memory cost and energy