The Missing Layer in AI Systems Design is Predictable, Scalable Data Movement

By Andy Nightingale

August 14, 2026

Over the years, AI chip designers have focused on scaling systems, connecting compute, memory, and accelerators. It’s led to an evolution in chiplet architectures. Significant progress has been made in standardizing die-to-die interfaces and advancing packaging, making it easier to assemble more capable systems.

These technological advances have received most of the spotlight in this arena. Far less attention has been paid to what happens after everything is connected, specifically how data moves, how it’s prioritized and scheduled, and how reliably it’s delivered under real, multi-tenant AI workloads.

It’s not simply a question of bandwidth — it’s a question of deterministic data and control within the fabric. Data-movement predictability ultimately drives performance. When several workloads compete for bandwidth, memory, or interconnect, systems may appear efficient in ideal conditions. However, they quickly lose performance as real load increases. Utilization can become unpredictable, not because of insufficient computing, but because data isn’t delivered when and where it’s needed.

When Best Effort Breaks Down

The system tries to move data as quickly as possible, but under contention, it often lacks guarantees on latency, bandwidth, or ordering. That model breaks down in AI systems where multiple engines share memory and IO, and where training, inference, and control traffic all have different requirements.

To read the full article on Electronic Design , click here.

Explore Arteris IP:

• FlexNoC Interconnect IP
• CodaCache Last-Level Cache IP
• FlexGen Smart Network-on-Chip (NoC) IP
• FlexWay Interconnect IP