Building AI Infrastructure for Greater Choice and Flexibility

By Mayank Bhatnagar

October 14, 2026

AI infrastructure is changing fast. Training and inference are putting different demands on compute, memory, connectivity, and networking, and there is no single architecture that fits every workload.

For system architects, this opens up an important question: How much flexibility can be built into the system from the start?

The answer increasingly involves giving designers more choices across memory, compute, connectivity, and packaging, while making sure those technologies work together.

Different Workloads, Different Requirements

AI training and inference may run on much of the same underlying infrastructure, but they don't behave the same way.

Training places heavy demands on memory and communication between accelerators. Inference has a different mix of requirements, including memory capacity, latency, throughput, and efficiency.

Those differences matter when choosing an architecture. A memory technology that's right for one workload may not be right for another. The same applies to interconnects and networking.

This is why AI infrastructure is becoming more heterogeneous. Architects are combining different technologies based on what they need the system to do, rather than trying to force every workload into the same design.

Figure 1: Heterogeneous applications demand distinct solutions for optimized implementations

More Choice Through Open Standards

Open standards give designers more options for how they put those systems together.

UCIe is one example. The standard provides a common approach to die-to-die connectivity and is aimed at supporting an open chiplet ecosystem. It gives the industry a standardized way to connect dies within a package rather than tying the architecture to a single implementation.

System architects are also working across a wider range of process technologies, packaging approaches, and functions. A chiplet-based approach provides another way to partition those functions, allowing compute, I/O, memory interfaces, and other functions to be integrated within the same package.

UCIe provides a standardized die-to-die connection for that environment. This gives designers a common interface as they evaluate how different functions should be partitioned and integrated.

For designers, the value is choice. Different functions can be brought together in a package while using a common interface between them.

That freedom becomes particularly useful when architects are balancing performance, power, cost, packaging, and process technology.

Interoperability Matters

Open standards provide a common foundation for bringing technologies from different suppliers and foundries into the same system. Interoperability helps designers understand how those different implementations will work together.

Verification plays an important role here, particularly as systems incorporate more functions, interfaces, and technologies.

The Same Thinking Applies Across the AI Factory

The need for choice doesn't stop at the package.

An AI system can include several kinds of connectivity, each serving a different role. PCIe, CXL, UCIe, UALink, and Ethernet-based technologies address different parts of the architecture. Memory brings another set of decisions.

Different connectivity, different scope, same goal: economical, rapid data transfer

The requirements also change depending on where data is moving. HBM4E and LPDDR6 address different memory needs, while PCIe supports high-speed host and device connectivity. UALink addresses accelerator-to-accelerator communication, 224G high-speed SerDes provides the underlying high-speed signaling needed for demanding connectivity applications, and UCIe connects dies within packages.

Designers need to consider what they are connecting, how much data needs to move, and the requirements of the workload. Looking at those decisions together provides a more complete view of the system than treating each interface independently.

That perspective changes the design conversation. Rather than optimizing individual pieces independently, architects can look at how memory, compute, and connectivity fit together across the system.

Explore Cadence IP:

• Universal Chiplet Interconnect Express (UCIe™) PHY and Controller
• Controller for PCIe
• Controller for CXL
• UALink Subsystem

Designing With Room to Evolve

AI infrastructure will continue to change as workloads, memory technologies, accelerators, and interconnects advance. Building flexibility into the architecture gives designers more options as those requirements change.

Open interfaces and interoperability are an important part of that flexibility, allowing designers to choose technologies based on the needs of the system rather than a single approach.

For a deeper look at the memory, compute, and connectivity technologies shaping these systems, read our Building AI Factories with IP Solutions white paper. https://www.cadence.com/content/dam/cadence-www/global/en_US/documents/tools/silicon-solutions/building-ai-factories-with-ip-solutions-wp.pdf