From GPUs to Memory Pools: Why AI Needs Compute Express Link (CXL)

By Raghabendra Rout, Penguin Solutions Corporation

October 27, 2025

CXL 3.0 allows entire racks of servers to function as a unified, flexible AI fabric. This is especially significant for AI workloads where traditional GPU islands are constrained by memory limits. 

Artificial intelligence (AI) is entering an era of unprecedented scale. From training trillion-parameter large language models (LLMs) to enabling real-time multimodal inference, AI workloads are reshaping the very foundations of data center infrastructure. While GPUs and accelerators have become the face of AI, a critical bottleneck lies behind the scenes: memory, bandwidth, latency, and scalability challenges often determine the success or limits of AI systems. This is where Compute Express Link (CXL) steps in, offering a transformative solution. 

The memory bottleneck in AI 

These are some of the key items creating the memory bottleneck in AI: 

• Training foundation models require enormous memory capacity, often exceeding what is available in a single GPU. 
• Inference at scale demands rapid access to large datasets without duplicating memory across GPUs. 
• Traditional architectures force CPUs, GPUs, and accelerators to operate in silos, creating inefficiencies. 

To read the full article, click here