UA Link vs Interlaken: What you need to know about the right protocol for AI and HPC interconnect fabrics
Introduction: AI applications require high-performing interconnects
AI and high-performance computing (HPC) workloads are growing steadily, and the need for establishing high performance and low-latency interconnect links is placing pressure on interconnect technology. CPUs need to communicate massive amounts of data to accelerators, GPUs need to work together in synchronous fashion in clusters, and in many cases memory systems must remain coherent across these many actors. Protocol selection plays a crucial role in such high-speed environment directly affecting performance through factors like encoding efficiency and error resiliency. There is no single protocol that excels in all aspects, and the two different solutions presented, Interlaken a high-speed protocol with origins in networking, and UA Link a new interconnect for accelerators, will demonstrate the trade-offs that should be considered.
Protocol Architecture: Packets and Flits
Interlaken offers a channelized interface based on high speed Serdes and combines serial encoding and packet multiplexing like that found in SPI4.2. Its origins are in the networking vertical and the original companies behind the specification are Cisco and Cortina. It uses 64B/67B encoding for its PCS layer and allows for burst transfers and per channel flow control for added flexibility and efficiency in transporting packers across chip-to-chip links. The original specifications has been later extended with Retransmission for data recovery and RS-FEC for error correction. These features place Interlaken as an ideal protocol for packet-oriented traffic with speeds up to 2.6 Tbps per link.
UA Link is a much newer concept dedicated to connecting Accelerators in compute fabrics. The Data Link Layer (DL) uses fixed 640-byte data flits to communicate with the PCS which then maps these into a 680-byte codewords with RS-FEC protection. The Reconciliation Layer (RS) ensures lane alignment and flit framing and deploys 64B/66B encoding towards the PCS. UA Link focuses on connecting accelerators through switches and rather than transporting generic packets the focus is on low latency deterministic communication of a rather fixed format of data with up to 800 Gbps per station.
Error Control: Different Approaches
Both Protocols deploy the RS(544,514) Forward Error Correction, but the approach to Handling and correcting Errors is one of the key aspects that sets these two apart.
The Interlaken RS FEC Extension is used to correct symbol errors across long backplanes where the signal integrity can be degraded by connectors, or when the protocol is deployed with the use of PAM 4 signaling on the Physical layer. This scheme is deployed at the expense of the added latency introduced by the RS FEC Decoder. The Interlaken Retransmit Extension on the other hand provide a mechanism for error recovery without involving the upper layers but at the expense of increased buffering and loss of determinism in the link latency.
UA Link on the other hand makes use of Low Latency RS FEC modes thanks to 1-way and 2-way interleaving that reduce the latency compared to the 4-way interleaved option. And any need for retransmission is left to the higher protocol layers reducing buffering need ensuring predictable latency. Avoiding latency jitter in tightly coupled and synchronized training clusters is a critical requirement.
Scaling and Performance
Flexibility in scaling is another major factor when deciding between the two protocols.
Interlaken scales by adding more Serdes lanes and increasing the Serdes line rate. Traditionally running at 10G or 25G it has been later extended to 56Gbps with PAM2 and even 112 Gbps with PAM 4 signaling. The overhead introduced by the protocol channel multiplexing and bursty nature as well as deploying the retransmit extension can make the latency variable under high load. Interlaken remains a natural fit for backplanes, packet switches and line cards.
UA Link on the other hand by design deploys 200G x1/x2/x4 links per station so it is limited to 800 Gbps when using 200Gbps+ Serdes Lanes. The stack is tuned for deterministic latency even under high load with very fixed flit data mappings. It makes the protocol very well suited for chiplet based accelerators and CPU to GPU fabrics where memory synchronization and coherence is equally important to throughput.
Quick Comparison
Feature | UA Link | Interlaken |
Origin | AI/HPC accelerator fabrics | Network and Switching
Extended to HPC I/O and accelerator Links |
Protocol Model | Flit based deterministic transfer | Channelized packet-based multiplexing, ideal for many concurrent data streams |
Encoding | 256B/257B coding with RS(544,514) FEC overhead | 64B/67B encoding with optional RS (544,514) FEC |
Error Detection | CRC per 640 bit Flit | CRC per burst and per lane |
Error Recovery | Retry at higher layers | Retransmit extension |
Error Correction | RS(544,514) with 1/2/4 way interleave (low latency modes) | RS(544,514) with 4-way interleave |
Throughput | 200/400/800 Gbps per station (200+ Gbps Serdes) | 1-2600 scales with Serdes speed and number of lanes |
Flow Control | Flit scheduling with credits | Per channel flow control |
AI/HPC Fit | Latency critical compute fabrics | Resilient large-scale fabrics |
Conclusion
Despite their differences, UA Link and Interlaken are both relevant resources for developing AI and HPC systems. Interlaken with its channelized transmission mechanisms, RS FEC and Retransmit extensions, offers a resilient and proven fabric for connecting boards, and networking accelerators where error resilience and long reach are of high importance. UA Link with its reduced FEC latency performs excelent in short-reach high-bandwidth connections, where synchronization and latency predictability are key factors. Thanks to its Flit oriented design, and deterministic low latency PCS.
Related Semiconductor IP
- Interlaken Controller
- UALink PCS IP Core
- Verification IP for UALink
- UALink IP Solution
- 224G SerDes PHY and controller for UALink for AI systems
Related Blogs
- The Semiconductor World vs TSMC vs EDA
- Intel vs. ARM : In the Smartphone Era (Part 1)
- Intel vs. ARM: In the Smartphone Era (Part 2)
- Intel vs. ARM: In the Smartphone Era (Part 3)
Latest Blogs
- The Perfect Solution for Local AI
- UA Link vs Interlaken: What you need to know about the right protocol for AI and HPC interconnect fabrics
- Analog Design and Layout Migration automation in the AI era
- UWB, Digital Keys, and the Quest for Greater Range
- Building Smarter, Faster: How Arm Compute Subsystems Accelerate the Future of Chip Design