RapidIO moves up to Fabric

Eran Strod
(08/02/2004 9:00 AM EDT)

Communications fabrics are perhaps the last major segment where the light of open standards has yet to shine-84 percent of the fabric silicon that shipped in fiscal 2003 was based on proprietary interconnect technology. That reality may be due in part to the fact that systems have such divergent technical requirements, with some implementations using simple, brute-force interconnects while others employ sophisticated traffic-management features.

To counteract the inefficiency of this diversity in communications fabrics, engineers are in the midst of developing extensions to the RapidIO interconnect to create a flexible and efficient fabric for a wide variety of communications systems. RapidFabric will provide a protocol-agnostic fabric with a flexible traffic-management scheme and flow control mechanisms supporting good bandwidth utilization.

RapidFabric adds to RapidIO a serial physical layer, flow control, multicast, protocol encapsulation/internetworking and high-end traffic-management capabilities. Engineers have completed specifications for a flow control logical layer, internetworking/encapsulation, multicast and a serial physical layer. Specifications for traffic management and a next-generation physical layer should be done early next year. Also in the works is a data-streaming logical layer.

RapidIO's modular architecture accommodates specification enhancements without breaking compatibility with existing RapidIO products. By dividing RapidIO into separate layers (physical, transport and logical), the founders created a framework that could be enhanced in ways that could not have been anticipated at the start.

For example, all RapidIO packets share a common transport-layer header. This allows switches to route any type of packet without having to be aware of the logical packet format. Future packet types, or even user-defined protocols based on the Type 0 packet, conform to the common transport, so future RapidIO protocol enhancements will run through existing RapidIO switches.

Most of the RapidFabric extensions are fully backward compatible and interoperable with other parts of the RapidIO standard. In addition, new RapidFabric features may be adopted individually or not at all.

Key to RapidFabric is its approach to multicast, flow control, encapsulation and traffic management. Quite a few bandwidth-hungry applications, including gaming, videoconferencing and broadcasting, require multicast support as a means of avoiding congestion. RapidIO uses a device-based routing scheme to handle multicast traffic. Thus, multicast in RapidIO becomes merely a generalized case of unicast.

Each RapidIO endpoint is assigned a unique 8- or 16-bit ID. When a switch receives a packet, it performs a simple classification-say, a table lookup-and forwards the packet out one or more ports. In unicast, the lookup returns a single port. In multicast, the packet is elaborated to multiple ports.

Competitive interconnects that use path-based routing are unable to easily support multicast. One of them opted to add a special addressing scheme just for multicast. In other words, switches would have to support the complexity of two addressing schemes: path-based routing for unicast and device-based routing for multicast.

Flow-control features
Last fall, the RapidIO Trade Association released the first in a series of flow control specifications that provide congestion control for applications that require ever-higher levels of bandwidth utilization. RapidIO flow control defines a congestion control packet that exerts back pressure based on RapidIO flows.

The data-streaming logical-layer specification will include more extensive flow control features based on traffic class and end-to-end flow control. When combined, these mechanisms can allow an equipment vendor to achieve high levels of fabric utilization.

Ethernet and some other protocols used in backplanes rely solely on link-level flow control, which leads to very high peaks and very low valleys in bandwidth utilization. OEMs cope by overprovisioning Ethernet in the backplane, in some cases by more than 50 percent. This is a remarkable waste of bandwidth and leads to very expensive network infrastructure. To help OEMs cope with the need to support numerous line card protocols, RapidIO provides encapsulation and internetworking transport for common protocols such as Ethernet/PPP, Utopia/ATM, packet-over-Sonet, Common Switch Interface and System Packet Interface.

RapidIO does not dictate a single way to transport protocols across the fabric. The payload in RapidIO data streaming has no inherent semantic. Implementations can employ encapsulation or true internetworking.

While many mission-critical applications require bulletproof fabrics, others explicitly want to introduce lossiness at certain times. When a lossless fabric is congested, nothing gets through. This is unacceptable to many communications applications that view lossiness as a congestion-control strategy. RapidFabric can manage congestion intelligently by viewing traffic as a series of flows that can be policed and shaped according to any desired heuristic. The fabric manages congestion by separating traffic into thousands or perhaps millions of individual streams with specific priorities, latency requirements and throughput requirements.

RapidFabric's traffic-management specification allows 64,000 streams to be defined between any source and destination pair. It supports the transport of up to 256 classes of traffic without losing information in its 8-bit class field. A switch or endpoint may prioritize packets based on an application-specific semantic assigned to data-streaming class and flow identifiers.

Finally, RapidIO provides speed advantages by leveraging the volume physical-layer technology used in Ethernet. It will continue to scale by adopting PHY standards such as those being defined in the Optical Internetworking Forum. Today, RapidIO defines both parallel and serial physical-layer protocols.

Eran Strod (Eran.Strod@freescale.com) is chairman of the Strategic Marketing Group of the RapidIO Trade Association.