switched serial interconnect, switch fabric, backplane

PICMG 2.16 - New fabric technologies and the Compact Packet Switching Backplane - A Comparison

Ever since the PICMG 2.16 specification was ratified in the summer of 2001, it has seen growing acceptance. Originally proposed by Performance Technologies, the specification has become quite popular with its compatibility to the omnipresent Ethernet protocol. Versions with 10/100 Mbit/s links have been shipping in volume, as well as a few with 1000 Mbyte/s links. Since adoption, several switched fabric technologies and new architectures have stepped forward to challenge PICMG 2.16. At the same time many of these technologies are competing with PICMG 2.16, some are integrating high-speed Ethernet links themselves. The key backplane-based technologies we will focus on are StarFabric, GigaBridge, and AdvancedTCA. As Infiniband, Rapid I/O (parallel version), and some other technologies are not generally backplane-focused, we won't go into them. We're also focusing on CompactPCI-based and PICMG initiatives, so we will not mention VITA 34 or VITA 41(the VXS backplane).

Note: It is important to point out that although the technologies below claim superiority on some levels to PICMG 2.16, the specification is extremely useful. It is ideal for those who want higher performance using Ethernet as a traffic transport. We support PICMG 2.16 and believe it will continue to have a strong niche in the embedded systems market.

PICMG 2.16 and the recently ratified StarFabric specification PICMG 2.17 have many similarities, and a few key differences.

Both compatible to CompactPCI, existing hardware and software
Both mainly utilize centralized topology, have distributed topology options
Both accept H.110bus optional for Computer Telephony
Both have design wins and early acceptance
Both currently have hardware developed
They may be integrated in some versions

Clearly, PICMG 2.16 is a strong choice for those who want to use Ethernet as a traffic vehicle or a low-end control plane. The protocol is widely accepted and many PICMG 2.16 products have hit the market.

StarFabric is geared toward many of the same applications as PICMG 2.16, but focuses towards ones requiring higher performance and ability to handle multiple classes of traffic. The key differences between PICMG 2.16 and StarFabric are that StarFabric has higher bandwidth, handles both asynchronous and isochronous traffic simultaneously, and has reduced processor overhead. StarFabric has higher bandwidth capabilities than cPSB, with current version of 2.5 Gbps speeds and 10 Gbps performance on the roadmap.

Many in the embedded computer industry claim that GigE has higher processor overhead. In fact, using GigE as an interconnect requires 1 Ghz of processing power to run the protocol stack. StarGen has stated that they believe GigE has 21K interrupts per second to StarFabric's 1 interrupt. Also, they state that one network message on GigE can have 10K Instructions (10 Instruction/Byte), while StarFabric has only 20 instructions total per message.

Although you can say the same things about StarFabric, PICMG 2.16 has the advantages of having the lead in achieving wide acceptance, is quite low in implementation costs (for 10/100 version), and easy to integrate.

A StarFabric system with PICMG 2.16 links? Yes, the StarFabric specification has implementation options to integrate Ethernet traffic based on the PICMG 2.16 pinouts in a dual-dual configuration.

StarFabric has announced that they plan to have interoperability between the StarFabric architecture and the PCI Express™ Advanced Switching interconnect standard. The StarFabric architecture incorporates many of the features and capabilities found in the PCI Express Advance Switching specification. This is a key element of StarFabric's scalability and roadmap to 10 Gbps speeds. The higher speed links would likely go across the StarFabric implementation of AdvancedTCA (PICMG 3.3). The core specification PICMG 3.0 is coming along, and PICMG 3.3 is following along with it.

There has been a great deal of product development recently in StarFabric. StarGen has announced their PICMG 2.17-compliant switch cards and their SG3010 TDM Bridge. Companies have begun to announce PMC modules that are compliant to 2.17. Today, a 17-slot PICMG 2.17 Development Backplane is also available.

The PICMG 2.17 Development Platform is a significant new development. It allows testing of PICMG 2.17 system designs in various configurations. Elma has created a development system using Bustronic's 17-slot backplane. The backplane has 10 basic node slots, dual fabric slots, two node slots with H.110, a StarFabric system slot with cPCI and H.110, and two standard cPCI with H.110 slots. This topology allows the system designer to test the implementation of StarFabric, along with standard cards using the CompactPCI bus and/or the H.110 bus. System testing can be performed in the NEBS grade rackmount enclosure designed for High Availability systems. The example in Diagram #1 shows a 12U version with a push-pull airflow technique using fan trays exerting 350 cfm of forced air per tray. The intake air is filtered using Bellcore-compliant foam air filters that are easily removable. The 1200-watt 48V PSU input power supplies are N+1 redundant pluggable units. This chassis solution provides advanced prototyping of the StarFabric system, including allowances for testing hot-swapability and redundant system components. Once prototyping is completed using a Development System, the designer can contact the manufacturer for a specific solution.

By the time this article is published, there will likely be StarFabric Adapter Cards (SAC) on the market. These cards act as PCI-to-StarFabric bridges. Standard CompactPCI cards can be plugged into the StarFabric Adapter Card, which in turn, is plugged into the backplane. The adapter card takes the cPCIbus traffic, serializes it, and sends it across the backplane in two 2.5 Gbps StarFabric links. Both 32 bit/33 Mhz and 64-bit/66 Mhz traffic can be converted via StarGen's SG2010 chip which resides on the adapter card. These cards will be a useful tool in prototyping a StarFabric system.

GigaBridge is a PCI-switching technology developed by PLX Technology Inc. It is run on a scalable, highly available, self-healing ring topology for communications equipment requirements of OC12 to OC48 trunk speeds. The GigaBridge HA backplane is a new product, however, GigaBridge has been around for a few years and is in 2nd generation of silicon.

Both compatible to CompactPCI, existing hardware and software
Both currently have hardware, software developed
Scalable solutions

GigaBridge is also geared toward many of the same applications as PICMG 2.16, but focuses on PCI-based applications that require even higher performance than 2.16, while preserving current software and hardware investments. There are many differences in performance and capabilities. The fabric speed difference is vast as GigaBridge uses 6.4 Gbps LVDS links. (See performance chart - Diagram #2). The GigaBridge ring has built in high availability, with redundant counter-rotating rings running the fabric. While PICMG 2.16 increases the performance of the data plane only, GigaBridge enhances the performance of both the control plane and data plane. According to PLX, using an Ethernet/IP solution for the control plane would add undesirable latency, and adding a control plane to the 2.16 fabric involves a massive software modification.

Bustronic has been working with PLX to develop the GigaBridge High Availability backplane. The backplane uses a scalable, highly available, self-healing ring topology using a 6.4 Gbps Low Voltage Differential Signaling (LVDS) link interface. This development system consists of the chassis, backplane, bridge-enabled card and a switch module. The bridge-enabled card converts the PCI bus to the switched-PCI bus via a GBP device. A cell-based fabric with independent PCI bus segments connected to each port, each GBP device can drive up to four PCI slots and interoperates with other controllers as ports on the ring. Each device is linked via two 16-bit-wide, point-to-point, low-voltage-differential links clocked at 400MHz. The developer's CompactPCI board plugs into the bridge enabled card. In turn, the bridge enabled card plugs into the backplane. The switched-PCI network is contained on the backplane. (see diagram 3)

As every slot has a switch module plugged in the rear, it is not necessary to populate the entire backplane. When the slot is left empty, the switch module completes the switched-PCI network. The switch module detects when a bridge-enabed board is present and shuts off, allowing the bridge to handle network traffic. The network can tolerate a maximum of 8 empty slots on the backplane. The backplane features a 6U height and a twelve-slot controlled impedance design.

The GigaBridge HA backplane is compatible with standard Eurocard specifications. Different options in chassis can be used with the technology. Versions that offer redundant cooling, hot-swappable power supplies, etc. would be a good fit.

The AdvancedTCA (PICMG 3.x) specifications have piqued the interest of the industry, with well over 100 participating companies in PICMG. The new architecture is geared for central office and high-end communications applications. It's 8U form factor and 280mm depth requires some hardware modification, but the industry is already gearing up . The larger cards allow more space for more components, while the wider spacing between slots allows for taller components. This is significant as newer card blades are loaded with larger components and more of them are on the same board. The backplane is approximately 5U high, with 3U of space at the top for I/O. As there is not a defined I/O interface, the draft specification leaves a wide range of I/O options. Prototype backplanes will likely be designed in 5U heights with separate boards for I/O connections or the area left empty for direct plugging from the board to an interface mounted in the subrack. A guide-pin on the backplane helps with alignment to the board.

There are currently three sub-specifications -- PICMG 3.1 for Ethernet, PICMG 3.2 for Infiniband, and PICMG 3.3 for StarFabric. The backplane has sections for the Base Interface, the Fabric Interface, Telephony Synchonization Clocks , and a Board to Board Update Port The Base Interface is always configured as a Dual Star and connects the fabric boards, node boards, and system management controller. The Fabric Interface is comprised of 13 Channels spread across P20 through P23 in a 19" chassis system. The 13 Fabric Channels can be used in a variety of configurations to provide connectivity between up to 14 Boards/Slots in a shelf. Supporting the Update Port and any Telephony Clock requires the use of the P20 connector.

By the time this article is published, a few AdvancedTCA prototype models should be released. One plan is to have a 14-slot version utilizing Ethernet in a 12U vertical 19" rackmount chassis. The 5U backplane is spaced at 1.2", and features a Dual Star topology with two fabric slots. The chassis would feature rear I/O mounting, dual 48V DC input, front-to-rear dual redundant cooling (disspating 200 watts/slot), and an ESD terminal on the front. The system management interface would comply to PICMG 2.9 specification. (A photo of a 14-slot demo unit is on the front cover of this magazine.)

The performance of AdvancedTCA depends on the fabric that runs over the architecture. The current ZD connector is capable of handling 5 Gbps speeds. The connector is becoming popular in many high-speed designs (See Diagram #4)

PICMG 2.16 is more geared for OC12 to as high as OC48 for the Gigabit version. Its targeted towards enterprise class systems, while AdvancedTCA is mainly for carrier grade systems. Some of the AdvancedTCA specification uses concepts based on PICMG 2.16. The base interface of AdvancedTCA uses Ethernet traffic and is similar in design to PICMG 2.16 rules. Of course, the PICMG 3.1 sub specification is for Ethernet. It will be interesting to see if higher-end versions of PICMG 2.16 continue to be designed or if the market will move the higher-end to AdvancedTCA.

New front panels will need to be developed for AdvancedTCA. One example is an 8U panel with sheilding gasket for EMC, and egronomic handles for easier removal. (See diagram).

PICMG 2.16 is a widely available, successful standard. There are several technologies that are vying for the market in advanced backplane-based communications systems. PICMG 2.16 should retain a strong niche in system designs for cPCI-compatible, high data bandwidth applications. Expect PICMG 2.17 and GigaBridge to gain popularity in higher-end applications that retain compatibility with legacy products, while AdvancedTCA gears up for the telecom central office.