StarFabric Today; The Onramp to 10Gbps PCI Express Advanced Switching Tomorrow In August of 2002, the StarFabric Trade Association and the Advanced Switching (AS) Working Group announced plans to ensure interoperability between the StarFabric architecture and the AS for PCI ExpressT standard. The AS specification augments the PCI Express core specification by adding features critical to the communication and embedded markets. Today, the StarFabric technical architects are active members of the AS working group, making significant contributions to the AS specification. With the substantial alignment in features and capabilities of the two architectures, the StarFabric Trade Association has selected AS as its higher performance roadmap. Today, StarFabric is in volume production and is solving many challenging system issues. With StarFabric, system designs are reaching new levels of scalability and high-availability. As system requirements continue to increase and reach the 10Gbps level, the two technologies will be interoperable using an adaptive bridging mechanism. Investments made today at the current StarFabric 2.5Gbps level will have a smooth evolution to the AS 10Gbps level and beyond. Commonality One reason for the alignment between StarFabric and Advanced Switching is the tremendous commonality they share. Not only were many of the goals the same, but also, many of the specific technical methods were nearly identical. Below are some of the shared goals.
Many of the technical aspects of the two technologies are also shared. For example, methods for handling credit-based flow control, embedded clock serial links, 8B/10B encoding, and so on, are nearly identical. StarFabric and AS also have a common software model. This model, illustrated in Figure 1 is a layered architecture that utilizes bus drivers, fabric-primitives libraries, fabric-distributed libraries, and a fabric management layer. This software-stack approach provides an environment where the software engineer (depending on the application) is able to program directly to the required level. In most cases, engineers do not have to be concerned with the low-level, register-based programming environments. Typical functions implemented include configuration, high-availability features, fabric discovery, and maintenance functions. Today, a designer can create an application using StarFabric, and when they need to move to 10Gbps, they can re-use the higher level code. Fabric component drivers provided today for StarFabric will also be provided in the future for AS. There are a few differences between the two architectures. The primary difference is the physical layer or PHY. StarFabric selected a PHY based on 622Mbps LVDS (low voltage differential signaling). Each StarFabric link is made up of 4 pairs creating a 2.5Gbps full duplex link. PCI Express and subsequently, AS, will use 2.5Gbps per pair for four times the bandwidth of StarFabric. The StarFabric links can be implemented cost-effectively, using the standard 2mm HM connectors that are used in CompactPCI and CAT5 cabling. The PCI Express PHY is higher performing but requires higher performance (more expensive) connectors and PCB techniques. The connector issue will be solved in the PICMG 3.3 specification. It has specified the ZD connector, which can handle up to 5Gbps speeds over standard FR-4 material. Backplanes can be designed to handle higher data rates before the upgrade. In other words, the backplane would be designed to handle 2.5Gbps channels (for the AS upgrade), even though today's data rates would be 622Mbps channels. As such, the StarFabric Trade Association has selected AS as the high-speed roadmap extension to StarFabric. The PICMG 3.3 subcommittee is focused on providing support for both StarFabric and AS in the AdvancedTCA framework. The specification will define switched connectivity among node boards, fabric boards, and full mesh boards for AS and StarFabric. The specification will also define a hybrid system that utilizes both AS and StarFabric switch fabrics simultaneously. In this scenario, StarFabric and AS will interoperate via an adaptive bridge device, located on the StarFabric switch card. Below is an example of how PICMG 3.3 will specify an evolutionary path whereby designers can develop a 2.5Gbps system and gracefully migrate to a higher speed system in the future. As illustrated in Figure 2, the system on the left is a 2.5Gbps StarFabric system with two 1+1 redundant switch blades. Each node board is connected in a point-to-point fashion to each other. In the case, where the system needs 10Gbps of switching, such as migrating a WAN uplink from OC-48 to OC-192, two 10Gbps switch blades based on AS are incorporated into the system. Any number of node boards could be upgraded as required. The 2.5Gbps topology would interoperate with the 10Gbps topology via a StarFabric-to-AS adaptive bridge. The empty slots shown above do not necessarily need to be empty. It could potentially contain another StarFabric switch card or any other technology that was required in the box. The node cards would have to support the protocol implemented by these switches in the predefined channels. For example, in a dual-dual star topology, each node card has 4 channels. Switches 1, 2, 3, and 4 have routes to channels 1, 2, 3, and 4 respectively. For higher bandwidth or additional redundancy, all the switches could have StarFabric and each node board could have four StarFabric channels. For the migration to AS, AS switch boards could be installed in slots 1 and 2. The StarFabric node boards would not use channels 1 and 2 for StarFabric traffic. New AS node boards could then be installed in any slot and would have the AS links connected to channels 1 and 2. For hybrid systems, the StarFabric physical layer can be easily bridged to an AS physical layer through the use of an adaptive bridge. The adaptive bridge has StarFabric links with 622Mbps signaling on one interface and AS links with 2.5Gbps signaling on the other. Although they will not auto-negotiate on the same link, the StarFabric and AS hardware devices can be used in equipment where slow speed devices need to communicate with higher speed devices. In the example above, the StarFabric nodes provide a cost-effective line card solution, while AS provides the high-speed uplink capability. Using the StarFabric approach, investment is protected and total replacement upgrades are eliminated. PICMG 3.3 AS-'ready' standard open platform systems can start deployment well before the Advanced Switching components become available. Also, the hybrid solution allows one to scale to an optimum mix of low-cost, lower speed and higher speed end-points where needed. Conclusion The PICMG 3.3 specification takes the open communications platform to a new level. With the standard supporting both StarFabric and PCI Express AS, customers have the best range of solutions to support specific applications. StarFabric is available today and is ideally suited for 2.5Gbps and below applications. To date, over 50 different equipment manufacturers have shipped thousands of StarFabric ports. StarFabric is being deployed today across a diverse set of markets and applications including carrier and enterprise communications, medical imaging, video distribution, storage, military and aeronautical, industrial control, scientific clustering, and automated test equipment. AS will provide similar capabilities as StarFabric in a number of years, but at higher performance. The AS for PCI Express specification is scheduled for submission to the PCI-SIG in early 2003. The roadmap for AS products with 10Gbps performance is in 2004/2005, but one can start building the blocks to higher performance with StarFabric today. Justin Moll |
 |
