Switch Fabric Interconnects Solution to the scaling of Single Board Computers By Todd Comins,
Chief Technical Officer, StarGen, Inc. Today communication equipment OEMs are under pressure to meet aggressive time to market goals. Competition is fierce, the pace of technology development is accelerating, and the ability to get the right product to market quickly separates the successful companies from the also-rans. This trend has had a tremendous positive impact on Single Board Computer (SBC) producers. SBCs are valuable building blocks for OEMs. By using off-the-shelf SBCs OEMs don't need to design and produce there own system host cards and can rely on existing, proven control software thus reducing the risk to their development schedules. This is part of a broader trend by OEMs to utilize more and more off-the-shelf building block components as starting points for their system designs. The rapid growth in use of the Compact PCI platform as a starting point for many communication equipment designs is evidence of this trend. By using open standard technology OEMs can focus their scarce engineering resources on pieces of the system design where they add significant value and differentiation. Limiting the growth of this trend however is the fact that the off-the-shelf open technology available today is built on bus-based architectures, typically PCI. This architecture is simply running out of gas when asked to meet the needs of next generation communication equipment. It does not provide the scalability required, it unduly constrains the physical form-factor options, it does not support cost effective high availability features, and is not designed to support the multiple traffic classes traversing next generation communication equipment. For SBC manufacturers this puts a limit on their market reach and growth potential. If these deficiencies could be overcome the market opportunity for SBCs grows enormously. The solution to these problems is a new open technology approach based, not on a bus architecture but, on a switch fabric architecture. Properly defined a switch fabric can provide the system level functionality next generation communication equipment requires and provide an easy to adopt migration path from the existing open platform technologies. StarFabric is a switch fabric developed by StarGen which addresses this need. It is specifically designed for the needs of next generation communication equipment and provides 100% backwards compatibility with PCI. A number of pressing problems facing SBC designers can immediately be addressed by deploying StarFabric technology in communication equipment design. For example the challenges around supporting gigahertz processors in communication equipment from a thermal and mechanical perspective can be eliminated. Also limits on building large-scale systems of loosely coupled SBCs can be removed. We will come back and look at these two system level challenges in more detail. Today's Open Technology Platforms PCI is currently the main I/O interconnect used in the control plane of mid-range and low-end communication equipment like DSLAMs, Voice over Network gateways, wireless basestations and multiservice access platforms. It is also found as the control interface between SBCs in web infrastructure equipment like web servers and load balancers. PCI is utilized because it is generally low cost and has had acceptable performance for these applications. It provides the benefits of cost, ease of design, wide component availability, multi-vendor interoperability, and software compatibility. However being a bus it has limitations. It is a shared transport medium for all connected devices. When one device is utilizing the bus all other devices must wait. So, although the total bandwidth of a PCI bus can be quite high, up to 4Gbps, all devices on the bus must share this bandwidth. In general the number of slots is limited to 8 or less with the number decreasing as the width and speed of the bus increases. Because of its shared nature, latency can also become an issue. Reliability is impacted by the fact that one aberrant device can cause the entire bus segment to fail. PCI has physical design limits in terms of bus length. Because the bus length is limited to about 1 meter and because it cannot easily be supported through cabling, the size of a system is limited to a single chassis. Therefore chassises must be designed to meet the requirements of both the I/O line cards and SBC host cards. With movement of processing technology to gigahertz speed and beyond it is becoming an increasing challenge to house the CPU and I/O sub-systems in common chassise. What is needed is a new open technology which meets next generation communication equipment requirements and can bring all the cost and ease of design benefits associated with existing standards; ideally with an elegant migration path from those existing standards. Switch Fabrics The fundamental architecture required to meet these requirements is switch fabric technology. Switch fabrics have many benefits over bus-based interconnects, critical among these are scalability and reliability. Unlike the shared medium of a bus architecture, switch fabrics are point-to-point. Each end-point is connected to every other end-point through one or a series of switches. End-points can be considered 'bridges' to existing standard buses or components. In a bused environment only one device has access to the bus at a time and each device must request the bus and an arbitration algorithm is used to grant requests. In a switch fabric many devices can be transmitting and receiving simultaneously. Through the building of systems with series of end-points and switches a diverse and flexible universe of system typologies can be created. As more connections are added to the system the total bandwidth of the system increases. A switch-based design allows simple scaling of connections and bandwidth. Additionally it has flexible routing capabilities. Topologies can be created that have multiple routes between the same two end-points. If one route fails or becomes unavailable traffic can be redirected onto the alternative route. With point to point connections a single end-point failure does not impact the rest of the system. In a bus model a bad device can bring down the entire bus. Additionally, point-to-point connection is friendly to device insertion and removal. The point-to-point nature of switched interconnect is well suited to serial physical layer technology. The transmission line characteristics of serial technology are superior to bus technologies. To gain bandwidth buses increase their parallel nature. This limits frequency and introduces tight tolerances unfriendly to off board transmission. Serial physical technology can allow much longer distances between end-points measured in meters rather than inches depending on the cabling characteristics. StarFabric The StarFabric technology provides these benefits of switch fabric generally but also provides a simple migration path from existing open platform architectures. It provides 100% backward compatibility with PCI allowing the use of existing device drivers, BIOs and operating system support. It is also focused on providing this functionality in an easily adoptable way by not requiring exotic system design in terms of power or signal integrity, and by allowing use of standard cabling and connector technology. Its cost structure is in-line with traditional bridging technology. The initial silicon components leveraging StarGen's technology will include a high throughput switch providing 30Gbps switching capacity with six ports. Bridge chips provided by StarGen and partners will provide access from existing standard interconnects to the advanced functionality of the switch fabric. These devices offer manufacturers a new option for building high-speed, scalable and highly reliable systems. System Design Implications StarFabric can greatly increase the market opportunity for SBC producers. One important benefit of StarFabric technology in relation to SBCs is the ability to break current mechanical constraints. An example is shown in Figure 1 which provides a solution to the gigahertz processors problem. With StarFabric the system can be decomposed with the CPU complex housed its own enclosure with a separte power and cooling system. The I/O subsystem can be isolated in traditional Compact PCI chassises. A switch enclosure can be used to interconnect numerous I/O chassises and CPU enclosures. This type of system can now scale to unprecedented size with the most powerful host processors. StarFabric can also be utlized to create very large systems of loosely coupled SBCs. StarFabric bridges can be configured to act like transparent or non-transparent PCI-to-PCI bridges. By building a system of SBCs using StarFabric bridges in a non-transparent mode each SBC can provide address isolation and act in a loosely coupled fashion. Conclusion Open switch fabric technology can provide the logical evolutionary path from existing bus-based architectures and thereby solve a number of the issues limiting the market opportunity for SBCs in the communication equipment market. To be successful the switch fabric technology must provide backwards compatibility with existing bus-based open standards and provide an elegant migration path so that system designers can adopt the new technology at the pace that makes sense for their business. The interconnect must inherently support Quality of Service so that it can effectively handle all classes of traffic simultaneously while meeting each ones unique delivery requirements. The interconnect must be cost effect, on the same order of magnitude of existing standard interconnect solutions both at the silicon level as well as at the system level. It must not require exotic or expensive packaging, connector or cabling. And finally it must be open and widely available so that multiple vendors can supply devices thus insuring a wide and ever evolving set of compatible products and a ready and diverse supply base. |
 |
