The Power is the System

There are many new bus architectures that have been released over the past two years. In some cases these new documents have revealed significantly new approaches to system power and system management. This article will examine how power and management features are being addressed in AdvancedTCA, MicroTCA, CompactPCI Express, VITA 41 and VITA 46.

It is fair to say that no other single aspect of system design will have the potential to radically change the operation and capability of future systems as the dramatic escalation in power density. Until recently, system voltage distribution has continued to follow a downward trend which is the result of low voltage signaling methods such as LVDS.

However, lower voltages have required higher and higher current levels within the power system. Lower voltages require highe current level at the backplane interface. For example it takes nearly 17 amperes to provide 200 watts at 12 volts where as it only takes less than 9 amperes to provide the the same 200 watts at 24 volts.

Therefore, distributed system power that had migrated slowly over the past ten years from 5 to 3.3 volts is now returning to much higher distributed voltage levels in many of the newer backplane architectures. These higher voltages allow more power for a given connector blade or power cable.

To meet module voltage requirements, the bussed 12, 24 and 48 VDC is then down converted on each board as required for the components. At the same time, the focus of many new embedded architectures are being implemented to support the communication needs of both the public network infrastructure as well as industrial and military networking infrastructure requirements. Therefore, there is an additional compelling reason for system designers to provide for 48VDC power distribution.

Despite this apparent fluctuation in system power requirements one simple rule has remained constant. Driven by Moore’s Law, each new system requires more power in a smaller space than ever before. This trend has not changed and the resulting thermal challenge threatens to require radical solutions in the not too distant future.

To understand the change that is taking place, let’s make a quick comparison between a newer architecture and two of the older architectures.

The two most suggestive rows in the above table, should be the “shelf management” and “management bus” entries. Early VME and CompactPCI systems may have operated unattended, but in the event of a failure, it was assumed that a human would be there to restart the system. Even in the case of redundant installations, these two architectures were expected operate as autonomous systems. Even if they were on a network, they would typically continue operating regardless of the condition of other systems in the network.

In contrast, the environment that AdvancedTCA systems were intended to operate was envisioned to be a large central office with many systems operating in a managed network environment. Individual failed modules or chassis could not be allowed to degrade the service to any single customer so redundancy and remote management was central to the overall design philosophy.

AdvancedTCA

Developers of the AdvancedTCA specification intended for ATCA systems to be operate in today’s typical unmanned central office environment while remaining under close coordinated control of a remote system manager. To achieve this goal, the ATCA document devotes one hundred and fifty pages to the subject of system management. This represents over 30% of the massive 450 page specification. Another twenty-two pages are devoted exclusively to power requirements. The sophisticated collection of management and power system features is possible, in large part, because of the adoption of another important specification, PICMG 2.9 which represents another thirty additional pages.

Key features that deserve mention are the following:

PICMG 2.9 is required for all removable node or fabric switch cards and this specification provides the necessary support for remote or shelf based system management. This system bus provides a message and address structure that allow any modules or subsystem to be controlled. The IPMB bus is used by boards and subsystems to provide status information and the redundant Ethernet base fabrics provide the network for instructions to be distributed.

Payload cards are only allowed to consume 10 watts of power until they have negotiated with the system manager for more power. This is a dynamic process and a board may have its power restricted at anytime based on the system manager’s evaluation of resource demands.

The use of an intelligent IPMI interface between various system components results in the ability to control fan speeds, board power consumption by the local system manager or a remote management entity in response to local environmental factors or the requirements of other system resources.

AdvancedTCA utilizes the concept of field replaceable units (FRUs). These could be node or switch boards, fan or blower assemblies, power inlet modules, power bricks or individual mezzanine modules on a carrier card. Any FRU which includes its own IPMI controller is referred to as an intelligent FRU and can be can be addressed and controlled by either or both of the two system management busses. All node and switch cards must be intelligent FRUs. Other components or subsystems such as blowers, blower assemblies or power supplies may or may not be intelligent FRUs.

Groups of FRUs can be aggregated and managed by a single IPMI controller. In that case, the individual components can be represented by their host controller and still managed by the system controller through the host controller or the host controller can simply control the subsystem locally and in this case it will exist as a single managed entity to the shelf manager. Advanced Mezzanine Modules are examples of managed FRUs that would be intelligent but hosted by an IPMI controller on the carrier card that hosts the modules. Such AMC modules would be visible entities and directly managed by the shelf controller through the local IPMI host on the carrier card.

Power is typically supplied to an AdvancedTCA chassis through a power entry module (PEM) which is another FRU. This component provides filtering to control conducted emissions, over current protection and transient fault protection. An IPMI controller could be added and then it would make sense to have a current or voltage monitor as well. Fusing and diode protection and inrush current limiting would usually be located

Although not spelled out within the specification, the PEMs are ideally located on either side of the chassis because plant wiring practice is usually to bring one battery feed down each side of a rack. It is ideal that the cables connecting to the PEMs do not cross in front of any other cabling or removable units such as fan or filter assemblies or cards.

The PICMG 3.0 requires that each slot be able to dissipate 200 watts of continuous power. This management system, however, makes it practical to monitor and control any level of power consumption that the system designer provides for. Already some customers are asking for 250 watts per slot and others are asking for 300 watts per slot.

It is nearly unimaginable that three ATCA chassis with sixteen 300 watt slots thereby consuming 14.4KVA could possibly exhaust that much heat. However, because of the flexible aTCA power and system architecture, a chassis that is equipped to provide that much power to any given slot could selectively permit any one card to utilize the maximum power while at the same time limiting other adjacent cards to compensate.

The ATCA specification provides for two 48VDC power circuits: “A” and “B”. Power distribution schemes are defined that allow both power circuits to be fed separately or if one battery feed fails a special circutit will allow the single remaining battery source to feed both the “A” and “B” backplane power domains. In addition, wiring instructions are defined to allow the backplane to be supplied by separate power systems. This redundancy would allow each switch slot in a dual star system to be on different power circuits. In addition it is also possible to have separate power domains for each node card of a pair. By such power schemes, highly fault tolerant systems can be constructed.

MicroTCA

The MicroTCA specification has not been completed at the time this article is being written. However, enough of the features have already been defined so that some detailed comments can be made with confidence.

MicroTCA is a backplane architecture designed to support large numbers of removable AMC modules outside of an ATCA chassis. The MicroTCA backplane and system architecture, however, manages the individual AMC modules much the same way that these same modules would be managed if they resided instead on a carrier card in an ATCA slot. In fact the control system of a MicroTCA chassis is referred to as a virtual carrier manager (VCM) which is an acknowledgement that it is serving instead of an actual carrier card. Switching could be provided by an actual AdvancedTCA switch card, however, the mechanical requirements of the MicroTCA architecture and the signaling needs of such a system make this unlikely. However, the interface for the VCM and the MicroTCA switch is as yet undefined.

The MicroTCA specification defines the necessary wiring for a variety of different power source voltages. These power supply environments include: 48VDC battery plants, 60VDC battery plants, 24VDC battery plants, 100V AC mains, 120V AC mains, or 230V AC mains.

A modular power supply interface connector has just been defined by the MicroTCA working group. It is intended that a variety of common modular power units will become commodity components and any MicroTCA system can be equipped with the proper power supply modules for the power source environment where it is to be deployed.

The major difference between a MicroTCA chassis and an AdvancedTCA chassis is the power distribution approach. Unlike ATCA which only provides 48VDC on the backplane, a MicroTCA backplane provides 12VDC and 3.3 VDC. The reason is that the AMC modules expect these voltages to be supplied and do not have provisions to be powered directly by 48VDC.

Because in an aTCA system the 12VDC and 3.3VDC are provided by the carrier card on which a given AMC module is supported, this means that in a MicroTCA system where the AMC modules plug directly into a common backplane, those voltages must be distributed to each slot by the backplane.

A typical MicroTCA shelf will have one or two VCM modules which will serve as the local shelf manager. The MicroTCA document requires that a single modular power supply be able to supply at least one VCM but not more than two VCMs. This is intended to force a level of granularity such that, the failure of a single power supply could never cause the failure of more than a single shelf. A VCM cannot draw more than 80 watts of power.

Interestingly, the power up sequence defined within the MicroTCA specification can take place without the intervention or presence of a VCM. However, start up power is managed initially at a defined low level and more power is negotiated in a similar process to that in an ATCA system. The actual initial start up power is contained in the FRUs virtual carrier activation and current management record. This is within an EPROM that the VCM interrogates during initialization. If no record is discovered default values are used.

The shelf manager communicates with the VC controllers over an IP capable channel. This means that the shelf manager can reside almost anywhere including outside of the immediate shelf.

AMC modules come in four physical form factors of single and double width and full and half height versions. A single AMC module of the largest size (double width, full height) is to draw no more than 80 watts, and the smallest module (single width, half height) is to draw no more than 20 watts.

Because a MicroTCA chassis is intended for deployment in a 300 mm deep front load environment, cooling can be a real challenge. At the present time various cooling models are being considered but no solution is obvious. Due to this challenge and the limited space, solutions to this design challenge will most frequently be dependent on the regulations that apply to the end customer location.

CompactPCI Express

CompactPCI Express is the natural migration path for CompactPCI systems. There are two aspects of CompactPCI Express that will dominate all engineering options.

Each slot type has different power requirements which can be seen in the table below. A quick calculation shows that if all pins in a system slot were used to their maximum, the slot could draw some 480 watts. To keep power consumption under control, there is a stipulation that the combined current of all inputs for a system slot cannot exceed 45 amperes. A Type 1 slot has a similar stipulation of 50 amperes for the maximum combined power.

CompactPCI Express implements hot plug in accordance with the PCI Express Base Specification 1.1 and the electrical requirements also included in the PCI Express Card Electromechanical Specification Revision 1.1.

The cPCIe specification permits power entry in accordance with PICMG 2.11 which uses the 47 pin Positronic power connector. This is the preferred implementation when a pluggable modular power supply needs to be used. The more typical implementation is for pressfit power studs that are intended to be cabled to directly from a fixed power supply using standard cabling, ring terminals, loc washer and hex nuts.

Although cPCIe makes provisions for the PC “wake on LAN” function, this backplane architecture is intended for embedded autonomous applications. The bandwidth made possible by the differential PCIe signaling allows individual slots to have huge dedicated bandwidth.

The electrical requirements tightly defined with regards to minimum decoupling capacitance, noise, and ripple. For hot plug boards there are additional requirements for maximum current slew rate, initial hot-plug capacitance, peak precharge current and maximum board capacitance.

A new version of the 2mmHM connector provides a closed end wall designed to accept a properly keyed daughter card and exclude other cards. This is particularly useful for 6U systems. The entire maximum implementation of cPCIe is accomplished in a 3U height. When a 6U board is supported, the conventional 2mm HM P3, P4 and P5 connectors are used. This means that other standards such as PICMG 2.16, H.110 and StarFabric could all be supported on the new modules.

VITA 41 VXS

The VITA 41 standard is a convenient hybrid accommodating the established VME64x architecture but providing a much higher center connector P0/J0 that supports four serial lanes of 3.1 to 6 Gb/s for a cumulative bandwidth of 10 to 20 Gb/s for each of two fabrics.

With the exception of the new MultiGig P0/J0 connector, supporting two high speed fabrics and IPMI signals and a combined keying and guide module, the payload cards are unchanged. The two switch cards in the case of a dual star implementation are implemented entirely with 5 new MultiGig connectors and a six pin Positronic power connector. This switch card power connector can supply 150 watts at 5VDC.

System management for VXS is based on a redundant set of radial IPMI signals. These signals are brought from the switch slot via the SMB extension connector to the IPMB shelf manager. The VXS system management architecture is defined within VITA 38 which is, in turn, based on PICMG 2.9. This means that all the system management features that can be accomplished in a typical AdvancedTCA or MicroTCA system can be implemented in VXS systems. One major difference, however, is that although the base specification for VXS which is VITA 41.0 only requires that the backplane signal routing necessary to support the system management bus is present. The utilization and implementation of specific management functions are left to the subsidiary specifications. Unfortunately, the full benefit from this capable management architecture will rarely be achieved unless users are conscientious in specifying VITA 38 compliant shelf managers as a requirement with each purchase.

VITA 46 VPX

VITA 46 is the very latest VME derived architecture. It may not be fair to judge this standard as it is not yet complete. However, in contrast to aTCA, the VITA 46 55 page document is, at present, a truly minimalist approach; devoting only two sentences to system management and a single page to power issues. Although VITA 38, the system management standard is referenced and the I2C and JTAG signals are required in the backplane, modules are not required to implement or support either function.

VITA 46 continues to use system reset (SYSRESET#) as its main control mechanism. This feature is required and this suggests that VITA 46 is targeted at stand alone applications where human intervention will be immediately available in the event of any system failure.

At present, backplane power voltage is designated to be between 5VDC and 50VDC and held to within 10 percent of the nominal and peak-to-peak ripple is not to exceed 10 percent. There are 6 contacts provided for +Vs1 and an equal number of –Vs1 return contacts. There are also 6 contacts provided for +Vs2 and an equal number of –Vs2 return contacts. Each contact is rated at 8 amperes.

With 48VDC supplied to the backplane this could provide for a whopping 4,608 watts of power for each slot. Even with Vs1 and Vs2 allocated to 3.3VDC, there would be provisions for 422 watts of power per slot. Luckily, VITA 48 is a mechanical chassis system that provides for liquid cooled modules. This could be the explanation for this unusually generous power allocation.

There are no features presently defining power inlet module function, power initialization function, system management command set, conduction emissions or negotiated power usage. There is no information regarding the implementation of multiple power domains or wiring for N+1 power redundancy.

This bus architectural standard provides a location for VME signals but the implementation implemention of these signals is entirely optional.

Conclusion

As it has been shown, new architectures are not all implementing the same provisions for either system management or power distribution. However, where power distribution has been addressed, the trend is clearly a 48 volt distribution. Even for systems that are intended to be powered off of 120 VAC or 230 VAC mains, 48VDC distribution still offers an important advantage by allowing lower current levels at the backplane interface while still meeting higher power needs. Where this trend has not been observed is for desktop and standalone embedded architectures.

Probably the largest body of new specification language has been the definition of a comprehensive system management architecture. Both VITA and PICMG have developed parallel system management architectures which make use of I2C IPMB circuitry. These two standards, PICMG 2.9 and VITA 38, have nearly identical capability. However, unless the base system specification defines a message and address structure as well as various system control states the benefits of this management technology will not be realized. For systems that are intended to be operated remotely or with high levels of reliability this level of compliance must be mandated.

Top