Te c h n o l o g y W h i t e P a p e r

UNITRONIXPtyLtd
POBox486,MorissetNSW2264 NSW:Tel:61249773511Fax:61249773522 WA:Tel:61894552424Fax:61894552458 unitsyd@unitronix.com.au www.unitronix.com.au

How Does the FMC (FPGA Mezzanine Card) Standard Measure up Against the PMC/XMC Format for Embedded Defense/Aerospace Applications?

Abstract Interest in reconfigurable embedded computing in the defense and aerospace market has grown significantly as new generations of FPGAs present developers with a level of processing performance and potential I/O bandwidth that cannot easily be matched by conventional CPU configurations. There are many COTS solutions that allow developers to readily make use of FPGAs for processing, but the real challenge to an application is often measured in terms of I/O bandwidth, latency and connectivity. For example, military Electronic Counter Measures (ECM) applications require high bandwidth data input, processing and data output with minimum latency. FPGA Mezzanine Card (FMC) directly addresses the challenges of FPGA I/O by solving the dual problem of how to maximize I/O bandwidth, while still being able to change the I/O functionality. The elegance of the FMC solution is in its simplicity. On FMC modules there are only I/O devices, such as ADCs, DACs or transceivers. The modules have no on-board processors or bus interfaces, such as PCI-X. Instead, FMC modules take advantage of the intrinsic I/O capability of FPGAs to separate the physical I/O functionality on the module from the FPGA board design of the modules host, while maintaining direct connectivity between the FPGA and the I/O interface.

Introduction Darwins theory of evolution doesnt necessarily apply to just the plant and animal world, as evidenced in the embedded computing industry, where only the fittest mezzanine card formats have survived. A wide variety have come and gone, with only the best formats gaining broad market appeal, with some specializing and excelling in niche areas. Others have been consigned to the drawing board of history. The reasons for this are many. Mezzanines for Rugged Computing Perhaps the strongest mezzanine format for defense embedded computing is PMC (IEEE 1386.1-2001[1]), which uses the PCI and more recently PCI-X bus (ANSI/VITA 39-2003[2]), and offers higher levels of ruggedization defined in ANSI/VITA 20[3]. PMC has succeeded because it has been able to evolve through speed improvements and environmental specifications. PMC has also been able to meet a wide range of market needs including sufficient space to implement useful functionality. PMCs latest incarnation is XMC (ANSI/VITA 42.0[4]) where the parallel PCI or PCI-X bus has been replaced with a serial interface, of which the most common protocol supported is PCI Express (ANSI/VITA 42.3-2006[5]). Interfaces such as PCI, PCI-X, PCI Express and Serial RapidIO have evolved to address the needs of computer systems dominated by conventional CPUs, and the need for standard interfaces that abstract the specific details of their hosts.

Page 1

cwcembedded.com

However, as the performance barriers are pushed back, some applications look toward FPGAs as the only practical way of achieving the necessary throughput, which could be beyond the capabilities of PMC or XMC. FPGAs can also be used to implement the necessary interfaces, so advantage can be made out of the direct coupling of processing performance and I/O bandwidth. This applies very well to applications such as Electronic Counter Measures (ECM), which can require latencies and bandwidth exceeding the theoretical capabilities of PCI Express (2Gbytes/sec using a x8 interface for PCI Express, Generation 1). PCI Express or PCI-X latency can be in the order of 1-2S. FPGA Mezzanine Cards (FMC) FMC (ANSI/VITA 57.1-2008[6]) is aimed at solutions that require the benefits of an FPGA. This standard is showing a lot of promise in terms of longevity, as it does not compete with PMC/XMC the recognized mezzanine leader for rugged computing but rather solves a problem for high-end applications (Figure 1 shows a typical FMC module). In fact, the FMC specification leverages some of the goodness of the PMC and XMC specifications. The alternative for using FMC for high bandwidth, low latency data throughput is not to use mezzanines at all, but a monolithic PCB, a route that would lose the advantage of flexibility.

Figure 1: Typical FMC Module (Curtiss-Wright FMC-516 ADC [8])

Mezzanine cards for FPGA-based solutions are not new, but they are invariably based on a proprietary standard which means users are locked into a particular vendor, and the evolution of the standard is not subject to peer review. The FMC standard has been ratified for three years now, so it is worth reviewing how successful the FMC specification has become. But first, it would be useful to review what the FMC specification entails. The general connectivity is outlined below in Figure 2 and includes a large number of parallel and serial connections directly to a host based FPGA.

Figure 2: Summary of FMC connectivity

Power Supply
FMC (VITA 57) Connector

Power (12V, 5V, Vadj), JTAG, I2C, Clocks

High-speed Serial Connectivity

Up to 10 MGT pairs

Analog, Digital, Fiber, Video, FPGA/CPU, Memory, etc.

HPC: Up to 160/80 (SE/Diff pairs) or LPC: Up to 68/34 (SE/Diff pairs)

Parallel Connectivity

Page 2

cwcembedded.com

Host FPGA Interface

Parallel or High-speed Serial I/O Device(s)

The purpose of the FMC specification is to allow (usually) one FPGA on a host card to connect directly with the I/O devices on the mezzanine module just as if the device were on the host board. Busses like PCI-X are redundant and would get in the way of the FPGA and its I/O devices. This intimacy means the interface can be optimal and savings can be made in real estate, cost and power, while boosting bandwidth and reducing latency. An FMC is similar in height and width to a PMC, but around half the length. The reduced width, compared with PMC or XMC, allows up to three FMCs to be fitted to a 6U host. The FMC specification has a default stacking height of 10mm, but permits a stacking height down to 8.5mm for low profile solutions. The majority of FMC host/carriers use 3U/6UVPX, VXS and AMC formats but there are also PCI Express solutions such as the Xilinx ML605 Virtex-6 evaluation card[7]. The FMC specification provides for a large number of differential connections up to 80 pairs or 160 single-ended signals to support high speed parallel interfaces between the FPGA and I/O devices. There are also a number of serial connections (up to ten pairs) suitable for Multi-Gigabit Transceivers (MGTs) operating up to 10Gb/s. FMC modules and hosts support two connector options; a Low Pin Count (LPC) 160 pin connector or High Pin Count (HPC) 400 pin connector. The majority of FMC solutions are likely to use the HPC variant. Although aimed at I/O, FMC can be used for any function that might connect to an FPGA including DSPs, memory or even another FPGA. Connectivity Connectivity for FMC modules is unusual in that only the upper limit of active connections is defined, as opposed to the number of active connections. This means that host carriers need not provide the same number of FPGA signals as another host. This matters only in defining a given hosts ability to support certain FMC modules. To fully populate an HPC
Page 3

solution may require a large FPGA, so reduced pin-out offers cost sensitivity. This is something to be aware of, but the specification defines the signals to populate the LPC or HPC connector starting at a given position and add to the connector in a given sequence, such that if two hosts provide x signals, they will use the same connector pins and be compatible. Power Supply For power supply requirements, the FMC specification employs a useful trick, at least for the FMC: the host detects what the FMCs power should be on its primary power rail and provides it. This is achieved through the host interrogating the FMCs E2PROM, coupled with an adjustable power supply. The benefit to the FMC is a simplified power requirement, thereby freeing up valuable real estate for more I/O on the FMC. Usable Printed Wafer Board (PWB) Real Estate Although occupying around half the PWB area of an XMC, the FMC can sometimes achieve greater I/O functionality, most notably for rugged applications. If the solution requires a large FPGA and if the XMC module complies with the VITA 20 specification, there are restrictions as to where the FPGA can be located. In turn, this may limit the available area to fit the I/O devices. Consider an actual example with a pair of designs using the same I/O devices for a rugged application; one using an XMC format card and one using an FMC format card (Figure 3). Since the rugged XMC specification requires an area across the middle of the board to mate up with a host stiffening bar (which doubles as a primary thermal interface on conduction-cooled variants), a large FPGA (for example 35x35mm) invariably needs to be fitted to the area of the circuit board closest to the front panel, just where the design would want to fit the I/O devices. The useful space in which to fit the I/O devices is perhaps a quarter of the overall real estate of the XMC and not very efficient. In comparison, the FMC, though around half the size of the XMC, has a far greater real estate area for the I/O devices. In this example, the FMC is able to support two ADCs for two 3GS/s channels compared cwcembedded.com

to the single channel of the XMC. Of course an XMC using a smaller FPGA, or not restricted by the rugged XMC specification, may not be affected to such an extent, provided it still has a sufficient number of I/O connections to the devices. An FMC may be smaller, but it may still be able to support greater functionality than its larger XMC equivalent. Figure 3: Rugged XMC with large FPGA real estate example comparison with an FMC

Cooling Cooling rugged FPGA-based XMC cards can be a significant challenge as such boards can easily exceed 20W power dissipation. Typical rugged aircooled specifications for such cards define upper air-inlet temperatures of 70C and conduction-cooled cold walls of 85C. Making a mezzanine work within this environment with much lower power levels is already difficult. To compound this, XMC hosts often have two XMC sites. Consider the size and orientation of an XMC. When plugged onto a 3U host card, such as 3U VPX, the XMC covers the majority of hosts real estate which means, if there are any hot devices on the host, they are under the XMC. This is not ideal and can seriously affect cooling. The XMC mezzanines devices face down onto the host, not the outside, placing the heat generating devices opposite those on the host and compound the cooling problem further. To cool the XMC, the air needs to be squeezed between the host and mezzanine which can be a very small cross section, thus limiting the volume of air available to cool the assembly. Conduction cooling is less difficult but having all the heat generators in one plane is still a problem as there may be hot spots. A 6U solution is not really any better; some of the hosts real estate is not covered by mezzanines, but the thermal paths to either the cooling air inlet or cold wall interface are longer. Cooling is another advantage for FMC based designs. An FMC, being smaller than an XMC, ensures that a larger amount of space on the host carrier is not covered by the mezzanine itself. Appropriate FMC host design allows for suitable heat sinks to be implemented in the areas not restricted by mezzanine placement. Superior cooling with either greater airflow or greater heat sink cross sectional areas may be the factor that determines whether the solution is viable. In addition, as the FMC has no FPGA, only I/O devices, the FMC will be easier to cool as well especially if the devices are not above a hosts thermal hot spot (see Figure 4). The FMC specification limits the power dissipation of a single width module to 10W.

Page 4

cwcembedded.com

Figure 4: 6U VPX FMC host with two FMC sites (and 4x FPGAs)

Bandwidth XMCs might get around the bandwidth problem, compared with FMCs, through decimation or using newer generation serial interfaces operating at ever higher speeds. However, some applications may not be able to tolerate this reduction in data off the mezzanine. Beam-forming applications may fall into this category where high bandwidth data, from potentially a large number of channels, must be shared between processing elements. Therefore, high bandwidth beam-forming is another good application area for FMC technology because it would not suffer from data reduction problems. Simplicity FMCs by nature promise a faster development turnaround cycle, through lower complexity and a focus on the I/O itself within the design. This makes FMCs a good choice for system upgrades as newer, faster and high resolution devices come onto the market. This is an advantage for Software Defined Radio (SDR) applications. There is a great deal of flexibility afforded by implementing different modulation and coding schemes which can be performed in a common FPGA host. Additional flexibility results from being able to upgrade to new higher resolution ADCs as they become available, without the need to develop complex interface structures and new power supplies. FMCs and FPGA processing make an ideal platform for a wide range of digital receivers. Flexibility RADAR has a thirst for increasing resolution and bandwidths, and benefits from direct RF digitization for coherent sampling. Lower bandwidth solutions require IF stages and I/Q sampling which introduces noise. FMC provides the easiest format to upgrade the platform as new faster ADCs become available. Direct sampling of up to X-band frequencies, with appropriate digitizers, means that applications such as marine, ATC, weather and surveillance RADAR are well within the capabilities of FMC bandwidth. Faster ADCs are continually being developed so upgrading is quick and easy.

Key Benefits of FMC Compared to PMC/ XMC Latency In determining which applications best suit an FMC approach, the decision really hinges on whether there is a benefit from using an FPGA. Latency is very low for FMCs. If an FMC supports both input and output, then a solution can be realized in which one or more analog signals are digitized and read directly and processed by an FPGA, with the data then transmitted back out through the same FMC. In this case no busses, not even between FPGAs, are required. The only delay is the FPGA processing which can be as low as a few nanoseconds. This type of solution is ideal for an application like Electronic Counter Measures (ECM) and Electronic Support Measures (ESM) where processing response time is critical.

Page 5

cwcembedded.com

Customization If the appropriate FMC function does not exist, it is now easier for customers to design their own cards to fit on a third party FMC host and more easily track new technology, such as higher resolution ADCs as they become available. The conceptual simplicity of FMC makes this approach more viable and reduces system risk. See Figure 5 for example applications mapping onto different latency and bandwidth requirements. Limitations Finite Connectivity No mezzanine specification is perfect because its target markets are usually varied, and therefore a compromise. If the compromises are few, then these imperfections can be overlooked. Supporting up to 80 differential signals, it is easy to conceive of parallel interfaces providing data throughputs in excess of 10GB/s and largely limited only by the host FPGAs capabilities. However, although 80 differential signals represent significant connectivity, it is finite. A monolithic design, not using a mezzanine, could provide more than 80 FPGA pairs to connect to the devices on the FMC. The practicality of FMCs is down to the host FPGA and I/O devices. An example is a 3.6GS/s 12bit ADC, which exists today and might use 1:4 multiplexer to allow it to be interfaced to an FPGA. Such a device would require 48 LVDS pairs, probably clocked at 450MHz DDR. This would use more than half of the FMCs connectivity for parallel connections for the data path alone, on top of which control signals would be needed. In this example only a single ADC device could be implemented on an FMC even though there may be sufficient space and power budget to fit on a second part. However, by comparison, the performance bandwidth for FMCs over XMCs is considerable.

Front Panel I/O FMC is a front panel only format, unlike XMC and PMC which define user I/O signals that might be routed to a backplane for convenience. However, this is rarely used for analog signals where noise might be introduced through such routing. Mechanically, the FMC front panel width is slightly narrower than its XMC counterpart, leaving less room for connectors. FPGA Centric By its very definition, FMC requires an FPGA. This means the FMC will never achieve the universal appeal of PMC or XMC, but for applications that require such levels of performance as provided by an FPGA, this is of minor consideration. Today, there are relative few FMCs and FMC host carriers, where PMC and XMC have built up an extensive portfolio number in the 100s or even 1000s with proven track records. Cross-Vendor Compatibility By definition, software and HDL for FMCs is defined by the host. And because the FMC is controlled by an FPGA, there is no real concept of a driver. So although FMCs from different 3rd parties will fit together electronically and mechanically, there will be differences in the hosts environment leading to incompatibility. However, within the FPGA world, electrical and mechanical compatibility are of primary performance. HDL is often user specific and often not an issue. FMC Market Traction The FMC format is gaining momentum. There are already more than fifteen vendors providing FMC modules, host or complete combinations. The majority of FMC solutions on the market today are high speed analog input or output with good coverage of channel density, resolution and ruggedization levels. This is not surprising because this is an area that requires raw bandwidth and FPGA processing. In addition, there are now approaching a hundred products covering functionality such as tuners, digital I/O, fiber-optics, 10GEthernet and even FPGA co-processors. There is even product supporting system busses such as 1553. Analog input products include solutions up to 250MS/s 16-bit for high resolution, and 5GS/s (10b) for high speed capability. cwcembedded.com

Page 6

Summary The choice of which mezzanine format is best for rugged embedded computing solutions will ultimately be down to issues such as application details, perception of risk, development timeline and personal preference. The baseline for choosing which mezzanine is most suitable for certain applications is how it compares with a monolithic board (i.e. a single PWB with all functionality onboard). A monolithic card usually provides the best technical solution because it does not have the restrictions imposed by segmenting the design, such as number of connector I/O pins to the mezzanine. While not absolute, Table 1 below provides a first cut assessment as to which technology scores best against different parameters. However, differences in the requirement from one application to another may affect these conclusions. Table 1: Relative comparison of mezzanine capabilities
PMC Bandwidth Latency Installed base of users Time to design new board Power dissipation, ease of cooling Note: (aaa best) XMC FMC Monolithic

FMC Continues to Evolve Although FMC is now a published standard, there are some parallel specifications in development within the VME Industrial Trade Association (VITA). These deal with abstractions to simplify integration into systems such as FPGA driver concepts. When ready, these additional standards will strengthen some of the perceived weakness with FMC. FMC is ready now as an electro-mechanical solution that will serve a wide range of applications now. Conclusion FMCs promise to do for FPGA based solutions what PMC and XMC did for embedded CPU based systems. However, FMC is not really competing with PMC or XMC, but actually complements it, particularly for high bandwidth, low latency applications.
References [1] IEEE 1386.1-2001 Standard Physical and Environmental Layers for PCI Mezzanine Cards [2] ANSI/VITA 39-2003: American National Standard for PCI-X Auxiliary Standard for PMCs and Processor PMCs. [3] ANSI/VITA 20 (R2005): American National Standard for Conduction Cooled PMC [4] ANSI/VITA 42.0-2008: STANDARD FOR VITA 42.0 XMC [5] ANSI/VITA 42.3-2006: American National Standard for XMC PCI Express Protocol Layer Standard [6] ANSI/VITA 57.1:-2008: American National Standard for FPGA Mezzanine Card (FMC) Standard [7] Xilinx ML605 Virtex-6 evaluation card: www.xilinx.com [8] Curtiss-Wright Controls Embedded Computing web site www.cwcembedded.com

a a aaa aa a

aa a aa aa a

aaa aaa a aaa aa

aaa aaa aa a aaa

All other brands and names are property of their respective owners.

Figure 5: How typical applications map onto bandwidth and latency requirements

Copyright 2011, Curtiss-Wright Controls All Rights Reserved. MKT-WP-FMC-052611v1

UNITRONIXPtyLtd
POBox486,MorissetNSW2264 NSW:Tel:61249773511Fax:61249773522 WA:Tel:61894552424Fax:61894552458 unitsyd@unitronix.com.au www.unitronix.com.au
Page 7

cwcembedded.com