Document providing test procedure of the prototype / Functional test of the prototype with a COMe module provided by CERN
Open questions from INCAA
As the PXIeCOMe will be a part of a computer, which it forms together with the COMe-module and the PXIe chassis, the document describing the test will be the documentation of a functional test. This test procedure can be followed manually or can be automated.
The basic setup will consist of the PXIeCOMe (DUT) and some additional required parts.
-
Kontron COMe module; provided by CERN This can be the COMe-bBD6 like discussed before.
-
Kontron COMe module heatspreader; provided by CERN When the COMe-bBD6 is used, the Kontron 68002-0000-99-0 heatspreader (this is the threaded version) is needed. (€18,72 via Mouser)
-
Kontron COMe module mount kit; provided by CERN Kontron 38017-0000-00-5 COMe mount kit. (€6,60 via Mouser)
-
PXIe chassis
The PXIeCOMe is mainly an interconnection board for a number of front, internal and rear interfaces.
- Front USB0 and USB1 (USB3.0)
- Front USB2 to USB5 (USB2.0)
- Front Gigabit Ethernet (RJ45)
- Front Displayport
- Front Serial RS232 (SubD9)
- Front Trigger (SMB)
- Internal mSATA
- Internal Serial RS232 to FPGA
- Internal GPIO to and from FPGA
- Internal COMe specific functions
- Rear Power management
- Rear SMBus
- Rear PXI Trigger bus
- Rear PXI LBR6
- Rear PXI Star trigger
- Rear PXI CLK10
- Rear 4 Links with PCIe 4x
Front interfaces Almost all front interfaces can be connected to a device to test the connections. Like a USB3.0 memory stick for USB0 and USB1, a keyboard and a mouse for USB2, USB3 and another memory stick for USB4. By connecting the RJ45 to a network the test can be performed remotely of a ping command to another computers can be used to test the Ethernet connection. The front Serial interface can be looped back to USB5 using a USB-Serial converter. And the Displayport needs to be connected to a display. The trigger interface needs some extra attention, which I will come back on. But it can be tested using an external digital signal generator and an external digital input, like a counter.
Internal interfaces Only the internal mSATA interface needs a device to be able to test the internal interfaces. The other internal interfaces can be checked using a test specific FPGA design.
Rear interfaces The test of the rear side interfaces is less straight forward. The power management consists of power on/off signals and can easily be tested within a PXIe chassis. The SMBus is not clear yet since it is currently unknown if a SMB device is present on the PXIe backplane and how it could be accessed to check the SMBus connections. Or even if a program can be created of used to access the SMBus. The test of the 4 Links of PCIe 4x (4 lanes) has some options. This also applies to the PXI signals, and is very dependent on the used PXIe chassis/backplane.
The PXIeCOMe provides 4 links of 4 lane PCIe to the backplane. For this reason the PXIe chassis should have a 4-Link configuration. A PXIe chassis with a 2-Link configuration can’t be used! Most of the high-end PXIe chassis by NI have a 2-Link configuration. Every 4 lane link should make a connection to a device to check the interface. Within mostly larger PXIe chassis, so with more than 8 slots (like our NI PXIe-1075), every of the 4 links is connected to a PCIe switch in the backplane. So accessing every PCIe switch and checking its capabilities could be a valid test to check the interconnections of the PXIeCOMe. This can be done using the command lspci under Linux. Smaller PXIe chassis (like the 8 slot NI PXIe-1082) usually have less PCIe switches so an extra PCIe peripheral should be used as an endpoint device to test a 4 lane link. With the NI PXIe-1082 this applies to slot 2, for which a SPEXI module could be used. Unfortunate the placement of a board in slot 2 will prevent easy access to the Xilinx JTAG connector, some LEDs and buttons of the PXIeCOMe during the test.
Larger PXIe chassis also have a disadvantage. Here the PXI Trigger bus is created in segments, usually of about 8 slots, which are interconnected using a PXI Trigger bridge. The PXIe-1075 is an 18 slot chassis with three PXI Trigger bus segments. By default this bridge is not connecting the segments, but this can be configured through NI Measurement & Automation Explorer (MAX). This is available under Windows, but I don’t know how this is done under Linux. Leaving the PXI Trigger bridges in their default state is the easiest way of using the PXI Trigger bus, but limits the locations at which another PXI Trigger device can be placed to test the PXI trigger bus signals of the PXIeCOMe. So to test the PXI Trigger feature of the PXIeCOMe it needs to be checked by a PXI peripheral located within the first segment of the PXI Trigger bus. This can be a SPEXI which is also a 4 lane PCIe peripheral.
Since we have produced and tested the SPEXI, and it as a PXI Trigger bus which needed to be checked during the use of the SPEXI PTS, we have created a test module for it: CPE-PXIT-17208. The CPE-PXIT-17208 is a very simple System Timing Slot device which is capable of placing the PXI_CLK10 clock signal on all PXI_STAR[0..16] signals, and on all PXIe_DSTARA[0..16] differential signals to all other slots. All PXIe_DSTARC[0..16] differential star triggers are connected to PXIe_DSTARB[0..16] as a loopback. None of these differential star triggers are by the way supported by the PXIe System slot and so not available to the PXIeCOMe. The CPE-PXIT-17208 also has a loopback by which PXI_TRIG[0..3] are connected to PXI_TRIG[4..7]. In this way all PXI signals but the PXI_LBR6 and PXI_LBL6 of the SPEXI are tested.
Below I have create a table which shows different setups of the test system and how the PXI and PCIe signals and Links can be tested.
- The FPGA of the SPEXI has to be designed especially for this test.
- When the SPEXI is placed in slot 2, the PCIe access to the SPEXI can be performed using the command lspci under Linux . There is no need for an actual read/write access to the SPEXI FPGA via PCIe.
NI also has the PXIe-1088, which is also a 4-Link configuration backplane, but doesn’t have a System Timing slot so PXI_STAR can’t be tested, and it has three links directly connected to three slots without PCIe switches. So extra PCIe peripherals will be needed to check these connections.
Setup B The PXIe-1075 (like ours) is not sold by NI anymore, but is has many advantages for us, especially in Setup B. The same setup can also be used in a nVent-Schroff PXI-Express System with 18 slots, and with a ADLINK PXIe chassis with 18 slots or 9 slots. In the above table Setup B shows it doesn’t need an extra SPEXI but instead uses two variations of a to be designed test board based on our CPE-PXIT-17208. This new test board also provides a few opportunities of solving some previously mentioned disadvantages. The new board, to be places in slot 2, can be designed to provide access to the Xilinx JTAG connector, the LEDs and buttons of the PXIeCOMe. And it will be capable of providing a SMB connection to the front Trigger so a more automated check of all interfaces is possible.
As stated before nVent-Schroff sells a PXI-Express system (https://schroff.nvent.com/en/schroff/pxi-pxi-express-systems/PXI-Express-System-4-U-18-Slot-84-HP) which has a 4-Link configuration backplane with a PCIe switch connected to each link, and has PCIe Gen 3 and a System Timing Slot. Is costs about €8451 according to the nVent Schroff site.
ADLINK has two PXIe systems with a 4-Link configuration which can also be used. The PXES-2780: https://www.adlinktech.com/Products/PXI_PXIe_platform/PXIChassis/PXES-2780?lang=en (it costs about $7778) and the PXES-2590: https://www.adlinktech.com/Products/PXI_PXIe_platform/PXIChassis/PXES-2590?lang=en (it costs about $4731). The PXES-2785 can not be used as it has a 2-Link configuration, although it has PCIe Gen 3.
These chassis can be used as a replacement for our NI PXIe-1075. Btw the NI PXIe-1075 only provides PCIe Gen 2, like the ADLINK chassis. So our currently preferred setup uses our NI PXIe-1075 together with a new to design test board of which one variant is placed in peripheral slot 2 and the other in system timing slot 10. Using this test board the front trigger can also be tested. The nVent-Schroff or one of the ADLINK could be a future replacement for the NI PXIe-1075, but only the nVent-Schroff seems to provide PCIe Gen 3.
Setup C The NI PXIe-1082 (https://search.ni.com/nisearch/app/main/p/bot/no/ap/tech/lang/nl/pg/1/q/PXIe-1082/) starts from €4130 and will probably be the cheapest chassis, although the use of our PXIe-1075 doesn’t require CERN to provide a PXIe chassis. It has some advantages and some disadvantages, and is not our preferred setup.
Setup D The ADLINK PXES-2590 (https://www.adlinktech.com/Products/PXI_PXIe_platform/PXIChassis/PXES-2590?lang=en) provides a PCIe switch per link and so doesn’t need a PCIe peripheral. The currently available CPE-PXIT-17208 can be used for most of the PXI signal testing. Only PXI_LBR6 can’t be tested without using a peripheral board (like the SPEXI) in slot 2. This setup also doesn’t provide an extra Trigger I/O to test the front Trigger of the PXIeCOMe.
For the use of the PXIeCOMe in an operational system the only required specification for the backplane is its configuration, which should be a 4-Link configuration. So the PXIeCOMe does not demand the use of a NI PXIe-1075, NI PXIe-1082, nVent-Schroff 14579-040 or one of the ADLINK chassis. Other PXIe chassis with a 4-Link configuration could also be used in an operational system.