Simple PCIe FMC carrier 7 (SPEC7)

Project description

The FMC PCIe Carrier called SPEC7 is an FMC carrier that can hold one FMC card and an SFP connector.
This board uses a Xilinx Zynq FPGA with Dual-Core ARM processor integrated and is optimised for low jitter and cost and is usable with most of the FMC cards designed within the OHR project (e.g. ADC cards, Fine Delay).
The design is done in a collaboration between NIKHEF (NL) and CERN as both need a card with a similar functionality.

The SPEC7 is the follow-up of the SPEC of which the design started in 2010 and for which certain components are obsolete.
Boards with a very similar architecture are available for the VME bus (SVEC - Simple VME FMC Carrier) and for the PXI Express bus (SPEXI - Simple PXI express FMC Carrier Board).
Other FMC projects and the FMC standard are described in FMC Projects.

Preliminary view of the SPEC7. It will be a follow-up of the SPEC board

Main Features

• 2-lane PCIe Gen2
  • Same 1 GByte/s total speed as 4-lane PCIe Gen1 on SPEC. Bridge integrated in FPGA
• Xilinx Zynq FPGA with Dual-Core ARM processor integrated
  • XC7Z030-1FBG676C (-1= slowest, commercial temp range, fast enough for most applications)
    • 4 GTX receivers (2 used for PCIe, 1 for SFP, 1 external accessible, 1 for FMC - same as on SPEC)
    • 2 GTX Reference Clocks (1 for PCIe, 1 for WR Clock)
    • Note: FMC Multi Gigabit Tranceiver usage not supported for Z030 due to lack of GTX Reference Clocks
  • Possibility to mount Z035: XC7Z035-1FBG676C or XC7Z045-1FBG676C
    • 8 GTX receivers (2 used for PCIe, 1 for SFP, 1 external accessible, 45 for FMC)
    • 4 GTX Reference Clocks (1 for PCIe, 1 for WR Clock, 1 for FMC, 1 external accessible)
    • FBG676 6.6 Gb/s (-1 speedgrade), FFG676 10.3125 Gb/s (-2 speedgrade) 12.5 Gb/s (-3 speedgrade) GTX transceivers (DS191, Table 91)
• FMC slot with high pin count (HPC) connector
  • Fully populated LA bank
    • All 34 differential pairs connected, 1 GTP transceiver with clock, 2 clock pairs, JTAG, I2C (as on SPEC)
    • Z035 and Z045 support 4 GTX transceivers DP[3:0]_M2C/C2M
  • Under discussion. Possibly only GTX lines connected and Vadj fixed at 2.5V (23/10/18)
  • Vadj fixed 2.5V (HR bank of the ZYNQ only allows LVDS_25 at VCCO 2V5, see UG471 table 1-43) -1.8V not possible due to LVDS level restrictions on the 7-series HR-IO banks, as can be read on page 91 of the manual
    • 2.5V can only be used with a LPC FMC mezzanine: only the LPC pins (LA bank) are 2.5V tolerant.
  • Limited, partial connectivity of HPC part, 1.8V tolerant only
    • xx signals on HA bank
    • HA bank only 1.8V tolerant (Vadj set to 1.8V)
    • only when Xilinx Z035 or Z045 is mounted, the DP1_M2C/C2M and DP2_M2C/2CM are connected to two GTX transceivers
• Clocking resources
  • 1x Fixed frequency 33.33 MHz oscillator for Application processor unit (APU), (frequency?)
  • 1x 10-280 MHz VCXO controlled by I2C and a DAC with SPI interface. Starts up at 100125 MHz (Silicon Labs Si570/Si571, freely usable)
  • 1x 125.000 MHz VCXO controlled by a DAC with SPI interface (used by White Rabbit PTP core) (Crystek CVPD-922 or Abracon ABLANO model)
  • 1x 124.992 MHz VCXO controlled by a DAC with SPI interface (used by White Rabbit PTP core) (Crystek CVPD-922 or Abracon ABLANO model)
  • Low jitter external 10MHz via AD9516 (supporting mode Grand Master & AbsCal)
• On board memory
  • 1x 8 Gbit (1 GByte) DDR3 IC connected to the 32-bit wide Memory Interface (main use for the APU) - a 4 GByte SO-DIMM module cannot be used: has 64-bit interface, while the FGPA can only handle 32-bits and1 GByte max)
  • 1x 8 Gbit (1 GByte) DDR3 IC connected to the programmable logic (32 bit wide)
    • for bandwidth reasons
    • possibly a second 8 Gbit DDR3 IC to prevent the DDR3 problem of the latency and turnaround from READ to WRITE – two banks solve the problem.
  • 1x QSPI 256 Mbit flash PROM for multiboot FPGA powerup configuration, storage of the FPGA firmware
  • 64K (8K x 8-bit) I2C Serial EEPROM (24AA64T-I/MC) for storing serial number, calibration parameters and other critical data such as the MAC address of the card
  • 2x 2K (128 x 8-bit) I2C Serial EEPROM (24AA025E48) which provides EUI-48
    • 1 for APU MAC address
    • 1 for WR MAC address
  • 64 MB static memory (needed for booting?)
  • MicroSD slot for flash memory for storing programs
• Miscellaneous
  • Thermometer (XADC) and semi-unique ID (DNA_PORTE2) provided by the FPGA
• Front panel containing
  • 1x Small Formfactor Pluggable+ (SFP+) cage for fibre-optic transceiver (WhiteRabbit support). 1.25, and 2.5 Gbps
  • Programmable Red and Green LEDs
  • FMC front panel
• Internal connectors
  • 1x JTAG header for Xilinx programming during debugging
  • 1x USB-Cmini USB Type B connector
    • Can serve two UARTs over the same single mini-USB connector with CP2105 - Dual UART bridge IC
      • One to UART interface of the ARM
      • One to user logic (e.g., PTP core)
  • 1x USB-C USB Type A connector connected to USB 2.0 port of the ARM
  • Ethernet RJ45 connector + magnetics and MicroChip KSZ9031RNX, 10/100/1000 Mbps PHY (interface to ARM GigE)
  • Samtec Bulls-Eye connector (BDRA: 20 signals on the PCB, 1.3 x 5.0 = 6.5 cm2 land pattern, user mountable connector) or (BARA: 22 signals on the PCB, 1.8 x 2.1 = 4 cm2 land pattern, user mountable connector); BDRA probably better supported but slightly bigger, final choise BDRA/BARA will depend on PCB layout
    • ESD protection on all signals
    • 10 differential signals:
      • 125 MHz reference clock in (Z035/Z045 only)
      • 125 MHz reference clock out
      • 1x tx-abscal
      • 10 MHz reference clock in/out
      • PPS in/out
      • 1x GTX Tx/Rx (using 125 MHz reference clock; optional other reference clock when using Z035/Z045)
      • 1x spare
• 1x connector for optional cooling fan
• FPGA configuration. The FPGA can optionally be programmed from:
  • JTAG header
  • QSPI 256 Mbit FLASH PROM
  • User FPGA logic: via PCIe or ARM (i.e. using Internal Configuration Access Port: ICAPE2 [PS PCAP / ICA](see chapter 6.1.8 Zynq-7000 SoC Technical Reference Manual)
• Stand-alone features
  • External 12V 150W-ATX power supply connector
  • USB Type A connector
  • USB-Cmini USB Type B connector
  • 10/100/1000 Mbps copper Ethernet RJ45
  • SFP+ cage for fibre-optic transceiver(WhiteRabbit support)
  • 47x LEDs (2x front pannel, 4x on PCB, 1x PCI SMB-bus)
  • 25x buttons
    • 1 PS_POR connected to reset controller
    • 1 PS_SRTS_B
    • 1 PL system reset
    • 1 general purpose
    • 1 PROGram button for FPGA
• Power consumption: 5-1210-35 Watt, depending on application
• 14-layer PCB
• Optional cooling fans
  • for the mezzanine
  • for the FPGA

Project information

• Official production documentation
  • Schematics (Note: Design created with Mentor Graphics using a Xpedition).
• Design Specification
  • Specification gathering discussion - a list of wish items ...
  • Feature Alternatives - ... boiling down to a few alternatives ...
  • Design specification - ... to a precise specification that allows the design to start. A cleaned-up version can be found on this very page at Main Features.
  • Component Selection - ... list of selected components.
• Users
• Work partially funded by the EU EMPIR WRITE project
• Frequently Asked Questions

Contacts

Commercial producers

• Once designed and debugged, the board will be commercially available.

General questions about project

• Erik van der Bij - CERN
• Peter Jansweijer, Pascal Bos, Guido Visser - NIKHEF

Status

16 August 2019