SPEC7 Specification gathering
This wiki page describes different discussion points for the specification of the SPEC7.
Aim and Use cases
NIKHEF/JRP, CERN: as high-precision PCIe and stand-alone White Rabbit design
* Aim: new PCIe FMC carrier that is optimised as White Rabbit node with a target frequency instability of 1e-13 over 100 sec.
* Users will be first: research institutes and metrology institutes with the aim to disseminate UTC from metrology partners to European research institutes and industry partners. Later: high performance users, telecom, ....
- All users profit from an inherent high performance, relatively cheap (?), design without the bother of buying an expensive stable external oscillator that the metrology users need.
* Other possible user for 1 GSPS ADC cards requiring HPC FMC? Needs to be worked out as fast ADCs nowadays use many serial links and have no parallel readout.
* Quantity: 100 in 5 years for metrology and research needing high timing performance.
* Priority
- optimise for performance
- Try out ideas of presentation High quality standard standard frequency transfer, 2016, NIKHEF & CERN
- (cost)
CERN: as SPEC replacement
* Aim: smooth replacement of existing SPEC that is a cheap Simple FMC carrier with WR support.
-
It is believed that an ADEV of 1e-13 over 100 sec should be achievable on the SPEC7 if the same oscillator is used as in the WRS Low-Jitter Daughterboard (a 4-layer PCB).
-
Production needed by 2019 as Gennum PCIe chip not available anymore.
-
Long development needed: HW design, use of other PCIe bridge, become acquainted with partial reconfiguration.
-
The existing SPEC with WRS are the two flagship OH projects and the SPEC7 therefore may not fail as project.
* Used by
-
DIO
-
ADC100M
-
TDC
-
Fine Delay
-
WorldFIP master
-
and other FMC mezzanines.
* Users
-
CERN
-
Many current SPEC users have their in-house designed FMC mezzanines. Should provide a compatible alternative.
* Quantity: 500 cards expected in 5 years
* Priority
- cost
- performance
Under discussion
FPGA family: Xilinx
- Kintex 7 "Optimized for Best Price-Performance"
-
FBG676 package allows 70T, 160T, 325T and 410T sizes on the same footprint.
- XC7K160T-2FBG676C (~260 Euro) (default type?)
- XC7K70T-2FBG676C: (~180 Euro)
- cf. XC6SLX45T-3FGG484C on SPEC: (~84 Euro without STEP pricing)
- 8 GTX Transceivers max (4 used for PCIe, 1 for SFP, 1 for
FMC - as on SPEC - and 2 for SATA).
- A 70T in FBG484 package cannot be used as has only 4 GTX Transceivers (No need to check if can be used on footprint of FBG676)
- Smallest Kintex7 with 16 GTX transceivers: XC7K325T-2FBG900C (~1100 Euro)
-
- Artix-7 "Transceiver Optimization at the Lowest Cost and Highest
DSP Bandwidth"
- only if LPC
- FGG676 package allows 75T and 100T on same
footprint.
- XC7A75T-2FGG676C (~125 Euro)
- XC7A100T-2FGG676C (~180 Euro)
- FBG484 package allows only 4 GTX transceivers (1 used for PCIe 1x(!), 1 for SFP, 1 for FMC as on SPEC and only 1 for SATA instead of 2).
- Zynq 7000 - with ARM Cortex-A9
- Will have a hard processor in the FPGA, saving resources
- Xilinx is interested in this (via Sundance)
- More expensive?
- Has Altera been considered?
FMC HPC or LPC
- use case for HPC
- HPC would be interesting for some applications like high-speed ADCs where you need to go massively parallel to avoid too high bandwidth requirements on the data lines.
- However, not additional GTX lines available (or two when would not have SATA connectors anymore).
- Cannot fully populate HPC lines with proposed Xilinx package (TBC)
- Test and debug using a Xilinx FMC-XM105 debug card
- HPC: need new tests tools for production test
- use case for LPC
- Existing FMC mezzanines: DIO, ADC100M, TDC, Fine Delay, masterFIP, in-house designs of current SPEC users.
- Price estimation:
- LPC ($12.68/connector), n layers, Kintex 70T FPGA
- HPC ($20.40/connector), n+2 layers (or more)
FMC connector
- Vadj
- programmable IC, fixed 2.5V or just a few values and selection with jumpers?
Oscillators
- Connor-Winfield DOT050 VCTCXO
(DOT050V-020.0M)
- Same as used on WRS Low-jitter daughterboard, (~22 Euro)
RAM
- SO-DIMM socket for RAM
- Or directly assembled RAM IC?
- Use case: data memory for storing data coming from ADC card (e.g. fmc-adc-100m14b4cha)
FLASH storage
- MicroSD slot?
- eMMC ?
USB
- mini USB connector (as on SPEC)
- micro USB-OTG (so it can act as host and as slave). But then we should rename the card to SPUEC or SUPEC (U for USB). Perhaps this is a crazy idea, but it is nice to think that users can plug'n'play the SPUEC/SUPEC on their laptop :) (FV)
- The current mini-USB can serve two UARTs over the same single mini-USB connector if we use a CP2105 - Dual UART bridge. Quite handy if you have a WRPC gui and other logic in your FPGA.
PCB
- 10 layer PCB so that layout can be optimised for low-jitter oscillator (really needed?). HPC would surely need many layers.
- Compare to
- SPEC has 6-layer PCB
- WR Low-jitter board has just 4 layers
Additional connectors
- High-speed connector for PPS in/out, 10MHz in, tx-abscal, rx-abscal, refclock. 6 signals to be transferred in a differential way Samtec Bulls-Eye connector (just a land pattern for 22 signals on the PCB).
Other remarks from NIKHEF
Some extra's that were on our list:
- We need high performance timing-IO signals (including absolute calibration signals) on (accessible) connectors. The current DIO is really bad for timing. We (Guido) should look into this. A possible candidate is a Samtec Bulls-Eye which is just a land pattern on your PCB. But we should study this since a Bulls-Eye is the perfect phase plane reference but the phase plane will not be very accessible from the outside world (i.e. on the PCI bracket).
- Timing signals should be re-clocked with high speed FFs outside the FPGA. I remember that this was an issue that you CERN guys also proposed earlier.
- We need to think of some digital interface (including connectors) to close an external PLL loop using an external high performance oscillator. Ideas are still vague at this moment but the interface as such should already be on the list.
Accepted
SATA
- Keep same two SATA connectors as on SPEC.
- Planned to be used on SPEC for daisy-chaining triggers between SPECs with FMC-ADCs with modified gateware. However, no users that we know of now (WRXI may use it, and external users possibly?)
- Is a connector specifically for internal use (i.e., not panel-mounted).
- 8 GTX Transceivers max (4 used for PCIe, 1 for SFP, 1 for FMC - as on SPEC - and 2 for SATA).**** May be used as well for digital interface for external PLL loop.
Rejected
SFP
- Double SFP for allowing redundant links?
- not possible to fit on PCIe front-panel
- Image of double FMC cage with LEDs
- Image of double decker FMC cage with LEDs":https://media.rs-online.com/t_large/R7871705-01.jpg
- Could maybe fit if would make 2-slot card when optional heatsink/fan solution that extends it into the neighboring slot. Double decker would not work here though.
- Compact SFP (CSFP)?
- Has two bidi standard SFPs in one SFP module
- CSFPs have a little different pinout comparing to the standard SFPs. Same mechanics as standard SFP (Wikipedia).
- Difference between SFP, bidi SFP and CSFP
- CSFP Delta CSFP-33-A4K1DBT (pinout on page 4)
- Check if use of CSFP would limit the use of standard SFP modules
- Actually cannot be used to daisy chain modules: "The compact SFP (GLC-2BX-D) usually uses 1490nm to transmit signal and the 1310nm to receive signal".
29 January 2018