Skip to content

S
SPEC7

Specgather

Last edited by Erik van der Bij
Page history
This is an old version of this page. You can view the most recent version or browse the history.

SPEC7 Specification gathering

This wiki page describes different discussion points for the specification of the SPEC7.

Aim and Use cases

NIKHEF/WRITE JRP, CERN: as high-precision PCIe and stand-alone White Rabbit design

* Aim: new PCIe FMC carrier that is optimised as White Rabbit node with a target frequency instability of 1e-13 over 100 sec.

* Users will be first: research institutes and metrology institutes with the aim to disseminate UTC from metrology partners to European research institutes and industry partners. Later: high performance users, telecom, ....

  • All users profit from an inherent high performance, relatively cheap (?), design without the bother of buying an expensive stable external oscillator that the metrology users need.

* Other possible user for 1 GSPS ADC cards requiring HPC FMC? Needs to be worked out as fast ADCs nowadays use many serial links and have no parallel readout. With FPGA with 8 GTX, only a single GTX is available for FMC connector.

* Quantity: 100 in 5 years for metrology and research needing high timing performance.

* Priority

  1. optimise for performance
  2. (cost)

CERN: as SPEC replacement

* Aim: smooth replacement of existing SPEC that is a cheap Simple FMC carrier with WR support.

  • It is believed that an ADEV of 1e-13 over 100 sec should be achievable on the SPEC7 if the same oscillator is used as in the WRS Low-Jitter Daughterboard (a 4-layer PCB).

  • Production needed by 2019 as Gennum PCIe chip not available anymore.

  • Long development needed: HW design, use of other PCIe bridge, become acquainted with partial reconfiguration.

  • The existing SPEC with WRS are the two flagship OH projects and the SPEC7 therefore may not fail as project.

* Used by

  • DIO

  • ADC100M

  • TDC

  • Fine Delay

  • WorldFIP master

  • and other FMC mezzanines.

* Users

  • CERN

  • Many current SPEC users have their in-house designed FMC mezzanines. Should provide a compatible alternative.

* Quantity: 500 cards expected in 5 years

* Priority

  1. cost
  2. performance

Single development

  • Reduces development cost
    • Say 100 KCHF development cost, divided by market of 1000 is 100 CHF per board extra available
  • Developments made will be directly available for everyone instead of needing to be ported, reducing largely efforts

Under discussion

FPGA family: Xilinx

Zynq 7000 - with ARM Cortex-A9

  • Has a hard processor in the FPGA, saving resources

  • Xilinx is interested in this (via Sundance)

  • Hardware/Software splitting is mature so will make the SPEC7 very interesting

XC7Z035-2FBG676E (~1100 Euro)* (-2=faster, Extended temp range only, not available in commercial temp range)

** XC7Z035-1FBG676C (~800 Euro)

** 8 GTX receivers (4 used for PCIe, 1 for SFP, 1 for FMC - as on SPEC - and 2 for SATA). Makes it too expensive.

XC7Z030-2FBG676E (~285 Euro)* (-2=faster, Extended temp range only, not available in commercial temp range)

XC7Z030-1FBG676C (~206 Euro)* (-1= slower, commercial temp range)

** Only 4 GTX receivers (PCIe x2, 1 for SFP, 1 for FMC, no SATA anymore)

** comparable to Kintex 125 kcells

** pin-compatible up to 7Z045. (SBG485 package only 7Z030, cannot go larger)

** (FASEC uses also XC7Z030)

  • XC7Z015 is only one smaller with PCIe x4 integrated, but no larger versions in pin-compatible packages available

** like Artix-7 74 kcells

* XC7Z015-2CLG485E (~132 Euro)

** CLG485 package

FMC HPC or LPC

  • use case for HPC
    • HPC would be interesting for some applications like high-speed ADCs where you need to go massively parallel to avoid too high bandwidth requirements on the data lines.
    • However, not additional GTX lines available (just two when will not have SATA connectors anymore).
    • Cannot fully populate HPC lines with proposed Xilinx package and with limited number of PCB layers
    • Test and debug using a Xilinx FMC-XM105 debug card
    • HPC: need new tests tools for production test
    • Check if possible to make compatible to VFC-HD
  • use case for LPC
    • Existing FMC mezzanines: DIO, ADC100M, TDC, Fine Delay, masterFIP, in-house designs of current SPEC users.
  • Price estimation:
    • LPC ($12.68/connector), n layers, Kintex 70T FPGA
    • HPC ($20.40/connector), n+2 layers (or more)

Oscillators

  • Connor-Winfield DOT050 VCTCXO (DOT050V-020.0M)
    • Same as used on WRS Low-jitter daughterboard, (~22 Euro)
  • Two Crystek oscillators suggested by Peter (~20 Euro each)

RAM

  • SO-DIMM socket for RAM (less risk of obsolescence)
  • Or directly assembled RAM IC?
  • Use case: data memory for storing data coming from ADC card (e.g. fmc-adc-100m14b4cha)
  • Concluded: No SO-DIMM as will take more PCB board space, less reliable with connector

FLASH storage

USB

  • mini USB connector (as on SPEC)
  • micro USB-OTG (so it can act as host and as slave). But then we should rename the card to SPUEC or SUPEC (U for USB). Perhaps this is a crazy idea, but it is nice to think that users can plug'n'play the SPUEC/SUPEC on their laptop :) (FV)
  • The current mini-USB can serve two UARTs over the same single mini-USB connector if we use a CP2105 - Dual UART bridge. Quite handy if you have a WRPC gui and other logic in your FPGA.

PCB

  • 10 layer PCB so that layout can be optimised for low-jitter oscillator (really needed?). HPC would surely need many layers.
  • Compare to
    • SPEC has 6-layer PCB
    • WR Low-jitter board has just 4 layers

Additional connectors

Other remarks from NIKHEF

Some extra's that were on our list:

  1. We need high performance timing-IO signals (including absolute calibration signals) on (accessible) connectors. The current DIO is really bad for timing. We (Guido) should look into this. A possible candidate is a Samtec Bulls-Eye which is just a land pattern on your PCB. But we should study this since a Bulls-Eye is the perfect phase plane reference but the phase plane will not be very accessible from the outside world (i.e. on the PCI bracket).
  2. Timing signals should be re-clocked with high speed FFs outside the FPGA. I remember that this was an issue that you CERN guys also proposed earlier.
  3. We need to think of some digital interface (including connectors) to close an external PLL loop using an external high performance oscillator. Ideas are still vague at this moment but the interface as such should already be on the list.

Accepted

Rejected

SFP

  • Double SFP for allowing redundant links?
  • Compact SFP (CSFP)?
    • Has two bidi standard SFPs in one SFP module
    • CSFPs have a little different pinout comparing to the standard SFPs. Same mechanics as standard SFP (Wikipedia).
    • Difference between SFP, bidi SFP and CSFP
    • CSFP Delta CSFP-33-A4K1DBT (pinout on page 4)
    • Check if use of CSFP would limit the use of standard SFP modules
    • Actually cannot be used to daisy chain modules: "The compact SFP (GLC-2BX-D) usually uses 1490nm to transmit signal and the 1310nm to receive signal".

FPGA family

Kintex 7 "Optimized for Best Price-Performance"

  • FBG676 package allows 70T, 160T, 325T and 410T sizes on the same footprint.

* XC7K70T-2FBG676C: (~180 Euro)

* XC7K160T-2FBG676C (~260 Euro)

** cf. XC6SLX45T-3FGG484C on SPEC: (~84 Euro without STEP pricing)

** 8 GTX Transceivers max (4 used for PCIe, 1 for SFP, 1 for FMC - as on SPEC - and 2 for SATA).

  • A 70T in FBG484 package cannot be used as has only 4 GTX Transceivers (No need to check if can be used on footprint of FBG676)

  • Smallest Kintex7 with 16 GTX transceivers: XC7K325T-2FBG900C (~1100 Euro)

Artix-7 "Transceiver Optimization at the Lowest Cost and Highest DSP Bandwidth"

  • Too slow

  • only if LPC

  • FGG676 package allows 75T and 100T on same footprint.

** XC7A75T-2FGG676C (~125 Euro)

** XC7A100T-2FGG676C (~180 Euro)

  • FBG484 package allows only 4 GTX transceivers (1 used for PCIe 1x(!), 1 for SFP, 1 for FMC as on SPEC and only 1 for SATA instead of 2).

Altera

  • Most WR efforts are done on Xilinx

FMC connector

  • Vadj programmable IC or just a few values and selection with jumpers?
    • Fixed as decides Vio of FPGA, so can be fixed anyway

SATA

  • Keep same two SATA connectors as on SPEC.
    • Planned to be used on SPEC for daisy-chaining triggers between SPECs with FMC-ADCs with modified gateware. However, no users that we know of now (WRXI may use it, and external users possibly?)
    • Is a connector specifically for internal use (i.e., not panel-mounted).
  • 8 GTX Transceivers max (4 used for PCIe, 1 for SFP, 1 for FMC - as on SPEC - and 2 for SATA).**** May be used as well for digital interface for external PLL loop.

Remarks

  • HPC: need new tests tools for production test
    • How test SFP as SATA not 1 Gbps anymore and replaced by other connector?
    • How is HPC tested on VFC-HD?

16 February 2018