fmc-adc-100m14b4cha
Adding WR support toOverview
CERN currently provides the White Rabbit PTP Core (WRPC), an Ethernet MAC implementation capable of providing precise timing. WRPC simplifies the process of adding WR support in an FPGA design, by encapsulating all WR time-specific functionality and providing a standard MAC interface to the FPGA fabric for sending and receiving Ethernet frames.
The WRPC also includes a timing interface which provides current time to the FPGA fabric, in a form that can be easily used. It consists of a 1-PPS signal and a UTC timecode aligned to the time of WR Master.
Furthermore, the WRPC can "discipline" other clock signals via a loop-back mechanism. The FPGA provides the clock signals to the WRPC, and the WRPC generates a set of signals to control an external DAC, which in turns controls the clock source (eg. a VCXO).
In fmc-adc-100m14b4cha,, there are two clock signals that can be disciplined by WR. One is the FPGA system clock, which originates from the carrier board. The other is the ADC sampling clock, from an on-board crystal oscillator from Silicon Labs (Si570, datasheet).
The FPGA system clock is running at 125MHz. When aligned to WR, it will enable the use of the WRPC timing interface for timestamping of the acquisition.
The ADC sampling clock can be programmed via the I2C interface of the Si570 crystal oscillator to provide frequencies in the range between 10MHz and 945MHz (and select frequencies up to 1.4GHz). In practice however, this is limited by the fact that (a) the ADC chip (LTC2174, datasheet) has a maximum sampling frequency of 105MHz and (b) the deserialiser readout block in the FPGA expects a 100MHz source-synchronous sampling clock, a value that cannot be changed on the fly, without FPGA reconfiguration.
For the remainder of this discussion, the ADC sampling clock will be considered as being always fixed to 100MHz. When aligned to WR, it will allow to synchronise multiple FMC-ADC cards with a sub-nanosecond accuracy.
Implementation
The implementation will integrate the White Rabbit PTP Core (WRPC) to the existing FMC-ADC gateware design, for both the SPEC and the SVEC carriers.
The WRPC will be included as a git submodule, following the connection diagram in the White Rabbit Node Reference Design guide for upgrading an existing system.
FPGA system clock alignment
Currently, the 125MHz FPGA system clock is derived from the 20MHz VCXO of the carrier, through an FPGA PLL. The 333MHz DDR clock is also derived from the same source, via the same PLL. All remaining clocking resources on the carrier are unused.
In order to control the FPGA system clock, the first step will be to use one of the 125MHz differential clock outputs of the CDCM61004 clock generator (OUT3) as the clock source. The same clock signal will also be used as clock reference and system clock for the WRPC. However, because it is very difficult to synthesize the WRPC with a 125MHz system clock, an FPGA PLL will be used to divide it by two (62.5MHz) before feeding it into the WRPC system clock input. The same PLL will be used to generate the 333MHz DDR clock as well.
This in turn will free the 20MHz VCXO clock, to be used for the DMTD clock input of the WRPC, after it goes through another FPGA PLL which will scale it to 62.5MHz.
As suggested in the reference design, the 125MHz GTP clock will also be provided by the CDCM61004 (OUT1).
Using the above scheme, the FPGA system clock will be disciplined by the WRPC. Since this clock is used by the time-tag core, the time tags themselves will be referenced to White Rabbit time.
The only caveat is that since the WRPC clock will be running at 62.5MHz, all the wishbone interfaces of the WRPC (1x slave interface for control of the core, 2x pipelined interfaces for sending and receiving Ethernet frames) will be using the same clock as well. This means that for interfacing with the existing Wishbone bus in the FPGA we need cross-clocking adapters (avaible from general-cores).).
ADC sampling clock alignment
This option has already been studied and presented in here. Those notes will be updated and integrated into this page.
Roadmap
Global roadmap
- Trigger on timestamp (without WR)
- Port to SVEC
- Trigger on timestamp (with WR)
- Disable/Enable WR
- Trigger message in/out over WR (similar to external trigger in/out)
- Synchronise sampling clock to WR
Detailed roadmap
- trim unused clocks, switch to 125MHz clock input and sanitize reset [closes #1316, #1239, #1240]
- reduce glitch filter or replace with schmitt trigger [closes #1238]
- Add "trigger on time" functionality [closes #1311]
- Attach WR PTP core to FMC-ADC on SPEC and discipline FPGA system clock [related to #1289]
- Port to FMC-ADC on SVEC [related to #1289]
- Add option to enable WR [related to #1289]
- Add option to get time from WR instead of time-tag core [version:"Release 5.0"] [#1312]
- Stream to DDR in multi-shot mode #1288 (or figure out last DPRAM sample #1293)
- trigger sample in post samples #1292
- Expose more info in the status/irq registers [version:"Release 6.0"] #1294
- Add "FPGA fabric" trigger input
- Implement basic WR message "trigger on time" and connect to FPGA fabric trigger input [version:"Release 7.0"]
- Discipline ADC sampling clock [version:"Release 7.1"]
- Solve remaining issues [version:"Release 8.0"] [closes #1296, #1295, #1291, #940, ...]
Dimitris Lampridis - 7 June 2016