White Rabbit Newsletter, May 2017
CERN
Tsinghua University
We have adapted WR technology for a pre-clinical Positron Emission tomographic Instrument. The PET ring consists of 8 detector modules, each containing 6 LYSO arrays coupled to a 8*8 SiPM matrix. The signals from the SiPM are processed by a dedicated ASIC and then digitized and time-tagged with a commercial ADC and FPGA-based TDC respectively. Each module acts as a White Rabbit slave node to synchronize the time reference of the TDC. The extracted list mode data is also transmitted via the WR link. A standard WR switch collects the data from all 8 detector modules, but the firmware has been modified. The list mode data packages from each module are extracted while other packages enter the switching matrix as normal. The list mode data are then processed by dedicated timestamp coincidence logic. The outcomes are then re-packeted and send out via a separate port to a reconstruction PC. The scheme greatly simplifies the PET design by replacing traditional off ring electronics with a standard White Rabbit switch.
The design of dual-port Cute-WR mezzinine has been released. It can act as a slave with a redundant connection to improve the reliability or in daisy chain mode to support a cascade topology. We have verified the WR functionality on each port for both master/slave. The daisy chain for PTP transmission has been verified, and the routing of data frames is still under development. With the dual port mezzanine, we are also designing a portable calibration node which can play an important role in a WR auto-calibration procedure, WR array deployment and validating the synchronization accuracy, bringing the advantage of higher integration, more convenience and lower human workload (http://ieeexplore.ieee.org/document/7820218/).
A prototype array of 100 detector nodes are running in Tibet. For quite a long period, we suffered some instabilities, randomly losing nodes during power up and normal operation. The new released WRPC 4.0 has greatly improved the situation. The LHAASO project has been finally approved and more than 500 WRS and 7000 WR nodes will be deployed at an altitude of 4300m asl. At the end of 2018,a quarter of the full array is foreseen to operate.
Nikhef and Vrije Universiteit Amsterdam
Xilinx Family-7 GTH (Nikhef)
VHDL sources are available for a deterministic PHY for the Xilinx Family-7 GTH (used by some Virtex-7 devices).
Rabbit_FX: White Rabbit Oscillators on an FMC mezzanine (Nikhef)
A White Rabbit FMC module that includes the DACs and Oscillators needed to run White Rabbit has been built (https://redmine.nikhef.nl/et/project/rabbit_fx/wiki). It enables code development on a vendor FPGA development board before new hardware is available.
White Rabbit Switch mini backplane for embedded applications. (Nikhef)
In embedded detector electronics there is usually not much space available. Work is in progress to create a small WR Switch mini backplane with two 2x5 SFP cages.
Calibration (Nikhef, Vrije Universiteit Amsterdam):
Calibration of WR gear is addressed by the following three main items:
1) Absolute delay calibration
It is proven that hardware delay calibration parameters can be derived
(https://www.ohwr.org/project/wr-calibration/wiki). These parameters
define the timing relationship between measurable external phase planes
(i.e. the PPS connector and SFP electrical connector). Additional "mode
abscal" still needs to be added to PPSi.
2) Electrical/Optical and Optical/Electrical calibration
Although still in a laboratory state, it has been proven that hardware
delay calibration parameters of electrical/optical and
optical/electrical converters (i.e. SFPs in most cases) can be measured
with pico-second precision. Storing the delay parameters in SFP EEPROM
will enable exchange of SFPs without the need for re-calibration. A
definition for storing calibration parameters can be found here:
https://www.ohwr.org/project/sfp-plus-i2c/wikis/User_EEPROM.
3) In-situ asymmetry coefficient determination
Work is in progress to determine the fiber asymmetry coefficient in-situ
using SFPs with a tunable laser.
Tools for calibration have been developed.
a) It is planned to produce a batch of "SFP+ Loop Back Modules" (https://www.ohwr.org/project/white-rabbit/uploads/f6d07a9f48cfbaf3846f0b9b985571e4/SFP__timing_calibration_module.pdf) used for absolute calibration. Currently the mechanical housing causes some issues that prevent us from production. When these issues are solved an announcement will follow via the white-rabbit-dev mailing list such that people interested can sign up for a module.
b) A 10 Gbps capable Multi SFP crate is designed including "SaFariPark" software (https://www.ohwr.org/project/sfp-plus-i2c/wiki) which enables us to exercise SFP modules, both with respect to their serial link, as well as the I2C interface. The latter is needed for accessing digital diagnostics, storing calibration parameters and enabling SFP tuning. Moreover "SaFariPark" computes SFP checksums and can correct SFPs with corrupted EEPROM content (unfortunately something we experienced too often).
c) A proposal for a generic software library to access SFPs via the I2C
bus is launched (https://www.ohwr.org/project/libsfp/wiki). When this
software is deployed in WR gear then calibration
parameters stored in EEPROM are accessible, as well as reading out SFP
Digital Diagnostics.