What follows is a minimally edited set of contributions from different
institutes and companies.
CERN (including Alessandro)
V4 of the switch includes gateware with VLAN support, increased
throughput and lower latency than V3. The new gateware was tested with a
specialized IT traffic generator and analyzer. The tests show latencies
well below 10us.
Support for remote logging, review and merge of http work by University
of Granada, some fixes to initialization, support for SNMP using the
standard MIB files, support for a new barebox and ubifs, new improved
installation procedure, with support for bigger firmware files, minor
changes to build under latest Ubuntu.
The IEEE1588 High Accuracy committee works on standardizing WR within
the IEEE 1588 standard. The last PTP standardization meeting was
organized at CERN in April 2014. Likely, different aspects of WR will be
formed into optional features provided by the standard. The work of the
committee is now focusing on the first feature: L1 Syntonization. It
started by studying the way WR uses L1 syntonization and phase detection
to enhance the timestamping precision. A number of potentially useful
use cases were identified. Currently, the methods are being generalized.
The goal is to make them applicable in a standard way to different
media, syntonization methods and phase detection techniques.
We released ppsi-v2014.07. It includes a PTP simulator that allowed us
to fine-tune the servo algorithm for non-WR slaves. It also includes a
configurable frame loss rate as a fault-injection system. This helped us
reproduce and debug a relatively rare problem occurring to one of our
users. PPSi 2014.07 is also the subject matter of a paper submitted (and
accepted) for ISPCS 2014.
Node calibration procedure
After publishing two drafts for comments, we're now happy to announce
that v1.0 of the "White Rabbit calibration procedure" document is
available on OHWR. You can download it from the Documents section of the
main White Rabbit project page .
The document provides step-by-step instructions to be performed to
ensure your WR network is properly calibrated and synchronized with
sub-ns accuracy. It also contains additional hints on how to measure the
fiber link which is already deployed and how to recover a WR Calibrator
parameters if your primary device got broken. Two appendices provide the
mathematical proofs for the calibration correctness and the estimation
of measurement errors.
This is just an update to last years newsletter . The GSI Timing
Team is working on the General Machine Timing (GMT) system based on
White Rabbit for the upcoming FAIR facility . The next milestone
will achieve "readiness for test operation" with the CRYRING@ESR
project. CRYRING is a contribution by Sweden and presently being set up
behind the Experimental Storage Ring (ESR). Besides it serves as a test
case for FAIR relevant developments.
Timing Node Hardware
All hardware is based on Arria II GX or Arria V GX FPGAs. Production
readiness was achieved for the form factors PCIe , VME  and
SCU . The PEXARIA5 PCIe module  serves as a reference design
for future developments of timing nodes for FAIR.
Timing Node Gateware
During the last year we complemented existing VHDL cores including newer
version of the Timestamp Latch Unit (TLU), an Etherbone master, Message
Signaled Interrupts (MSI) and a Wishbone MSI Crossbar. Another important
achievement was the unification of form factor specific top designs by
providing the so-called "monster" design. In cooperation with other
teams of the control system group we finally achieved agreement on the
data format, which will be provided to the other layers of the control
system when time critical actions of the accelerator are triggered by
the timing system. Subsequently, the first production version of the
Event-Condition-Action (ECA) unit has been implemented. On Arria V GX
based platform we have improved the granularity for triggering actions
(ECA unit) and latching timestamps (TLU) from 8 ns to 1 ns. This does
not required external circuitry such as delay chips but is achieved by
using differential IOs of the FPGA clocked at 1 GHz and is entirely done
The data master schedules events by broadcasting relevant timing
messages across the timing network (remark: The ECA unit in a timing
node filters incoming messages. Relevant messages are decoded and
preconfigured actions on the timing node are executed on time). The data
master is implemented using a cluster of lm32 softcores instantiated on
a PCIe board . The softcores are synchronised using either MSI or
shared memory. Schedules are prepared beforehand and presently described
using a XML based syntax.
In June 2014 we released gateware, firmware for the hardware platforms
[3-6] under the codename "Asterisk" . Please note, that this
release includes many cool tools such as "eb-console", which allows
accessing the "White Rabbit GUI" remotely. The gateware of all form
factor also includes a flash controller that allows to flash devices via
any Etherbone supported protocol such a PCIe, VME, USB and UDP.
Integration with higher layers of the control system
Integration of timing nodes into the core of the Front-End Software
Architecture  has so far only been implemented as a prototype.
Nevertheless, it allows FESA classes to receive events from the timing
system by subscribing to a so-called timing event source in FESA core. A
next step of prime importance is to link the data master to the Settings
Management system of the accelerator control system.
So far, the timing system works reliably including the data master, a
timing network of 11 active White Rabbit Switches and about 20-30 active
timing nodes. Already today, the timing system serves for time stamp
distribution for Data AcQuisition (DAQ) systems at GSI. A first beam
time involving a DAQ system with synchronisation via White Rabbit PTP is
scheduled for September 2014. Timing nodes in form factors PCIe 
and VME  will be used.
This method is widely used in the trapped-ion community - where people
work on either optical clocks or quantum computing. When ions are
trapped in RF-fields using Paul-traps they undergo secular (usually ca 1
MHz) and driven (usually 10MHz) motion in the trap. This motion is
undesirable as it shifts the 'clock-transition' frequency due to the
doppler effect. The secular motion can be minimized by laser cooling the
ion, while the driven motion is minimized by optimizing the trap design
and applying additional compensation DC electric-fields. The motion can
be detected as a correlation signal in the fluorescence signal from the
ion. Using a modulated LED as the light-source, this demonstration shows
photon-counting with a PMT, time-stamping with the Fine-Delay, and
real-time computation of an arrival-time histogram on the PC. We hope to
apply this technique to our ion clocks in the future.
University of Granada
UGR is contributing to the White-Rabbit community with different R&D
projects in relation to the WR switch, WR nodes, porting WR components
to other development platforms as well as stand-alone versions of
current WR nodes. In addition to this, UGR has been also involved in the
development of the Web Management Interface for the WR switch.
At the same time, UGR is working in conjunction with Seven Solutions in
the development of a reliable WR node with redundancy capabilities to
avoid single point of failure issues.
All this work is being carried out in the framework of several national
and international projects, such as ACELTEC and the Square Kilometre
Array (SKA). UGR contributes to both projects in synchronization and
timing work-packages promoting WR as the main time&frequency
In addition, UGR is putting its efforts in Smart Grid technologies and
the applicability of WR to this field.
Stand-alone FMC Delay Card
The Stand-alone FMC Delay Card is a fully operational stand-alone FMC
Delay card based on a WR node that can be initialized remotely. It also
performs periodic calibrations without requiring being plugged in a PC,
leading to a reduction of final system costs, size and power
WR Port to 10 Gigabit Ethernet
UGR is working on the development of a new hardware architecture to
perform an analysis about the feasibility of porting WR from 1 Gigabit
Ethernet to 10Gps. We are designing and testing different FPGA
architectures looking for the best solution for this purpose.
At the same time, UGR has modified a Xilinx development platform VC7 by
adding external White-Rabbit oscillators to provide this platform with
this feature. In addition, we have modified the Phase-locked Loop (PLL)
to disseminate a frequency reference of 156,25 MHz on the platform.
This project includes a WR PTP Core port from an Altera Spartan-6 FPGA
to a Xilinx Virtex-7 one to take profit of the advantages of the new
Virtex-7 FPGA series.
WR Switch Web Management Interface
UGR has developed a friendly Web Management Interface for the WR switch
to facilitate its configuration and monitoring for non-WR-expert users.
This interface has been recently upgraded to the 4.0 switch release
adding features such as VLAN configuration, network setup, PPSi clock
customization, NTP Server, user login and remote flashing. Now, it is
also possible to load and backup the configuration files for all
services in addition to other improvements on the interface and previous
WR customization for KM3Net
KM3Net is a cubic-kilometer scale neutrino telescope at the bottom of
the Mediterranean Sea. KM3net is a highly complex network composed by
underwater sensors hosted at Digital Optical Modules (DOMs), the
onshore station and the optical connections between/inside them. Every
single DOM has a unidirectional 1Gb/s uplink to reach the onshore
station (through this link delivers the acquired data). On the other
side, the onshore station has a unique unidirectional 1 Gb/s downlink
to reach all the DOMs, this is called the broadcast or slow control link
(SC). The SC is shared by every single DOM, so that when a DOM receives
a packet, all the other DOMs also receive it. This topology highly
reduces the communication resources cost at this facility, but it
requires customizing the communication elements as the switches and
A deep customization of WR switch has been done to develop it from being
a synchronization point-to-point technology which uses bidirectional
links between two devices to be able to synchronize 360 DOMs by
asymmetric and unidirectional links. This customization allows sharing
the unidirectional downlink between 360 DOMs, highly optimizing the
WR Zynq Embedded Node (7S-UGR)
UGR and 7S are working together in the development of a new WR node in
which the main SoC is a Xilinx Zynq programmable chip. This chip is
composed of two Cortex ARM9 professors and a FPGA making this a
promising platform for Smart Grid applications. The FPGA is planned to
implement a new version of the WR PTP Core (7S) and the two ARMs will
run the operating system and the WR software (UGR).
This platform has been focused to ease the development of standalone
nodes by including a high number of communication ports and the
possibility to expand new features using the FMC connectors.
Additionally this platform has been focused for industrial markets and
will provide various redundancy features such as an internal backup
oscillators (MEMS) as holdover mechanism or the High-availability
Seamless Redundancy (HSR) protocol to avoid single points of failure. It
will also support standard timing distribution protocols (i.e, IRIGB,
NMEA, ToD, SNTP) required in Smart Grid or Telecommunication markets.
The PTS Core
Seven Solutions has been improving the Production Test Suite repository
to ease the creation of tests for its own products. This repository has
been designed as a submodule to include in the test repository of each
product. It contains documented classes to support common peripherals,
internal buses (e,g, i2c, spi, 1wire) and various drivers (wishbone
master). It also provides to the user a simple API to test devices
through SSH, to test devices with various boards and to log this
information in a clean way.
WB in EPICS
EPICS is a set of Open Source software tools, libraries and applications
developed collaboratively to create distributed soft real-time control
systems for large scientific experiments and it is commonly used by
particle accelerators and telescopes in the USA. Seven Solutions has
created an IOC driver to ease the integration of wishbone bus devices
into the EPICS platform by using a standard driver to access the
wishbone bus such as PCIe or VME. This solution also uses a modification
of the wishbone slave generator to create automatically the proper PV
(Process value) that needs to be integrated in the EPICS platform.
LLRF using WR
Seven Solutions has been working in collaboration with CIEMAT on the
creation and support of a fully Digital Low-Level Radio Frequency
platform to control several RF cavities such as RFQ or RF-Linac built
for a high-intensity CW deuteron accelerator. In this project we have
developed a CPCI-Serial carrier with two FMC HPC plugs to a loop-back
acquisition board (12xADC/2xDAC) and a digital IO board for interlocks
signals. The whole system is controlled through the EPICS platform and
its CSS/BOY interface. Finally WR is used to synchronize all the LLRF
along the accelerator.
LHAASO prototype array setup in Tibet
The LHAASO experiment has been finally located at Yading, Daocheng,
China, near the world's highest airport
A prototype array for LHAASO KM2A has been installed in YBJ, Tibet
(http://en.wikipedia.org/wikis/Yangbajain) to test the function &
stability of the white-rabbit network and the customized detector and
electronics of LHAASO.The prototype array includes 4 WR switches in a
chain topology, 50 electronic detectors and 2 muon detectors. The
Grandmaster switch gets the UTC time from a GPS receiver using NTP
service, the PPS signal from the same GPS receiver keeps the timestamps
updated. A 500ps synchronization among the units has been achieved after
calibration, more detailed tests are still ongoing.
Temperature Effect and Correction Method of White Rabbit Timing Link
The temperature effect and contributions from different components are
separately studied and analyzed. An online real-time temperature
correction method was applied based on the result which significantly
reduce the synchronization variation from 300 ps to 50 ps in a
temperature range of 50 degrees centigrade.
The related results can be found at: http://arxiv.org/abs/1406.4223
Dual-Port WR development
We are developing a WR node with two SFP ports. This node keeps most
compatibility with Cute-WR (using the same FPGA and other peripheral
circuits but just an additional SFP connector). The two ports can be
connected to different WRS and work simultaneously to improve the
reliability of the WR node in case of link failure. We are also
exploring the possibility to make the node work in chain-mode to support
a daisy connection topology.
Consulting for potential WR applications in China
JUNO (Jiangmen Underground Neutrino Observatory) known as the
following of Dayabay reactor neutrino experiment which had
successfully measured theta13 in 2012. The experiment is scheduled
for data taking in 2020. The WR network will be applied to provide
synchronized clock and timestamp to hundreds of under-water
electronics boxes with a high requirement for reliability.
The beam protection system for an ADS (Accelerator Driven
Sub-critical System) prototype. The scale for the prototype will
consist of only tens of WR-Nodes. The precision of WR could be
over-qualified for this but they raise the requirement of high
Timing infrastructure for the Jingping under-ground laboratory,
which is now the deepest under-ground lab in the world with 2800m
rock overhead. 8 new experiment halls are now under construction. WR
will be deployed as standard timing interface for all potential
Timing synchronization for a modularized human TOF-PET instrument.
The instrument will be divided into 18 separate DAQ sectors that
will need to be synchronized with few hundred pico-second level for
the TOF coincidence scheme.