Commit 1ebe9110 authored by Grzegorz Daniluk's avatar Grzegorz Daniluk

adding calibration document sources with figures

parent 1ebfc52c
all : wrcalib.pdf
.PHONY : all clean
wrcalib.pdf : wrcalib.tex
latex $^
latex $^
dvips wrcalib
ps2pdf -dPDFSETTINGS=/prepress -dSubsetFonts=true -dEmbedAllFonts=true -dMaxSubsetPct=100 -dCompatibilityLevel=1.4
clean :
rm -f *.eps *.pdf *.dat *.log *.out *.aux *.dvi *.ps *.toc
This diff is collapsed.
Before going further into the calibration procedure, some initial definitions
have to be clarified:
\item {\bf WR Device}: any device that supports White Rabbit
synchronization over an optical link. This can be e.g. the White Rabbit
Switch, the SPEC board, or any custom hardware made for a specific
application. A WR Device can run as WR Master or/and WR Slave.
\item {\bf WR Calibrator}: any White Rabbit Device selected from the
available set that is used as a reference to calibrate the
rest of the White Rabbit Devices used in a synchronization system.
\item {\bf WR Device TX latency} ($\Delta_{TX}$): WR PTP packet delay
from the moment it is timestamped in the transmission circuit
inside the FPGA (WR Endpoint module) to the moment when the
transmitted packet reaches a fiber link. It is the sum of:
transmission delay inside FPGA chip; propagation delay of
electronic components and PCB traces; SFP transmitter delay.
\item {\bf WR Device RX latency} ($\Delta_{RX}$): WR PTP packet delay
from the moment it reaches the receiving SFP module, to the
moment it is timestamped in the reception circuit inside the FPGA (WR
Endpoint module). It is the sum of: SFP receiver delay;
propagation delay of electronic components and PCB traces;
reception delay inside FPGA chip.
\item {\bf WR Device asymmetry} (2$\beta$): the difference between the TX
latency $\Delta_{TX}$ and the RX latency $\Delta_{RX}$.
\item {\bf Fiber round-trip latency} ($\delta$): the total round-trip
delay of a fiber link (without the Tx/Rx latencies of the WR Devices).
It is the sum of the one-way propagation delays: Master-to-Slave
($\delta_{MS}$) and Slave-to-Master ($\delta_{SM}$).
\item {\bf Relative delay coefficient} ($\alpha$): the parameter describing
the asymmetry (the difference between $\delta_{MS}$ and
$\delta_{SM}$) of the fiber used in a White Rabbit network. It
is the relative difference of Master-to-Slave and Slave-to-Master
propagation delays ($\alpha = \frac{\delta_{MS} - \delta_{SM}}
\item {\bf Round trip delay} ($delay_{MM}$): the total packet delay on
a White Rabbit link. It is the sum of communicating parties (Master and
Slave) fixed delays ($\Delta_{TXM}$, $\Delta_{RXM}$, $\Delta_{TXS}$,
$\Delta_{RXS}$), bitslide values ($\epsilon_M$,
$\epsilon_S$) and fiber round-trip latency ($\delta$).
\section{Measuring already deployed fiber}
Most of the methods described in section \ref{sec:calib_proc} require making a
measurement of 1-PPS signals produced by WR Devices. It's fairly easy to compare
them when you have a roll of fiber and the devices being calibrated lying on a
desk in the laboratory. However, sometimes the fiber infrastructure which
will be used in a WR network is already installed. WR Devices can still be
collected in a lab for calibration, but measuring the $\alpha$ parameter of a
buried fiber is not as straightforward as in section \ref{subsec:fiasym}. This
section proposes a way of measuring 1-PPS skew when the fiber cable is
already installed and you don't have both ends next to each other. The
measurements made this way can be used as a substitute for the simple
oscilloscope measurements in section \ref{sec:calib_proc}.
\subsection{Measurement with a loop-back fiber}
This method is based on a fiber delay calibration procedure used in the CERN
Neutrinos to Gran Sasso project \cite{cngs}. It requires:
\item an additional fiber which will be used to transmit the 1-PPS signal
produced by a distant WR Device to the place where it can be compared
locally with the other WR Device using an oscilloscope
\item two oscilloscope cables of the same type - having equal latency or a
known latency difference which can be taken into account and compensated
\item one pair of optical transmitter and receiver which will be used to
convert the 1-PPS electric signal into a light impulse sent through the
loop-back fiber. The transmitter and receiver must have constant
transmission/reception delays that don't vary on each power cycle.
\caption{Measuring 1-PPS offset using a loop-back fiber}
Measurement procedure:
\item When a link is established between the WR Master and the WR Slave,
connect the optical transmitter to the 1-PPS signal produced by the WR Slave
and the loop-back fiber. On the other side of the fiber connect the optical
receiver (fig.\ref{fig:loopback}, step 1).
\item Measure the 1-PPS skew ($skew_{PPS1}$) between the Slave (1-PPS
transmitted through the loop-back fiber) and the Master (1-PPS directly from
the node) using an oscilloscope.
\item Swap the optical transmitter and receiver so that you'll now transfer
the 1-PPS signal generated from the WR Master using the same loop-back fiber
to the WR Slave side (fig.\ref{fig:loopback}, step 2).
\item Do the same 1-PPS skew measurement between the Slave and Master, but
this time on the WR Slave side (the 1-PPS comes directly from the WR Slave
node and the one from the WR Master is transmitted through the loop-back
fiber). This way you will obtain the value $skew_{PPS2}$.
\item Calculate the actual skew between the WR Master and WR Slave using the
skew_{PPS} = \frac{1}{2} (skew_{PPS1} + skew_{PPS2})
The skew value calculated that way can be used in any equation from section
\ref{sec:calib_proc}. However, please remember that you will also need the
latency of the fiber ($\delta_1$ in figure \ref{fig:loopback}). That means, you
would have to start the calibration with two WR Devices on your desk, connected
with a short (few meters long) link (procedure \ref{subsec:refiber}) and later
use these two WR Devices with the already-installed link.
This diff is collapsed.
This document describes the calibration procedure for White Rabbit devices.
A White Rabbit link established between two devices is characterized by the
hardware delays and fiber propagation latencies presented in figure
\caption{White Rabbit link model}
Each WR Master and WR Slave has some constant transmission and reception delays
($\Delta_{TXM}$, $\Delta_{RXM}$, $\Delta_{TXS}$, $\Delta_{RXS}$). They are the
summed result of SFP transceiver, PCB trace and electronic component
delays as well as the delays inside the FPGA chip. Additional reception delay is
also caused on both sides by aligning the recovered clock signal to the inter-symbol
boundaries of the data stream. This is called the bitslide value and is marked in figure
\ref{fig:intro:link-model} as $\epsilon_M$ and $\epsilon_S$.
In addition to hardware delays, packets transmitted in fiber are affected with
propagation latencies in both directions ($\delta_{MS}$, $\delta_{SM}$). These
fiber propagation latencies are not equal since different light wavelengths are
used to communicate simultaneously in both directions using a single fiber.
Based on those parameters the round-trip delay ($delay_{MM}$) is defined as the
sum of all delay factors described above:
delay_{MM} = \Delta_{TXM} + \Delta_{RXS} + \epsilon_S + \Delta_{TXS} +
\Delta_{RXM} + \epsilon_M + \delta_{MS} + \delta_{SM}
However, since the bitslide values can be easily obtained from WR software,
the calibration procedure described in this document use a modified round-trip delay,
without $\epsilon_M$ and $\epsilon_S$:
delay_{MM}' = delay_{MM} - \epsilon_M - \epsilon_S
In the synchronization process, the PTP daemon tries to calculate the current
offset between Slave and Master clock. For that it needs a precise estimation of
the one-way link delay which, using the described link model, is the sum of four
delay_{MS} = \Delta_{TXM} + \delta_{MS} + \Delta_{RXS} + \epsilon_S
The calibration procedure described in this document is required since the
values of the hardware transmission and reception delays are not known
($\Delta_{TXM}$, $\Delta_{RXM}$, $\Delta_{TXS}$, $\Delta_{RXS}$). Moreover,
fiber cable and each device has unknown asymmetry that prevents from estimating
$delay_{MS}$ as half of the round-trip $delay_{MM}$.
This diff is collapsed.
import sys
import math
import matplotlib as mpl
import matplotlib.pyplot as plt
from matplotlib.ticker import ScalarFormatter
POINTS = 10000
INIT_LAT = 49970
STEP = 100
ALPHA = 2.57302194425528e-4
lat = []
for i in range(0, POINTS-1):
#calculate pps skew for latencies
skew = []
for i in range(0, POINTS-1):
#calculate pps factor
dalds = []
for i in range(0, POINTS-1):
dalds.append( (lat[i]/(0.5*lat[i]-skew[i])**2)**2 )
#draw graph of dalds
fig = plt.figure(0)
ax1 = fig.add_subplot(111)
ax1.plot(range(0, POINTS-1), dalds, marker='.', linestyle='None', c='b', scaley=True, scalex=False)
ax1.set_xlim([0, POINTS-1])
print "dalds[0]=" + str(dalds[0])
\section{Recovering the calibrator}
In a real life, when a WR network operates for a few years, the WR Device selected
to be a calibrator might become broken. The lack of a WR Calibrator makes it
impossible to connect new devices without recalibrating the whole WR network.
This section presents a method for obtaining the $\Delta_{TX}$, $\Delta_{RX}$
delays for a WR Device so that it can be used as the new WR Calibrator for an
already-deployed network.
\caption{Calibrating a new WR Calibrator for an already-existing WR network}
\item Initially please set the transmission and reception delays
($\Delta_{TXS}$, $\Delta_{RXS}$) of the new WR Calibrator to 0, but $\alpha$ has
to be set to the correct value for fiber $f_1$.
\item Connect your new WR Calibrator to one of the WR Devices originally
calibrated to the primary WR Calibrator
(fig.\ref{fig:recover_calibrator}). Use a short (few meters) fiber of known
latency and $\alpha$ coefficient or measure it first according to the
instructions in sections \ref{subsec:refiber} and \ref{subsec:fiasym}. Set
the WR Calibrator to Slave, and the WR Device to Master mode.
\item Run the monitoring software on the Slave node (\emph{wr\_mon} for the WR
Switch or \emph{gui} for the WR PTP Core). Write down the round-trip
delay and fixed delays of both Master and Slave. The transmission delay of
the Slave ($\Delta_{TXS}$) should be 0, but its reception delay
($\Delta_{RXS}$) will be equal to the RX bitslide value $\epsilon_S$. The
reception delay of the Master ($\Delta_{RXM}$) already includes the bitslide
value for this device.
\item Calculate the average (coarse) transmission and reception delay for the
new Calibrator using the values read from the monitoring software and the
latency of the fiber ($\delta_1$):
\Delta'_{TXS} = \Delta'_{RXS} = \frac{1}{2}\Delta_S = \frac{1}{2}(delay_{MM} - \Delta_{TXM} - \Delta_{RXM} - \Delta_{RXS} - \delta_1)
\item Write the $\Delta_{TXS}$ and $\Delta_{RXS}$ delays to the configuration
of the new WR Calibrator, restart the device, and let it synchronize again
with the new values.
\item Measure the Slave to Master offset by comparing 1-PPS signals with an
oscilloscope. The 1-PPS skew ($skew_{PPS}$) is used as
a correction value for the coarse transmission/reception delays:
\Delta_{TXS} = \frac{1}{2}\Delta_S - skew_{PPS}\\
\Delta_{RXS} = \frac{1}{2}\Delta_S + skew_{PPS}
\item Update the configuration of the new WR Calibrator to replace the coarse
delays with $\Delta_{TXS}$, $\Delta_{RXS}$.
\item The new WR Calibrator can be used to calibrate any other WR Device that is
supposed to be connected to the existing WR network.\\[12pt]
{\bf Note:} Please be aware that the measurement errors accumulate. It might
become an issue if the new calibrator uses values recovered with a WR Device
that was already calibrated to a recovered calibrator. The longer this chain
is, the more inaccuracy you should expect from the calibration procedure. To
minimize this effect, it's better to recover the values for the new WR
Calibrator using only devices that were calibrated to the primary WR
Calibrator for a given network.
author = {J.Serrano P.Alvarez J.Lewis D.Autiero},
title = {Inter-Laboratory Synchronization For The CNGS Project},
journal = {10th European Particle Accelerator Conference, Edinburgh, UK, 26 - 30 Jun 2006, pp.3092}
title = {Evaluation of measurement data - Guide to the expression of uncertainty in measurement (JCGM 100:2008)},
organization = {Working Group 1 of the Joint Committee for Guides in Metrology(JCGM/WG 1)},
month = {September},
year = {2008}
\section{Equipment requirements}
The calibration procedure described in this document makes no assumption on what
kind of WR device will be used as a WR Calibrator or what kind of fiber (length,
producer) is placed in between WR Devices. However, there are a few general
requirements listed below. The first two of them are essential for the
Calibrator pre-calibration steps described in section
WR Calibration needs:
\item two fiber cables having different lengths, one of them is a few
kilometers long (to make $\alpha$ coefficient measurement precise), while
the other is a few meters long.
\item two pieces of a WR Device selected to be the Calibrators: this means two
exactly the same WR Devices, having the same PCB design, the same bitstream
downloaded into the FPGA and using complementary SFP transceivers (the same
producer, matching Tx/Rx WDM wavelengths\footnote{Check the list of
supported SFP transceivers});
\item all WR Devices (i.e. the Calibrator and devices under calibration)
capable of producing 1-PPS output from their local clock
and providing it through an on-board connector or test point;
\item two oscilloscope cables of the same delay (i.e. the same type and
length) or different but known delays so that the correction
could be made for all oscilloscope measurements described in
the document;
\item a device of any producer that is able to measure time
spans of less than 1ns for measuring 1-PPS skew. This can be an oscilloscope or timer/counter device e.g. Pendulum CNT-91.
J.Serrano P.Alvarez~J.Lewis D.Autiero.
\newblock Inter-laboratory synchronization for the cngs project.
\newblock {\em 10th European Particle Accelerator Conference, Edinburgh, UK, 26
- 30 Jun 2006, pp.3092}.
Working Group 1 of the Joint Committee for Guides in Metrology(JCGM/WG 1).
\newblock {\em Evaluation of measurement data - Guide to the expression of
uncertainty in measurement (JCGM 100:2008)}, September 2008.
This is BibTeX, Version 0.99c (TeX Live 2009/Debian)
The top-level auxiliary file: wrcalib.aux
The style file: unsrt.bst
Database file #1: references.bib
Warning--empty year in cngs
You've used 2 entries,
1791 wiz_defined-function locations,
453 strings with 3893 characters,
and the built_in function-call counts, 241 in all, are:
= -- 18
> -- 5
< -- 0
+ -- 3
- -- 1
* -- 12
:= -- 43
add.period$ -- 5
call.type$ -- 2$ -- 1$ -- 0
cite$ -- 3
duplicate$ -- 14
empty$ -- 28$ -- 1
if$ -- 50$ -- 0$ -- 2
missing$ -- 1
newline$ -- 12
num.names$ -- 1
pop$ -- 7
preamble$ -- 1
purify$ -- 0
quote$ -- 0
skip$ -- 5
stack$ -- 0
substring$ -- 0
swap$ -- 2
text.length$ -- 0
text.prefix$ -- 0
top$ -- 0
type$ -- 0
warning$ -- 1
while$ -- 1
width$ -- 3
write$ -- 19
(There was 1 warning)
\documentclass[a4paper, 12pt]{article}
% creating subsubsubsection notation
% src:
{-3.25ex\@plus -1ex \@minus -.2ex}%
{0.0001pt \@plus .2ex}%
{-3.25ex\@plus -1ex \@minus -.2ex}%
{0.0001pt \@plus .2ex}%
\title{White Rabbit calibration procedure\\[0.5cm]
\large {version 1.0}}
\author{Grzegorz Daniluk\\ CERN BE-CO-HT}
\section{Calibration procedures}
Markdown is supported
0% or
You are about to add 0 people to the discussion. Proceed with caution.
Finish editing this message first!
Please register or to comment