Commit e329aaef authored by Tjeerd Pinkert's avatar Tjeerd Pinkert

specifications/SW/rfc: rewrote rfc, it is now much more consistent.

I have tried to add full consistency to the initial draft posted on the mailing
list. This is then a version which should be checked by others.

I would propose that sample programs for log parsing based on this format are
written to see how we can handle this type of log format in software. Since I'm
probably the one to start on this work or finish it later on, I propose Python
as the preferred language for such a program.
parent d16b8b5e
RFC: An internally consistent WR log format usable for all WR hardware.
- -----------------------------------------------------------------------
I think it would be nice to try to make the output of 'stat cont' on
SPEC, WR-LEN, Switch (wr_mon), WR-ZEN consistent for machine parsing.
Right now I see that the LEN/ZEN have yet another pretty print
implemented, that my current scripts can't read (and really it is
annoying to fix the scripts for each new format that is just not
compattible...)
I would suggest that each status update contains only one line of data
(not 4 as on the LEN/ZEN), or that a structured data format (e.g. by
use of {}) is implemented. On the LEN/ZEN there is currently a line of
data for the mode, each of the wr ports and the temperature...
Currently a format with field:value separated by spaces seems the most
imlemented. I could imagine that this can be extended to:
field:{field:value field:value} field:value
where the highest level of hierarchy is not embraced. I can imagine
that at some point we would come to comparison of various slave ports
on a switch where for each slave port an active servo state is kept.
Such a configuration, combined with hardware support for a latch of
all data to a static register, could e.g. be used to support real time
comparison of various clock domains. Currently the registers appear to
have no support for hard real time capture of the register values
(maybe I'm wrong there).
At this point I see in the switch four domains that could be called
"top level" in an extended format.
TIME: hardware timestamp (currently as close to as possible) of the
capture time of the data
SERVO: state of the servo that keeps track of the time, references which
port is tracked.
PORTS: status of each port of a device
TEMP: temperature of the temperature sensors on a device.
In the current and proposed update of wr_mon the words TIME PORTS
SERVO and TEMP are fields without value, showing this by the absence of :
It is a question whether we need this in a more structured format. In
my own parser such words are simply filtered (including the C in the
temperature field of the SPEC, which I would suggest to drop, or to
have a three semicolon separated field:value:unit form allowed too).
RFC: A machine readable WR log format usable for all WR hardware.
-------------------------------------------------------------------------------
TODO list:
- Decide if all values of a subset are obligatory
- If all values are made obligatory an empty type should be defined for hardware
where not all values are present.
- Decide if a binary blob format for _pushing_ of data in the native format to
databases should be defined here or elsewhere.
Introduction:
-------------
A consistent, machine readable log format (stat cont, wr_mon -a) for all White
Rabbit (WR) hardware would facilitate the use of a single software component to
process output values, e.g. to filter out specific field values for processing
with further softwares.
Scope:
------
It is the intention that this log format can be implemented on all White Rabbit
hardware supporting the printing of internal parameters on a (serial line based)
terminal. Because WR hardware has processors ranging from small hard real time
processors realised in an FPGA fabric to larger embedded processors, the format
must be suitable for use on machines with a small memmory footprint.
The format must be suitable for machine parsing, and be internally consistent.
Proposed format:
----------------
The format prints a set of parameter entries, read out at a single point in
time, together on a single output line. The parameter entries are separated by a
single space character. The parameter entries consist of two fields, <name> and
<value> separated by a semicolon (:). Subsets of parameter entries can belong to
a <name>. Such subsets are embraced in curly braces and the subset is placed in
the <value> field of <name>. No whitespaces are allowed in the fields of a
parameter entry.
The <name> field contains the name of the parameter entry. Names are specified
in this document. The TYPE of <value> for <name> is specified together with the
<name>. TYPES are specified in a separate section of the document. If a name is
given in a certain UNIT and UNIT is not specified explicitly, then UNIT is the
current SI base unit for this type of quantity, e.g. if <name> specifies a time
quantity, the unit is in second unless specified otherwise. It is recommended
that the UNIT is specified for all <names> except when a value is unitless.
The <value> field contains the value of the parameter entry, or a subset of
parameter entries embraced in curly braces (TYPE = subset).
Names can be written out in full, using underscores (_) as whitespaces, or be
abreviated. Both the full and abreviated forms must be specified in this
document. Either the full or the abreviated form is consistently used in the
log output of a WR device. Names can not have duplicates in this definition
document. In the log output doublings are only allowed when presented as part of
subsets.
The TYPE definitions:
---------------------
In this section all TYPE definitions that are to be used in the description of
the <name> fields are defined.
For numeric values three types are defined, integer (to be understood as an
arbitrary precision integer type), decimal (to be understood as arbitrary
precision decimal number type) and scientific (to be understood as arbitrary
precision scientific notation type).
Integer type [int]: The integer type is an arbitrary length string of numerical
characters (1234567890) representing an integer. Leading zeros are allowed,
but not recommended. Examples are: 1, 1286346234901235, 001000
Decimal type [dec]: The decimal type is an arbitrary length string of numerical
symbols including exactly one point (.) character denoting the separation
between the integer part and fractional part. The integer and fractional
parts are made up of integers with at least one digit. Leading zeros are
allowed in the integer part, but not recommended. Trailing zeros are allowed
in the fractional part, but not recommended. Examples are: 1.0, 0.10234,
10234.0, 2134.2134, 0045235.123000
Scientific type [sci]: The scientific type is an arbitrary length string of
numerical symbols including exactly one letter e. The part before the e is
the mantissa and can be an integer or decimal value. The part behind the e
is the exponent. It is composed of one positive or negative symbol (+-) and
an integer part denoting the power of 10. Examples are: 1.0e+0, 123.123e-9,
00123.123000e+009
Other TYPES defined are the boolean type, the string type and the string+integer
type. The boolean type is used when a value can be true or false and the string
type is used when textual information of some kind must be represented. The
only use of the string+integer type
String type [str]: The string type is an arbitrary length string of letter
symbols excluding the whitespace symbol (which is used as field separator),
single or double quotes and numerical symbols. Letter symbols are limited
to: abcdefghijklmnopqrstuvwxyzABCDEFGHIJKLMNOPQRSTUVWXYZ, and the following
punctuation characters can be used: .,:;<>/?(){}[]|\~!@#$%^&*_-+=
A whitespace is most conveniently represented as the underscore (_) symbol.
The use of punctuation other than the underscore (_) is not recommended.
Examples (within the quotes) are: "string", "this_is_a_string", "ASTring",
"ASTRING_wi,th_(BracketS!)".
Boolean type [bool]: The boolean type is a string with two restricted values for
true (T, True) and two restricted values for false (F, False).
String+integer type [str+int]: This type is the concatenation of a string and an
integer. This type is used only in <names> of agregated types where multiple
instances of the agregated type are present. It is also used as a value when
relations between <names> are made. The concatenation is not denoted by a
symbol. Examples are: string123, a_string0001
The UNIT definitions:
---------------------
In this section all UNIT definitions that are to be used in the description of
the <name> field are defined.
The unit definition can be preceeded by the following scaling factors:
Scaling factors:
----------------
full (abreviated): scaling factor
---------------------------------
yocto (y): 10^-24
zepto (z): 10^-21
atto (a): 10^-18
femto (f): 10^-15
pico (p): 10^-12
nano (n): 10^-9
micro (u): 10^-6
milli (m): 10^-3
--- (-): 10^0 (not to be used)
kilo (k): 10^3
mega (M): 10^6
giga (G): 10^9
tera (T): 10^12
peta (P): 10^15
exa (E): 10^18
zetta (Z): 10^21
yotta (Y): 10^24
The following UNITS are in use in the White Rabbit equipment:
Units:
------
full (abreviated)
-----------------
Celcius (C): Used to represent temperature in degrees Celcius
Kelvin (K): Used to represent temperature in Kelvin
second (s): Used to represent time
The <name> field definitions:
-----------------------------
The names of the various parameters that are occuring in the current log outputs
are defined here. The definition includes the TYPE and UNIT and which parameter
of the hardware is represented exactly. Unitless parameters have only their TYPE
defined.
TODO: should a PTP agregate be constructed?
Names:
-------------------
full (abreviated): [TYPE, UNIT] definition
------------------------------------------
time_precision_time_protocol+int (tptp+int): [str+int] subset containing
precision time protocol (PTP) time.
time_white_rabbit+int (twr+int): [str+int] subset containing white rabbit
(WR) time.
time_atomic_time+int (ttai+int): [str+int] subset containing
international atomic time (TAI) time.
time_universal_time+int (tutc+int): [str+int] subset containint universal
coordinated time (UTC) time.
second (s): [int, second] value of the second counter of the WR PTP core
nanosecond (ns): [dec, nanosecond] value of the nanosecond counter of the WR
PTP core, supplemented by the phase of the DDMTD recalculated to ns with
as many digits as are appropriate (this can go beyond the ps limit)
port_white_rabbit+int (pwr+int): [str+int] subset containing white rabbit
port information. Ports are numbered from 1.
port_ethernet+int (peth+int): [str+int] subset containing ethernet port
information. Ports are numbered from 1.
link_up (lu): [bool] the physical layer link of the network port is
connected and working (up).
packets_received (rx): [int] number of PTP packets received on this port
packets_transmitted (tx): [int] number of PTP packets transmitted on this
port.
white_rabbit_mode (wm): [str] this port is a WR master or slave port. Note:
restricted to (pwr+int) ports.
ptp_status (ps): [str] this port is a PTP master or slave port. Note:
restricted to (peth+int) ports.
port_locked (pl): [bool] only for slave ports, True if locked, False if not
locked to a remote master port.
servo+int (se+int): [str+int] subset containing servo information. Servos
are numbered from 1.
port (po): [str+int] indicates which port the servo is tracking
locked (lo): [bool] indicates the hardware layer (CDR) of the servo is
locked to a remote master port
sync_info_valid (sv): [bool] is the WR synchronisation information of this
servo valid (NOTE: or is this the hardware lock of the CDR?)
state (st): [str] state of the white rabbit servo
auxiliary (aux): [int] state of the auxiliary servo
update_count (ucnt): [int] completed PTP message sequences count
round_trip_time (mu): [dec, picoseconds] total round trip time on the link.
compensated_round_trip_time (crtt): [dec, picosecond] round trip time minus
the rx and tx delays of master and slave
delay_master_slave (dms): [dec, picosecond] crtt/2
clock_offset (cko): [dec, picosecond] clock offset measured on the master of
the link
link_asymetry (asym): [dec, picosecond] link asymetry
current_setpoint (setp): [dec, picosecond] current setpoint (int) TJP:
clarify how this is used.
master_tx_delay (dtxm): [dec, picosecond] tx delay of master on the link
master_rx_delay (drxm): [dec, picosecond] rx delay of master on the link
slave_tx_delay (dtxs): [dec, picosecond] tx delay of slave on the link
slave_rx_delay (drxs): [dec, picosecond] rx delay of slave on the link
helper_dac (hd): [int] soft PLL dac value of dac -1
main_dac (md): [int] soft PLL dac value of dac 0
auxiliary_dac (ad): [int] soft PLL dac value of dac 1
temperature+int (te+int): [string+int] subset containing temperature
information.
temperature (te): [dec, Celcius] value of the integrated temperature sensor
on the PCB
left_temperature (tl): [dec, Celsius] value of the left temperature sensor
on the PCB
right_temperature (rt): [dec, Celcius] value of the right temperature sensor
on the PCB
fpga_temperature (tf): [dec, Celcius] value of the fpga temperature
pll_temperature (tp): [dec, Celcius] value of the PLL chip temperature
The following names are depricated because they are too generic or have been
redefined in the table above. The reason of deprication is given and the new use
is specified.
Depricated names:
-----------------
name: reason
------------
port: replaced by the str+int agregator names for the ports
lnk: replaced by link_up
The <value> field definitions:
------------------------------
Definitions in full and abreviated form for the string value fields will be
defined in this section. The <name> references the name field for which this
string value can be used in a unified way.
Values:
-------
<name>: full (abreviated)
-------------------------
white_rabbit_mode: master (m)
white_rabbit_mode: slave (s)
ptp_status: master (m)
ptp_status: slave (s)
servo_state: Uninitialized (u)
servo_state: SYNC_SEC (ss)
servo_state: SYNC_NSEC (sn)
servo_state: SYNC_PHASE (sp)
servo_state: WAIT_OFFSET_STABLE (wo)
servo_state: TRACK_PHASE (tp)
Examples:
---------
Two examples are worked out in this section. The first is how the output of
'stat cont' command on the SPEC would look in the new format and the second
shows how the wr_mon -a output on the WR Switch would look in the new format.
A \ indicates the continuation of the line.
The depricated SPEC 'stat cont' output:
lnk:1 rx:783 tx:239 lock:1 sv:1 ss:'TRACK_PHASE' aux:0 sec:86210 nsec:641173592\
mu:770966 dms:382661 dtxm:162949 drxm:167123 dtxs:166830 drxs:165361 asym:5644\
crtt:108703 cko:-1 setp:5464 hd:62459 md:31650 ad:36800 ucnt:192 temp: 43.4375\
C
Would look as follows in the new format:
tptp:{sec:86210 nsec:641173592.0} pwr1:{lu:T rx:783 tx:239 wm:s pl:T}\
se1:{po:pwr1 lo:T sv:T st:tp aux:0 ucnt:192 mu:770966.0\
crtt:108703.0 dms:382661.0 cko:-1.0 asym:5664.0 setp:5464.0 dtxm:162949.0\
drxm:167123.0 dtxs:166830.0 drxs:165361.0 hd:62459 md:31650 ad:36800} te0:{\
te:43.4375}
Note that the new format has additional values, which are currently not shown,
but could be shown for full compatibility of log formats.
The depricated 'wr_mon -a' output on the WR Switch:
TIME sec:1458298037 nsec:353517 PORTS port:wr0 lnk:0 mode:S lock:0 port:wr1\
lnk:0 mode:S lock:0 ... port:wr17 lnk:0 mode:S lock:0 SERVO lock:1 sv:1\
ss:'SYNC_NSEC' mu:1345860043 dms:672864378 dtxm:0 drxm:171200 dtxs:-15157\
drxs:24757 asym:131287 crtt:1345679243 cko:11544402 setp:0 ucnt:37 TEMP\
fpga:52.81 pll:48.50 psl:40.19 psr:42.75
Would look as follows in the new format:
tptp:{sec:1458298037 nsec:353517} pwr1:{lu:F wm:s pl:F} pwr2:{lu:F wm:s pl:F}\
... pwr18:{lu:F wm:s pl:F} se1:{po:pwr1 lo:T sv:T st:sn ucnt:37\
mu:1345860043.0 crtt:1345679243.0 dms:672864378.0 cko:11544402.0 asym:131287.0\
setp:0.0 dtxm:0.0 drxm:171200.0 dtxs:-15157.0 drxs:24757.0} te0:{tf:52.81\
tp:48.50 tl:40.19 tr:42.75}
Notes:
- Due to the length ports were skipped at the ... sign.
- Many values, like port rx/tx and the DAC settings are not printed on the
switch. These could/should be exported.
Markdown is supported
0% or
You are about to add 0 people to the discussion. Proceed with caution.
Finish editing this message first!
Please register or to comment