wrs-developer-manual.in

\input texinfo    @c -*-texinfo-*-
%
% wrs-developer-manual.in - main file for the documentation
%
%%%%

%------------------------------------------------------------------------------
%
%                         NOTE FOR THE UNAWARE USER
%                         =========================
%
%    This file is a texinfo source. It isn't the binary file of some strange
%    editor of mine. If you want ASCII, you should "make wrs-developer-manual.txt".
%
%------------------------------------------------------------------------------

%
% This is not a conventional info file...
% I use three extra features:
%   - The '%' as a comment marker, if at beginning of line ("\%" -> "%")
%   - leading blanks are allowed (this is something I cannot live without)
%   - braces are automatically escaped when they appear in example blocks
%

@comment %**start of header
@documentlanguage en
@documentencoding ISO-8859-1
@setfilename wrs-developer-manual.info
@settitle wrs-developer-manual
@iftex
@afourpaper
@end iftex
@paragraphindent none
@comment %**end of header

@setchapternewpage off

@set update-month June 2016
@c the release name below is substituted at build time
@set release __RELEASE_GIT_ID__

@finalout

@titlepage
@title White Rabbit Switch: Developer's Manual
@subtitle Information about software in the White Rabbit switch, for developers and advanced users
@subtitle @value{update-month} (@value{release})
@author Alessandro Rubini, Adam Wujek, Benoit Rat, Federico Vaga, ...
@end titlepage
@headings single

@c ##########################################################################
@iftex
@contents
@end iftex

@c ##########################################################################
@c in texinfo we are mandated to have a Top node
@node Top
@top Introduction

The White Rabbit switch (or @sc{wrs}) is a major component of the
White Rabbit (@sc{wr}) network.  Like any modern managed switch, the
@sc{wrs} includes a CPU with its own operating system.

This manual is for people rebuilding or modifying @sc{wrs} software.
It explains how to rebuild the whole software stack from source
and how is the switch itself designed.

@c ##########################################################################
@node WRS Documentation
@chapter WRS Documentation


Up to and including release 4.0 of @sc{wrs} software this manual
was the only official documentation item; now that the device is mature
and the deployed base increases, we reorganized documentation.

@c ==========================================================================
@node The Official Manuals
@section The Official Manuals

This is the current set of manuals that accompany the @sc{wrs}:

@itemize @bullet

@item @i{White Rabbit Switch: Startup Guide}: hardware installation
   instructions. This manual is provided by the manufacturer: it describes
   handling measures, the external connectors, hardware features and the
   initial bring-up of the device.

@item @i{White Rabbit Switch: User's Manual}: documentation about
   configuring the @sc{wrs}, at software level.  This guide is maintained
   by software developers.  The manual describes
   configuration in a deployed network, either as a standalone device or
   as network-booted equipment.  The guide also describes how to upgrade
   the switch, because we'll release new official firmware images over
   time, as new features are implemented.

@item @i{White Rabbit Switch: Developer's Manual}: it describes the
   build procedure and how internals work; use of scripts and
   @sc{wrs}-specific executables and so on.  The manual is by developers
   and for developers.  This is the
   document to check if you need to customize your @i{wrs} rebuild
   software from new repository commits that are not an official release
   point, or just install your @i{wrs} with custom configuration values.

@item @i{White Rabbit Switch: Failures and Diagnostics}: describe various
   failure scenarios of a switch and ways how to recognize them.
   Additionally describe SNMP exports of a switch (@t{WR-SWITCH-MIB}).

@end itemize

The official @sc{pdf} copy of these manuals at each release
is published in the @i{files} tab of the software project in @t{ohwr.org}:
(@url{http://www.ohwr.org/projects/wr-switch-sw/files}).
This doesn't apply to release 4.0 and earlier.

The source form of all four manuals is maintained in @t{wr-switch-sw/doc}.
Within the repository, three of them the @i{User's Manual},
the @i{Developer's Manual} and the @i{Failures and Diagnostics}
are always tracking the software commits, while the @i{Startup Guide} may not
be authoritative because it is bound to device shipping rather than software
development.

@c ==========================================================================
@node Supported Hardware Versions
@section Supported Hardware Versions

This document applies to versions 3.3 and 3.4 of the @sc{wrs}
device.

Very few specimens of @t{wrs} 3.0 though 3.2 were manufactured; if you
are the owner of one of them, please refer to version 3.3 of the
@i{wrs-build} document, that includes appendixes about using older
versions. As usual, it is in the @i{files} tab of @t{ohwr.org}.

V1 and V2 were development items, never shipped.

@c ##########################################################################
@node Building WRS Software
@chapter Building WRS Software

@c ==========================================================================
@node Overview
@section Overview

To build your switch firmware you have to first clone the main git repository,
which contains all the sources and building scripts:
@example
   git clone git://ohwr.org/white-rabbit/wr-switch-sw.git
@end example

The scripts build over previous work by Tomasz Wlostowski, who first
made the whole thing work and stick together -- a serious result from
serious efforts, I am really amazed by his achievements.

The purpose of the build-script rewrite is achieving the following targets:

@itemize @bullet
@item One-command build.
	The non-technical user should be able to rebuild the whole software
        package with a single command. This includes the IPL and boot-loader
        even though they are expected to be pre-installed in the switch with
        no real need for upgrading.
@item Sub-package separation.
	Users and developers should be able to rebuild each
        sub-package by itself. Sub-packages are the kernel,
	@i{buildroot}, libraries and so on. If you have a problem (or a
        customization you need on one sub-package), you should be able to work
        on the specific part ignoring the whole as much as possible.
@item Documentation.
	The steps are documented as much as possible, because mishaps
        do happen, and you should easily understand where the problem is.
@item Avoid redundant downloads.
	People with non-mainstream network connections would rather
	avoid downloading the same package over and over. Thus, a centralized
	download directory is defined where all external packages are
        retrieved. Even if you ``make distclean'' in the build scripts you
        will not need re-get everything from the network.  In a similar
        mood, people who already have a local copy of the big packages
        (kernel, barebox, white-rabbit) will not need to re-download not
        even the first time they build the WRS software.
@end itemize

After release 3.3, we decided to add @i{Kconfig} support. This means
that the first build step is expected to be ``@t{make menuconfig}'',
like it happens for the kernel.  The default configuration is selected
by default when one of the build scripts is run, so the procedure for
the final user is the same as for v3.3 and earlier.

After release 4.1 we support dynamic reconfiguration. As opposed
to build-time configuration, run-time configuration is expected to
be frequent, and it's thus documented in the @i{@sc{wrs} Users' Manual}.

The build system is set up as a mix of scripts and makefiles. Every
sub-package is built by its own script and/or Makefile, and configuration is
passed over through environment variables. The top-level build script
sets all environment variables, while keeping defaults from your
preexisting environment -- so you can override anything even when
rebuilding it all from scratch.

@c --------------------------------------------------------------------------
@node Other Repositories
@subsection Other Repositories

Not everything that runs in the switch comes from this package:
both the @i{gateware} image and the @i{lm32} firmware binary
are built from sources external to this package.

The @i{gateware} is being downloaded as a pre-build tar archive
(currently called @code{wrs-gw-v4.2-20150826.tar.gz}).  This
is built from the @code{wr-switch-sw-v4.2} tag of the @code{wr-switch-hdl}
repository.  Please note that the repository uses @i{git} submodules,
so it depends on other @code{ohwr} repositories too, which in turn
have not been tagged because the submodule mechanism ensures you'll
get the exact version you need. Anyways, all relevant commit identifiers
are shown by command @t{wrs_version -t} or in the SNMP version fields
(within @t{WR-SWITCH-MIB.txt}).

The LM32 program is provided as a pre-compiled binary in
@code{binaries/rt_cpu.bin}. The respective source code is the
@i{wrpc-sw} package, because all WR installations run the same
@sc{pll} software code and we chose to avoid duplication. Moreover,
@i{wr-switch-sw} builds to not require an LM32 development environment.

If you need to rebuild the @t{rt_cpu.bin} file from source, to make
your own modifications, you can run @t{make wr_switch_defconfig}
in @i{wrpc-sw} and then @t{make}. Please checkout the
@code{wr-switch-sw-v4.2}
tag to get the exact commit.

@c --------------------------------------------------------------------------
@node Portability
@subsection Portability

The scripts in their current status are not expected to be very
portable. I am sure a number of @i{bashisms} exist, and I did no effort
to even identify them.  To relieve the user from possible pain,
internally the name @i{bash} is used instead of @i{sh}, so things
work in systems where the default shell is @i{dash}, provided @i{bash}
is installed.

Similarly, the scripts are likely to fail if you use spaces in directory
names; that is because not all
uses of shell variables are properly quoted. I urge you to use directory names
with no spaces in them, or to submit a patch to fix the scripts.

It should go without saying that the build environment is expected to
be a native GNU/Linux system; success reports about other environments
(e.g. cygwin) are welcome, possibly with associated patches.

@c --------------------------------------------------------------------------
@node Environment Variables
@subsection Environment Variables

The scripts use a number of environment variables; you can pre-set
them as you wish. If they are not pre-set, defaults apply as
described below.

When building running the @i{build/wrs_build-all} script (whether you
build everything or rebuild individual steps) the defaults are applied
for each unset variable.  Developers working under the hood will need
to set the variables.  Each sub-package complains if it needs
variables that are not set in their environment.

The following variables are used in one or more parts of the scripts;
let me restate, though, that sensible default values apply by default,
so this list is mainly for your curiosity unless you are a developer.

They are listed in an order that seems logical to me, but may sound
random to a different person, please forgive this.  Most of the
variables are prefixed with @code{WRS_} to make them easily identified
in the overall mess of variables and command names (all scripts used here
have a similar prefix for the same reason).

@table @code
@item WRS_BASE_DIR
	The absolute pathname of the build directory (i.e., the @code{build/}
        subdirectory of @code{wr-switch-sw}).
        The variable is internally set to the directory
        name of the main script. Note that the script cannot
        be run from the same directory or from the wr-switch-sw project
        directory (i.e.: @code{./wrs_build-all} @code{./build/wrs_build-all}
        are not allowed), you must call it from your output directory
        using a pathname to  invoke it.  This variable cannot be
        overridden in the main script, but must be pre-set if you run
        a sub-script to rebuild only part of the software suite.

@item WRS_OUTPUT_DIR
	The absolute pathname of the directory where output is placed. It 
	defaults
        to the current directory whence you invoke the script (i.e., you
        can invoke @code{/path/to/wrs_build-all} to have all output
        in the current directory).  Compilation
        happens in a @i{build} subdirectory of @code{WRS_OUTPUT_DIR},
        done-markers are placed in a @i{_done} subdirectory
        and final images are placed in a @i{images} subdirectory.

@item WRS_DOWNLOAD_DIR
	The absolute pathname of the directory where downloaded files 
	are placed. If unset
        it defaults to @code{$WRS_OUTPUT_DIR/downloads}, which is
        created if needed.  By pre-setting this variable you can
        simply recursively delete
        the output directory to force a full rebuild, without
        the overhead of re-downloading everything. I personally pre-set
        this so it always points to the same place, even when I remove
        the whole output directory.

@item WRS_HW_DIR
	The absolute pathname of the directory where you build HDL, if any.
        If this variable is set, FPGA binaries will be copied from there
        instead of being extracted by the official archive on @code{owhr.org}.
        This is only used by HDL developers.

@item CROSS_COMPILE
	The variable is the usual cross-compilation prefix. For example,
        @code{arm-linux-} if you have @i{arm-linux-gcc} in your path,
        or a full pathname without the trailing @code{gcc}. If unset,
        it defaults to the compiler that @i{buildroot} self-builds.
        See @ref{The Compiler} for some more details.

@end table

Other variables are used internally in the script; since they are only
useful to people working on the script itself, they are documented in
place.


@c --------------------------------------------------------------------------
@node Downloading Files
@subsection Downloading Files

Every downloaded file is saved to the @code{downloads} directory 
(@code{$WRS_DOWNLOAD_DIR} if set, or the default place
@code{$WRS_OUTPUT_DIR/downloads}). You should
arrange not to remove that directory when you recompile over and over
during development. Download process is divided into two parts. Firstly,
our buildsystem downloads only base packages (at91bootstrap, barebox,
linux kernel, switch's gateware and the builroot). The rest of packages are
downloaded by the buildroot.
In the first step the script downloads mentioned packages, before starting
any build, to help telling download errors from other issues.

For each upstream archive needed, the following steps are performed:

@itemize @bullet
@item If the file exists in the download directory, the @i{md5sum} is
      checked; on success, nothing else is done.
@item If the previous step fails, the file is retrieved from upstream.
@item If the previous step fails, the file is downloaded from the
      buildroot web site.
@end itemize

The policy just described is implemented in @i{wrs_download}, in the file
@code{scripts/wrs_functions}, based on @code{download-info} in the
main build directory.

The messages of a download run are like the following ones:

@smallexample
    2016-06-02 17:10:46: --- Downloading base packages
    2016-06-02 17:10:50: Retrieved at91bootstrap-3-3.0.tar.gz from upstream
    2016-06-02 17:10:51: Retrieved barebox-2014.04.0.tar.bz2 from upstream
    2016-06-02 17:11:21: Retrieved linux-2.6.39.tar.bz2 from upstream
    2016-06-02 17:11:22: Retrieved wrs-gw-v4.2-20150826.tar.gz from upstream
    2016-06-02 17:11:27: Retrieved buildroot-2016.02.tar.bz2 from upstream
@end smallexample

After buildroot is downloaded, it is unpacked and then configured. Buildroot
uses simillar mechanism to the one described above to download packages that
it needs. Buildroot prints the progress of download of each package.
After downloading is over you can work even without a network connection.

@c ==========================================================================
@node Building Procedure
@section Building Procedure
If you just want to build stuff, with no concern about network
downloads and without even knowing what is happening, just create a
directory where you want the output to be generated and start
compilation. Note that it takes around 4GB of storage in excess
of the 330MB downloaded.

Then run this (but please read more for a better command):

@example
   /path/to/wr-switch-sw/build/wrs_build-all
@end example

Note that progress messages are sent to @i{stderr}, so you may want to
save @i{stdout} to a file, like this (again, it is recommended you read
further for a better command):

@example
   /path/to/wr-switch-sw/build/wrs_build-all > logfile
@end example

Please note that there are also a number of warning messages being
printed to @i{stderr}. It is a few hundred lines over the many
minutes it takes to build @i{buildroot}, but you can safely ignore them,
trusting the build process will complete successfully.

The progress messages look like what is shown here below.  The log
file will be rather big (~18MB), as all the compilation steps are
quite verbose.

The following example shows a run on a quad core, dual hyperthreaded system
(8*6800 bogoMips in total). If files had already been downloaded, the first few
step takes only a few seconds, as shown, to verify the checksums:

@smallexample
2016-06-02 17:26:39: --- Downloading base packages
2016-06-02 17:26:39: --- Buildroot: unpack and configure
2016-06-02 17:26:39: Uncompressing buildroot
2016-06-02 17:26:40: Configuring with "[...]/../configs/buildroot/wrs_release_br2_config"
2016-06-02 17:26:40: Patching buildroot
2016-06-02 17:26:40: Reconfiguring buildroot
2016-06-02 17:26:41: --- Buildroot: download packages
2016-06-02 17:26:48: --- Buildroot: compiler and filesystem
2016-06-02 17:26:48: Compiling buildroot
2016-06-02 17:47:54: --- AT91Boot
2016-06-02 17:47:54: Patching AT91Boot
2016-06-02 17:47:54: Building AT91Boot
2016-06-02 17:47:55: --- Barebox
2016-06-02 17:47:55: Patching Barebox
2016-06-02 17:47:55: Building Barebox
2016-06-02 17:48:03: --- Linux kernel for switch
2016-06-02 17:48:52: --- Kernel modules from this package
2016-06-02 17:48:56: --- PTP daemon (ppsi repository as a submodule)
2016-06-02 17:49:05: --- User space tools
2016-06-02 17:49:15: --- Deploying FPGA firmware
2016-06-02 17:49:15: Using pre-built binaries from wrs-gw-v4.2-20150826.tar.gz
2016-06-02 17:49:16: --- Wrapping filesystem
2016-06-02 17:49:21: --- Packing into wr-switch-sw-v4.2-20160602_binaries.tar
2016-06-02 17:49:21: Complete build succeeded, apparently
@end smallexample

You may prefer to save @i{stderr} with @i{stdout} to the log file
but still see the time-stamped messages from the scripts. In this
case you can issue the following command -- which is what I used
to generate the terse output shown above:

@example
   /path/to/wr-switch-sw/build/wrs_build-all 2>&1 | tee logfile \
        | grep "^20..-..-.. ..:"
@end example

If you are lucky, everything completes by itself. The time taken
depends on you CPU, disk and network speed.
At the end you will find your final files in the @code{images}
subdirectory, 

If you are not too lucky, the build stops because
you have found a bug in the build scripts; most likely because your
setup differs from the ones we have been testing on.

In order to re-run the build from the beginning, please remove (or
rename) the output directory and reissue the command.  To only
redo some steps, please see @ref{Rebuilding Parts}.

@c ==========================================================================
@node Build Script Description
@section Build Script Description

The @code{wrs_build-all} can be used to quickly build the White Rabbit 
Software as seen above. However it admits other functionalities detailed
in this chapter. You might also want to check its embedded documentation using:

@example
   /path/to/wr-switch-sw/build/wrs_build-all --help
@end example

@c --------------------------------------------------------------------------
@node Release Package
@subsection Release Package

By default, a complete compilation creates a ``release'' package,
i.e. a @i{tar} archive of all files needed to flash a brand new WR
Switch.  The example above shows that the name is something like:

@example
	wr-switch-sw-v4.2-20150828_binaries.tar
@end example

In other words, we include both the tag name (from @t{git describe})
and a date.  If the repository is not checked-out at a release (a ``tag''),
this will be apparent in the filename used for the output.

The ``official'' release package is available from @t{ohwr.org}, in
the @i{Files} section of the @t{wr-switch-sw} project.

In any case, the file must be renamed to @t{wrs-firmware.tar}
to be used at installation time. See @ref{Flashing Procedure} for details.

@c --------------------------------------------------------------------------
@node Build Time Configuration
@subsection Build Time Configuration


Some details of the complete firmware archive depend on the values of
active @t{Kconfig} variables. If no manual configuration is performed,
what applies is @t{configs/wrs_release_defconfig}. Please note that
we now support dynamic reconfiguration at run-time. See the @i{@sc{wrs}
Users' Manual}.

@c --------------------------------------------------------------------------
@node Rebuilding Parts
@subsection Rebuilding Parts

When the main script succeeds in building one part (sub-package),
it creates a file in the @code{build/_done} directory.

When you rebuild everything, steps for which the marker file exists
are not rebuilt. To force rebuilding of one specific part, just remove
the marker.  Markers are numbered, reflecting the order of compilation
steps, but they also have a name: names like @code{06-kernel} should be
self-explicative. 

To ease the rebuilding of a specific module a shortcut has been created 
in the @code{wrs_build-all} script. For example if you want to recompile
the kernel alone you should execute.

@example
   /path/to/wr-switch-sw/build/wrs_build-all --step=06
@end example

You can list all compiled modules by calling

@example
   /path/to/wr-switch-sw/build/wrs_build-all --list
@end example
 
If you want to rebuild various modules at the same time, you should run
something similar as:

@example
   /path/to/wr-switch-sw/build/wrs_build-all --step="5 7"
@end example

Please note that the final step (``@i{wrap root filesystem''}) is
always performed, to ensure any changes are applied in the generated
firmware file.

An alternative way to build parts, though a more difficult one, is running the
individual script from within @i{build/scripts/}, after setting the proper
environment variables.


@c --------------------------------------------------------------------------
@node Rebuilding From Scratch
@subsection Rebuilding From Scratch

If you have updated the repository with new modifications, you might want 
to check that you can rebuild from scratch. To clean your output 
directory by deleting all compiled modules (except downloaded files), just call:

@example
   /path/to/wr-switch-sw/build/wrs_build-all --clean
@end example


@c ==========================================================================
@node The Individual Build Steps
@section The Individual Build Steps

This chapter details the individual build steps, for the users that want
to customize one or more of them to build a different switch firmware
image (or kernel, or whatever).

@c --------------------------------------------------------------------------
@node The Compiler
@subsection The Compiler

The predefined compiler used here is the one built by @i{buildroot}.
The default configuration selects this choice.  If you pre-set a
different @code{CROSS_COMPILE} prefix in your environment, your own
choice will be used by modifying the @i{buildroot} configuration file.
Note, however, that not all cross-compilers will work (@i{buildroot}
wants one that has been configured with @code{--sysroot} and it is
quite unlikely yours has been).

In practice, you may want to set @code{CROSS_COMPILE} when you compile
the boot loader and kernel by themselves, and avoid it when compiling
the complete package.

@c --------------------------------------------------------------------------
@node Buildroot
@subsection Buildroot

The distribution being used here is @i{buildroot}. It is the first
step being built, because it creates the cross-compiler it will use. This
compiler is later used to compile all other software for the White
Rabbit Switch.

The configuration for @i{buildroot} comes from
@code{configs/buildroot/wrs_release_br2_config}. The configuration
is then changed only if you pre-set your own @code{CROSS_COMPILE}
variable.  A different configuration can be chosen in the Kconfig
interface, by running ``@t{make menuconfig}'' or equivalent, in the
top-level source directory.

@c The following can be shown in a set of commands instead of this hard
@c to read explanation.

If you want to change the configuration, you can do so after the first
build iteration: change directory to @code{build/buildroot-2016.02}
and run @code{make menuconfig} (this the Buildroot configuration,
not the one of wr-switch-sw). After making your choices, copy back
the file @code{.config} to @t{configs/buildroot} in this package,
so you can select it by running @t{make menuconfig} in wr-switch-sw.

Then, please run @t{configs/buildroot/RUNME} (without arguments)
in order to remove your local pathnames in the configuration file;
the file without local pathnames can be committed and published for
other people to use.

You can also set @code{WRS_BUILDROOT_CONFIG} to the full pathname of
your configuration file of choice (this used to be the only way
to pass a custom configuration file). The file must be a copy of the
@code{.config} after the @code{make menuconfig} step described above,
within buildroot.
Note that if the variable is not pointing to a regular file it is
ignored with a simple warning -- rather than stopping the build procedure.

@c --------------------------------------------------------------------------
@node The IPL
@subsection The IPL

The version of @i{at91bootstrap} being used in the switch as
@i{Initial Program Loader} is version 3.3, download from  @code{timesys.com/}
(the full URL is in @i{build/download-info}.
The patches we applied are in the directory @i{patches/at91boot/v3.3},
and we are piggy-backing on the Atmel
evaluation board without even changing the board name):

@example
   0001-printf-added-files-from-pptp-unchanged.patch
   0002-printf-fixes-and-addition-to-makefile.patch
   0003-build-Add-gitversion-to-binary-and-a-script-to-compi.patch
   0004-board-9g45ek-fix-ddr-config-for-WRS-V3.patch
   0005-boot-disable-watchdog-asap-added-flip_leds-count-run.patch
   0006-boot-Correct-crash-due-to-an-Atmel-bug-during-boot-w.patch
   0007-gpios-Correct-FPGA-LED-problems-and-add-CPU-LEDs-FAN.patch
   0008-fix-refresh-value.patch
   0009-one-more-fix-to-RAM-timing-according-to-datasheet.patch
@end example

The script @i{wrs_build_at91boot} uncompresses, patches and builds, leaving
@code{images/@-at91bootstrap.bin} after it is over.  This file is
the one to be loaded in the hardware.

If you build using a local @i{git} repository, we suggest to use
@code{git am --whitespace=nowarn} because we have a number of
white space errors, and we apologize for that.

@b{Warning}: with most distributions, this compilation step will print
a scary message about memory corruption. The message is reporting a bug in
the configuration program which has no actual effects and can be ignored.
Maybe we will switch to another version in the future that doesn't show
the bug, or to the newer @i{barebox} that obsoletes @i{at91boot}.

@c --------------------------------------------------------------------------
@node The Boot Loader
@subsection The Boot Loader

The switch uses @i{barebox} as a boot loader. We are running version
2014-04, with a few local patches and the chosen configuration
file. Note that we are piggy-backing on the Ronetix PM9G45 board, out
of laziness.

The patches are part of this package in @i{patches/barebox/v2014.04/} and
the set is made up of the following ones:

@smallexample
   0001-sam945-include-mtd-nand.h-in-device-file.patch
   0002-arm-change-prompt-for-pm9263-wrs-piggy-backs-on-that.patch
   0003-nand-wrs-our-nand-is-16-bit-connected-fix-accordingl.patch
   0004-gpio-add-function-to-check-them.patch
   0005-wrs-on-pm9g45-change-nand-setup.patch
   0006-wrs-on-pm9g45-add-dataflash-initialization.patch
   0007-barebox-add-MAC-addresses-to-environment.patch
   0008-wrs-on-pm9g45-force-memory-to-64MB.patch
   0009-pm9g45-init-for-wrs-move-environment-for-the-UBI-mov.patch
   0010-pm9g45-init-for-wrs-more-relaxed-nand-timings.patch
   0011-lib-sdb-and-sdb.h-from-fpga-config-space-sdbfs-commi.patch
   0012-libsdb-integrate-in-build-system.patch
   0013-commands-add-sdb-commands-to-list-read-setvar.patch
   0014-Read-EDI-bytes-in-JEDEC-to-support-AT45DB641E.patch
@end smallexample

If you build using a local @i{git} repository, we suggest to use
@code{git am --whitespace=nowarn} because we have a number of
white space errors, and we apologize for that.

The @i{barebox} boot loader is organized as a small Unix-like
environment, and its own configuration and scripts live in a small
filesystem.  To ease modification of such configuration and boot steps
the build script copies over the configuration instead of patching it
in the sources.  You can thus edit the files you find in
@file{patches/barebox/v2014.04/env} and rebuild your customized bootloader.
The script that is executed at boot time is @file{env/bin/init} and as
you see it calls the other ones.  The menus included in the shipped
configuration are described in the the @i{@sc{wrs} User's Manual}.

Building @i{barebox} relies on a @i{Kconfig} setup, and the
configuration file we use is
@file{patches/barebox/v2014.04/@-wrs3_defconfig}. Again, this is copied over and
not patched in (see the simple @file{build/scripts/@-wrs_build_barebox}
for details).

@c FIXME: describe how to change the barebox configuration file

After patching and copying over the files, the following commands
build the boot loader using the
cross-compiler built by @i{buildroot}. If you run these
by hand you can use a different compiler (as shown):

@example
   export CROSS_COMPILE=/opt/arm-2010q1/bin/arm-none-eabi-
   export ARCH=arm
   make wrs3_defconfig
   make
   cp barebox.bin images/
@end example

To use the same compiler the scripts use, you need this setting (which is split
in two lines with a local variable to fit the page with in documentation):

@smallexample
   BR=${WRS_OUTPUT_DIR}/build/buildroot-2016.02
   export CROSS_COMPILE=${BR}/output/host/usr/bin/arm-linux-
@end smallexample

@c --------------------------------------------------------------------------
@node The Linux Kernel
@subsection The Linux Kernel

The kernel is currently version 2.6.39, compiled from an uncompressed
tar file (so not within a @i{git} repository).  The upstream
vanilla kernel is downloaded, then
local patches are applied (they come from a @i{git}
repository, but they are currently applied with a simple @i{patch}
command).

The relevant patches are available in @i{patches/kernel/v2.6.39},
and are currently the following ones:

@example
   0001-wrs3-changes-to-g45ek.patch
   0002-initramfs-stop-after-one-cpio-archive.patch
   0003-at91-NR_IRQS-increase-by-64-to-fit-custom-muxes.patch
   0004-irq-export-symbols-for-external-irq-controller.patch
   0005-Change-Vbus-pin.patch
   0006-arm-fiq-allow-modules-to-exploit-the-fiq-mechanism.patch
   0007-mtd-nand-sam9g45-can-hwecc-like-9263.patch
   0008-wrs3-use-correct-nand-partitioning.patch
   0009-at91-udc-force-full-speed.patch
   0010-sam9m10g45ek-for-wrs-new-partitioning.patch
   0011-sam9m10g45ek-for-wrs-final-partitions-for-V4.1.patch
   0012-sam9m10g45ek-for-wrs-more-relaxed-nand-timings.patch
   0013-mtd_dataflash-Read-EDI-JEDEC-to-support-AT45DB641E.patch
   0014-sam9m10g45ek-for-wrs-provide-bootcount-using-scratch.patch
@end example

The configuration we use to build the kernel is not a patch but a plain
@code{.config} file, in the same directory as the patches, so you
can change it easily, if needed. As an alternative,
you can also set @code{WRS_KERNEL_CONFIG} to the full pathname of
your configuration file of choice. The file must be a copy of the
@code{.config} found in the main kernel directory,
(for example the one left after the @code{make menuconfig} step).
Note that if the @code{WRS_KERNEL_CONFIG}
variable is not pointing to a regular file it is
ignored with a simple warning, without stopping the build procedure.

The build scripts copy both @i{zImage} and all compiled kernel
modules to the @i{images/} directory of the build place. This currently
includes modules 

@c --------------------------------------------------------------------------
@node Kernel Modules
@subsection Kernel Modules

In the next step the scripts compile modules that are part of this
package.  The step depends on the kernel being available in the
build directory. The modules are then copied into the
@file{images/wr/lib/modules/} subdirectory of the main build directory.

Please note that modules (and later user-space) are compiled in-place;
ie. not in the output directory.  The disadvantage is that your repository
becomes dirty with output and intermediate files. The advantage is that
any modification you make to the code is already in the repository
for your to commit.

Currently, the package includes the following modules:

@itemize @bullet

@item @i{wr_vic.ko}: the interrupt controller for in-FPGA devices.
@item @i{wr-nic.ko}: the network ``card'' driver for WR ports.
@item @i{wr_rtu.ko}: the routing-table interface between the
      switching core and the associated user-space daemon.
@item @i{wr_pstats.ko}: exports per-port statistics to /proc/sys.

@item @i{wr_clocksource.ko}: uses WR time as a source for system time.
This driver uses the @sc{wr} counters to make host time flow at the
right speed as soon as @i{ppsi} synchronizes. This ensures no drift
will accumulate in system time, keeping the same
offset (well lower than a second, after the initial ntp-driven setting event).
This is considered acceptable, because system time is only
used for logging.

@item @i{at91_softpwm.ko}: a driver that generates a PWM signal for the fan.

@end itemize

@c --------------------------------------------------------------------------
@node PTPd
@subsection PTPd

Configuration used to support two different PTP engines, but now
we only support PPSi.

The code is hosted in its own
repository; it is a @i{git} submodules in this package.
The repository is hosted on @code{ohwr}, like others.

PPSi is @i{Kconfig} based: you may build it in its own directory by
using its @i{wrs_defconfig} and the proper @t{CROSS_COMPILE} variable.

@c --------------------------------------------------------------------------
@node User Space Applications
@subsection User Space Applications

The filesystem of the switch includes some user-space applications
and tools.  Some of the @i{tools} are actually used by the init
scripts and some are just utilities for the developer.

The subdirectories in @file{userspace} include the various applications
needed for the operation of the switch itself, as well as support libraries
used by the applications themselves.

The main components are:

@table @i

@item mini-rpc
	A remote procedure call library used by most other programs
        to exchange information among themselves or query the LM32
        that is running on the FPGA.

@item libwr

	A series of utility functions to access the switch itself.

@item wrsw_hal
	The main application program for the White Rabbit Switch
        operation. The script installs the executable in @code{images/wr/bin}.

@item wrsw_rtud
	The daemon for the routing table unit, used for routing around
        data frames. It is installed in
        @code{images/wr/bin}.

@end table

@sc{wrs} user space includes also some tools and scripts.
They are all described in the User's Manual, even though some of
them are only useful to software developers.

Please note that to compile the applications and tools outside of the build
scripts you need to specify both the kernel
directory (@code{LINUX=}) and the cross-compiler to use
(@code{CROSS_COMPILE=}).

@c --------------------------------------------------------------------------
@node VHDL Binaries
@subsection VHDL and LM32 Binaries

The gateware binaries that are needed to run the FPGA are added to the
target filesystem by the @file{wrs_build_gateware} script. If the
variable @code{WRS_HW_DIR} is set, the script uses it to retrieve the
binaries you just compiled (but the script is not compiling gateware).

If the variable is not set, the script extracts a tar file downloaded
from @code{ohwr.org} as part of the initial download step.

The LM32 program is provided as a pre-compiled binary in
@code{binaries/rt_cpu.bin}.

More details about the binaries are in @ref{Other Repositories}.

@c --------------------------------------------------------------------------
@node The Complete Filesystem
@subsection The Complete Filesystem

The final step in building the switch software is wrapping together
the filesystem for the switch, also making the archives.

The step of setting up the complete filesystem is performed by
@file{build/scripts/@-wrs_build_wraprootfs}.  The script
does not leave a directory tree on the build system because that would require
administrator privileges. We think it is best not to call @i{sudo} from
within build scripts, to respect our users' security concerns.

The script creates an archive for the whole filesystem,
called @t{wrs-image.tar.gz}.  It is used by the installation
procedure and it is ready to be unpacked for NFS-Root. It is currently 
slightly less than 25MB of data.

To make your NFS-root place, you can run
the following command in a newly-created empty directory:

@example
   tar xzf $WRS_OUTPUT_DIR/images/wrs-image.tar.gz
@end example

To boot with NFS-root you should use a custom boot script,
as described in the section @i{Using wrboot}, in the @i{@sc{wrs} User's
Manual}.

The archives include a number of device special files in
@i{dev}. The pre-created devices come from
@i{userspace/devices.tar.gz}. Note that the buildroot output
directory, @i{build/buildroot-2016.02/output/target} does not
include any device (and no white-rabbit specific files), so it
cannot be used as a root filesystem by itself.

The content of the final filesystem comes from several sources:

@itemize @bullet

@item The @i{buildroot} output (from its own @file{output/target/}).
@item The switch-specific overlay (@file{userspace/roofs_override}).
@item The @file{images/wr} and @file{images/lib} trees,
      filled by the build scripts.
@item The file @file{userspace/devices.tar.gz}
@item The file @file{$WRS_BASE_DIR/authorized_keys} if it exists.

@end itemize

The final step allows a predefined set of users to enter as system
administrator without the need to type a password (which, anyways is
empty by default).  It is useful if you @i{scp} files in and out of
the switch. In the shipped binaries no user is authorized, but the
root password is still the empty string.

@c ##########################################################################
@node Flashing the Switch
@chapter Flashing the Switch

This chapter describes the steps to install the @sc{wrs} with the 
current firmware images. As far as hardware is concerned, this
procedure describes the installation of the switch with a @sc{scb}
version 3.3 or 3.4, and a @sc{Mini-Backplane} version 3.3.
Older versions are not documented here
any more (please get an older manual, if needed).

@b{Note:} Most likely you won't need to reflash your @sc{wrs}.
Even if you rebuilt software from scratch, the ``update'' procedure
that is available since August 2014 (release 4.0 of @t{wr-switch-sw})
allows complete replacement of the firmware image without physical
access to the device.

@c ==========================================================================
@node USB connections
@section USB connections

In order to perform the flashing operation easily, you should connect
two @i{mini-USB} cables to the switch ports (Actually, one is enough, but the
second one is useful to get more diagnostics while flashing).

The two back panel @i{mini-USB} sockets correspond to the serial 
port of the FPGA and the ARM. They are labeled @b{FPGA test} and 
@b{ARM test}.  You should connect to ``ARM test'' to get diagnostics.

You can connect to it using minicom 
@footnote{You can use other programs for accessing serial ports, for 
example @uref{http://brokestream.com/tinyserial.html, tinyserial}} 
like this:

@example
	minicom -D /dev/ttyUSB0 -b 115200
@end example

The port, however, will only appear on the PC after the switch is turned
on, so you may want to delay this command.

@sp 1

The front panel USB connection, labeled as @b{management} USB port, communicates 
with the internal ROM of the CPU. It is the one used to perform the 
flashing procedure.  No special program is needed, as the flashing tool
will communicate with this port by itself.

You first need to set up the switch in @emph{"Flashing mode"} to 
continue with the flashing procedure. To do so, you should turn on 
the power while pressing the @b{flash button} on the rear panel.