Skip to content

W
White Rabbit Switch - Testing
Commit ade56d62 authored by Jean-Claude BAU's avatar Jean-Claude BAU
Browse files
Options

switch-running-release-test document

parent 7f8f75eb
Expand all Hide whitespace changes
Inline Side-by-side
Showing with 17287 additions and 0 deletions
sw/xena-tester/running-tests/.gitignore 0 → 100644
View file @ ade56d62
*.aux
*.dvi
*.log
*.out
*.toc
sw/xena-tester/running-tests/Makefile 0 → 100644
View file @ ade56d62
wrs_version=6.0.0
document=switch-running-release-tests
document_tex=$(document).tex
document_pdf=$(document).pdf
document_ps=$(document).ps
all : $(document_pdf)
.PHONY : all clean
$(document_pdf) : *.tex
latex ${document_tex}
latex ${document_tex}
pdflatex ${document_tex}
clean :
rm -f *.eps *.pdf *.dat *.log *.out *.aux *.dvi *.ps *.toc *~
sw/xena-tester/running-tests/introduction.tex 0 → 100644
View file @ ade56d62
\section{Introduction}
\label{chap:intro}
This document describes how to execute the White Rabbit switch pre-release tests
described in the \hrefreleasetests document. \\
sw/xena-tester/running-tests/pictures/AvalaibleRessources.eps 0 → 100644
View file @ ade56d62
This source diff could not be displayed because it is too large. You can view the blob instead.
sw/xena-tester/running-tests/pictures/LoadTestCase.eps 0 → 100644
View file @ ade56d62
This diff is collapsed.
sw/xena-tester/running-tests/pictures/RunTest.eps 0 → 100644
View file @ ade56d62
This diff is collapsed.
sw/xena-tester/running-tests/pictures/RunTestDone.eps 0 → 100644
View file @ ade56d62
This source diff could not be displayed because it is too large. You can view the blob instead.
sw/xena-tester/running-tests/pictures/cleanLog.eps 0 → 100644
View file @ ade56d62
This source diff could not be displayed because it is too large. You can view the blob instead.
sw/xena-tester/running-tests/pictures/tree.eps 0 → 100644
View file @ ade56d62
This diff is collapsed.
sw/xena-tester/running-tests/pictures/wrmon.eps 0 → 100644
View file @ ade56d62
This diff is collapsed.
sw/xena-tester/running-tests/switch-running-release-tests.tex 0 → 100644
View file @ ade56d62
%
% switch-running-release-tests.tex
%
\def\us{\char`\_}
\documentclass[a4paper, 12pt]{article}
%\documentclass{article}
\usepackage{fullpage}
\usepackage{listings}
\usepackage[table,xcdraw]{xcolor}
\usepackage{pgf}
\usepackage{tikz}
\usetikzlibrary{arrows,automata,shapes}
\usepackage{multirow}
\usepackage{color}
\usepackage[latin1]{inputenc}
\usepackage{verbatim}
\usepackage{amsmath}
\usepackage{times,mathptmx}
\usepackage{chngcntr}
\usepackage{mdwlist}
\usepackage{float}
\usepackage{hyperref}
\usepackage{graphicx}
\usepackage{adjustbox}
\usepackage{tabularx}
\usepackage{wrapfig}
\graphicspath{ {./pictures/} }
\restylefloat{table}
%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%
% creating subsubsubsection notation
% src: http://www.latex-community.org/forum/viewtopic.php?f=5&t=791
%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%
\setcounter{secnumdepth}{6}
\renewcommand\theparagraph{\Alph{paragraph}}
\def \wrsversion {6.0.0}
\def \docversion {1.0.0}
%\def \gitbranch {master}
\def \gitbranch {jcb-proposed\_master}
\def \gitproject {https://ohwr.org/project/wr-switch-testing}
\def \gitrepo {\gitproject/tree/\gitbranch/sw}
\def \wikiorigin {\gitproject/wikis}
\def \wikidoc {\wikiorigin/Documents}
\def \wikiuploads {\wikiorigin/uploads}
\newcommand* \hrefreleasetests {\href{\wikiuploads/991d55af277ab43a0f672af2618f1cbc/switch-release-tests.pdf }{ '\textbf{switch-release-tests.pdf' }}}
\makeatletter
%\renewcommand\paragraph{\@startsection{paragraph}{4}{\z@}%
% {-3.25ex\@plus -1ex \@minus -.2ex}%
% {0.0001pt \@plus .2ex}%
% {\normalfont\normalsize\bfseries}}
%\renewcommand\subparagraph{\@startsection{subparagraph}{5}{\z@}%
% {-3.25ex\@plus -1ex \@minus -.2ex}%
% {0.0001pt \@plus .2ex}%
% {\normalfont\normalsize\bfseries}}
%\renewcommand{\thefootnote}{\fnsymbol{footnote}}
%\renewcommand{\thefootnote}{\alph{footnote}}
\counterwithin{paragraph}{subsubsection}
\counterwithin{subparagraph}{paragraph}
\makeatother
%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%
\newcommand{\codebox}[1]{%
\noindent\fbox{\parbox{\textwidth}{ #1 } }}
\newcommand{\eqoffset}[1]{%
{\ensuremath{%
{\text{offset}}_{#1}}%
}%
}
\newcommand{\eqdelay}[1]{{\text{delay}}_{#1}}
\newcommand{\eqasymm}{{\text{asymmetry}}}
%\renewcommand*{\refname}{\vspace*{-3em}}
\begin{document}
\title{White Rabbit Switch - version \wrsversion \\ Running pre-release tests \\ \docversion}
\author{Jean-Claude Bau\\CERN}
% \date
\maketitle
% \thispagestyle{empty}
\newpage
\tableofcontents
\newpage
%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%
\newpage
\input{introduction}
\newpage
\input{tests-environment}
\newpage
\input{tests-running}
\end{document}
sw/xena-tester/running-tests/tests-environment.tex 0 → 100644
View file @ ade56d62
\section{Tests environment}
\label{chap:environment}
%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%
\subsection{Hardware environment}
The WR switch to test must be connected to the Xena-tester as follow :
\begin{table}[h!]
\begin{tabular}{|c|l|l|}
\hline
\rowcolor[HTML]{C0C0C0}
\multicolumn{2}{|c|}{\cellcolor[HTML]{C0C0C0}{\color[HTML]{000000} \textbf{Xena-tester}}} & \multicolumn{1}{c|}{\cellcolor[HTML]{C0C0C0}{\color[HTML]{000000} \textbf{WR Switch}}} \\ \hline
\rowcolor[HTML]{C0C0C0}
{\color[HTML]{000000} \textbf{Module}} & \multicolumn{1}{c|}{\cellcolor[HTML]{C0C0C0}{\color[HTML]{000000} \textbf{Port}}} & \multicolumn{1}{c|}{\cellcolor[HTML]{C0C0C0}{\color[HTML]{000000} \textbf{Port}}} \\ \hline
0 & 0 & 1 \\ \hline
0 & 1 & 18 \\ \hline
0 & 2 & 2 \\ \hline
0 & 3 & 17 \\ \hline
\end{tabular}
\end{table}
\begin{note}
Be careful! Ports on Xena-tester are starting form 0 but from 1 in a WR switch.
\end{note}
The other ports of the WR switch must be connected as described in the \hrefreleasetests document.
Connecting the WR switch to an other module or other ports of the Xena-tester will imply to change all test cases configurations. It is not recommended.
%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%
\subsection{Software environment}
%%%%%%
\subsubsection{Xena application}
To run these tests, the Xena application \textbf{XenaManager-2G} is used. This application is only running under Windows.
Usually, settings of \textbf{XenaManager-2G} are stored in 'Documents/Xena/XenaManager-2G' on your local disk. Copy at this location
the file \href{\gitrepo/xena-tester/xena-tester-configs/xena.xmcfg}{xena.xmcfg} and the
full directory \href{\gitrepo/xena-tester/xena-tester-configs/testcases}{testcases}.
You are ready now to start the application \textbf{XenaManager-2G}.\\
To run a test, the following steps must be followed :
\begin{itemize}
\item Check that the chassis is well connected.
\item Reserve ports 0 to 3 on module 0
\item Select the testbed to execute in the 'Available Ressources' window.\\
\includegraphics[scale=1]{AvalaibleRessources.eps}
\item In 'Operations' menu, load the test case. Select the file with the same name appearing in 'Available Ressources' window
you selected. Click 'yes' to any popup windows that might appear. \\
\includegraphics[scale=1.5]{LoadTestCase.eps}
\item Configure the WR switch according to the test to execute.
See 'WR switch configuration' \ref{chap:WR switch configuration} section.
\end{itemize}
You are ready to start testing. The rest of the procedure for testing will be explained in next chapters because
it depends on the test to realize.
%%%%%%
\subsubsection{WR switch configuration}
\label{chap:WR switch configuration}
Each test need a specific configuration of the WR switch as decribed in the \hrefreleasetests document.
To simplify the execution of a test, all switch configurations are already prepared and a bash tool can be use to apply a configuration in the switch.\\
To use it, checkout first the git directory \href{\gitrepo/xena-tester/switch-configs}{switch-configs} on your local machine.\\
Run then the tool 'run.sh'. A menu will appear and invite you to select the configuration to be applied. \\
\includegraphics[width=17cm]{RunTest.eps}
Applying a switch configuration takes few seconds because a reboot is needed and in some configuration PPSi must be stopped.
The following message 'Type enter to continue' indicates that the configuration has been applied. \\
\includegraphics[width=17cm]{RunTestDone.eps}
Your switch is not ready yet. As few ports of the switch implements the 'Low Phase Drift Calibration', the calibration of the ports can take some time.
So, before to start a test, verify with the 'wr{\_}mon' tool that all ports are up (green color). \\
\includegraphics[width=17cm]{wrmon.eps}
You are now ready to run your test.
\begin{note}
The 'run.sh' script contains the name of the WR switch to configure. Currently it is set to 'ctdwa-774-cxnatest1'.
It can be changed by running the following command:
\codebox{
TARGET="switch{\_}name" run.sh
}
\end{note}
%%%%%%
\subsubsection{Data mining}
\label{chap:Data mining}
Some tests uses the 'Statistics Logging' facility in the \textbf{XenaManager-2G} application.
This facility logs statistics in a file during the execution of a test using the '.csv' format.
At the end of a test, a log file can reach a size of a few MB. Analysing these data can be
very complex. \\
So log files must be purged to keep only interesting informations and add test configurations.
The bash script \href{\gitrepo/xena-tester/tools/logCleaner.sh}{logCleaner.sh} makes this job
and can be apply on all tests executing the following test pattern:
\begin{table}[ht!]
\begin{tabular}{ l l l l}
FRAME & increment size [bytes] & A-B with step of N \\
& loads [\%] & l1, l2, l3, \ldots \\
\end{tabular}
\end{table}
The script is able to retreive all peer [frame size, load] tested and produce a new log file
keeping only results of each tests and adding the frame size and the load.\\
\includegraphics[width=17cm]{cleanLog.eps}
This script expects to find a file 'config.txt' in the directory from where it is called.
This file contains the name of the log file to analyse and the name of the log file to produce.\\
Contents of 'config.txt' file :
\codebox{%
logName=log{\_}3.1.1.csv \\
cleanLogName=cleanlog{\_}3.1.1.csv \\
dataCollectionName=Forwarding performance: one-to-one ...
}
Before to execute the 'logCleaner.sh' script it is recommended to purge manually the log file from :
\begin{itemize}
\item repeated lines at end of the log. \\
As the logging system cannot be controlled by the scheduler, it continue to log uninteresting data
at the end of the log file. These data can be easily removed using Calc in openOffice suite. It can
reduce significantly the size of the log file.
\item 'learning' frames at the beginning of the log.\\
At the beginning of a test, few frames are sent in order to leave the switch the time to learn the MAC adresses.
\end{itemize}
For tests which are not following this pattern for testing, specific log cleaner scripts 'spLogCleaner.sh' are provided.
Once the `clean log` is available, generation of data for the test results document can be started.
A set of few bash script are available to generate these data :
\begin{itemize}
\item 'genData.sh': This script tool generates few files with the extenxion '.dat' containing various extracted
data from the clean log file.
\item 'genGraph.sh': From '*.dat' files, it generates all pictures included in the document. This script uses
the gnuplot tool to do that.
\item 'genTable.sh': From '*.dat' files, it generates all tables included in the appendix.
\item 'genAll.sh': This script tool calls the previous scripts in the following order: 'genData.sh', 'genGraph.sh' and 'genTable.sh'.
\end{itemize}
%%%%%%
\subsubsection{Test report document}
\label{chap:Test report document}
The test report document must follow this directory structure:\\
\begin{table}[H]
\begin{tabular}{|l|}
\hline
\begin{lstlisting}
testReport
|
|-> SwitchVersion (e.g. 6.0.0)
|
|-> TestNumber (e.g. 3.1.1)
: |
: |-> data
\end{lstlisting} \\
\hline
\end{tabular}
\end{table}
The current structure in the testReport directory is represented in the following figure:
\includegraphics[width=3cm]{tree.eps}
All log files acquired during tests must be copy in the data directory corresponding to the test number.
All scripts for cleaning log files, generating data , pictures and tables must be executed in these data directory.
All resulting data needed by the document will be pushed in the directoty above (corresponding to the test number).\\
In the directory corresponding to the switch version, just run 'make clean all' to generate the report in pdf format.
sw/xena-tester/running-tests/tests-running.tex 0 → 100644
View file @ ade56d62
%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%
\section {Running tests}
\label{chap:running-tests}
%%%%%%%
\subsection{Forwarding functional tests}
\label{chap:fwd-functional}
These tests does not require to generate log files.\\
For each test, we have two complementary ways to check if a test is passing or not. We can first use
facilities available in the \textbf{XenaManager-2G} application :
\begin{itemize}
\item the 'Global Statistics' feature: give an overview of found errors, received frames, \ldots
\item the 'Capture feature': capture received frames and allow to analyze the different layers in a frame.
\end{itemize}
The second tool that can be use is 'wrs{\_}pstats -t' which must be launched in the tested WR switch. It gives an
overview of all received and transmitted frames on all ports. Compared to the \textbf{XenaManager-2G} application,
this tool gives a view of what's happening on the other switch ports (3-16).
%%%%%%%
\subsection{Forwarding performance tests}
\label{chap:fwd-performance}
These tests need to use `Scheduler` and the `Statistics Logging` features of the \textbf{XenaManager-2G} application.
The scheduler is automatically loaded when you load a test case. \\
To start a test you have to :
\begin{itemize}
\item start first the `Statistics Logging`
\item start secondly the `Scheduler`.
\end{itemize}
When the scheduler has finished is execution and stopped, you can then stop manually the `Statistics Logging`
and copy the log file into the associated test data directory.
Do not forget to rename the log file to fit the name declared in the 'config.txt' file relative to the test.
\newline
To execute these tests, the following scheduler configurations are used :
\newcolumntype{L}{>{\centering\arraybackslash}m{3cm}}
\begin{table}[h]
\begin{tabular}{lll}
\hline
\textbf{sched\_fp\_incr\_63\_s1} & Frame length: & 64 with step of 63 and 1522 \\
& Load (\%): & 10 30 50 70 80 85 90 92 95 97 99 100 \\
& Frame count: & \textgreater4000000\\
\hline
\textbf{sched\_fp\_incr\_63\_s3} & \multicolumn{2}{l}{Same as previous pattern but manage 3 streams per port}\\
\hline
\textbf{sched\_fp\_incr\_63\_s1\_nl} & \multicolumn{2}{l}{Same as previous pattern but without learning process}\\
\hline
\textbf{sched\_fp\_incr\_fixed\_imb\_s1} & Frame length: & 64 65 700 701 1521 1522 \\
& Load (\%): & 50 80 95 100 \\
& Burst duration: & 5mn \\
& \multicolumn{2}{{>{\arraybackslash}m{10cm}}}
{This pattern is played on port 0 and 1 with all possible combinaisons on port 2 and 3}\\
\hline
\end{tabular}
\end{table}
\break
These scheduler configurations are generated from \href{\gitrepo/xena-tester/tools}{bash scripts}:
\begin{table}[ht]
\begin{tabular}{|l|l|}
\rowcolor[HTML]{9B9B9B}
\hline
\textbf{Scheduler configuration} & \textbf{Command} \\
\hline
\textbf{sched\_fp\_incr\_63\_s1.xsch} & sched\_fp\_incr\_63\_s1.sh -s 1 \\
\hline
\textbf{sched\_fp\_incr\_63\_s1\_nl.xsch} & sched\_fp\_incr\_63\_s1.sh -s 1 -l\\
\hline
\textbf{sched\_fp\_incr\_63\_s3.xsch} & sched\_fp\_incr\_63\_s1.sh -s 3 \\
\hline \textbf{sched\_fp\_incr\_fixed\_imb\_s1.xsch} & sched\_fp\_incr\_fixed\_imb\_s1.sh -s 1 \\
\hline
\end{tabular}
\end{table}
These bash scripts can be changed to modify patterns to play. Attention, that any modification
of the pattern in \textbf{sched\_fp\_incr\_fixed\_imb\_s1} configuration, will involve an adjustment of the
specific clean logger used to decrypt the data.
Regarding tests to execute, the following table gives an overview of all scheduler configuration used :
\begin{table}[ht]
\small
\begin{tabular}{|l|L|}
\rowcolor[HTML]{9B9B9B}
\hline
\textbf{Scheduler} & \textbf{Apply to tests} \\
\hline
sched\_fp\_incr\_63\_s1 & 3.1.1, 3.1.2, 3.1.4, 3.1.5, 3.2.1\\
\hline
sched\_fp\_incr\_63\_s1\_nl & 3.3.1, 3.3.2\\
\hline
sched\_fp\_incr\_63\_s3 & 3.1.3 \\
\hline
sched\_fp\_incr\_fixed\_imb\_s1 & 3.2.2 \\
\hline
\end{tabular}
\end{table}
Markdown is supported
0% or
You are about to add 0 people to the discussion. Proceed with caution.
Finish editing this message first!
Please register or to comment