diff --git a/doc/spec-sw.in b/doc/spec-sw.in index 503e0bf93652d098b2e9b68aa5eaca8d2ea73504..14201a45bbc4f7b66bad5fa49127a91478a720d6 100644 --- a/doc/spec-sw.in +++ b/doc/spec-sw.in @@ -62,47 +62,33 @@ associated software project, hosted at @url{http://www.ohwr.org/projects/spec-sw}, whose @i{git} repository hosts the latest version. -@b{Warning}: this manual documents the setup we used for the first -FMC mezzanine driver (the @i{fine-delay-sw} package, version 1.1) with -a few additions by Tomasz Wlostowski in the tool set. - -This is the latest release of the package using the abstractions -described here. This @i{spec-sw} package is being completely -redesigned to take advantage of the Linux bus abstraction. The current -development is on the @i{fmc-bus} branch of the git repository, but it -will be merged immediately after this commit. - -@menu -* The Core Driver:: -@end menu - @c ########################################################################## -@node The Core Driver -@chapter The Core Driver +@node History and Overview +@chapter History and Overview -The core driver lives in the @i{kernel} subdirectory. Like the board -itself, it is a carrier for sub-modules. +This driver is pretty different from the initial implementation, such as +the one used by @i{fine-delay-sw-v1.1}. If you use such version, please +compile the manual you find in your source code repository. -@menu -* Compiling the Driver:: -* Driver Operation:: -* Future Plans:: -@end menu +The package currently includes both the @i{spec} driver and the +@i{fmc-core} driver (which benefits from its own documentation file +which you may want to read. -@c ========================================================================== -@node Compiling the Driver -@section Compiling the Drivers +@c ########################################################################## +@node Compiling the Drivers +@chapter Compiling the Drivers - To compile -it, set the following variables in your environment: +The kernel modules part of this package live in the @i{kernel} +subdirectory. To compile them, you need to +set the following variables in your environment: @table @code @item LINUX The top directory of the kernel sources for the version you - are going to run the driver under. I'm testing mostly with 3.2, - but I at least compile-test back to 2.6.35. + are going to run the driver under. I'm testing mostly with 3.4, + but this version compiles against Linux-2.6.30 and later ones. @item CROSS_COMPILE @@ -132,205 +118,93 @@ For example, if your target computer's filesystem is mounted under @end example The modules are installed under the subdirectory @code{extra}. In -the previous case your driver will end up in +the previous case your driver will end up being installed +(together with the other modules) as @example /mnt/target/lib/modules/3.2.0/extra/spec.ko @end example -@c ========================================================================== -@node Driver Operation -@section Driver Operation +@c ########################################################################## +@node Role of spec.ko +@chapter Role of spec.ko -The driver @code{spec.ko} will simply register itself as a PCI driver, +The @code{spec.ko} driver depends on @code{fmc-core.ko}, that must +be loaded first (if you don't rely on automatic dependencies). + +The driver @code{spec.ko} registers itself as a PCI driver, using both the ``old'' vendor and device ID (the Gennum identifiers) and the new ones (CERN vendor and SPEC device). -The kernel will call the @i{probe} method of the driver, which in -turn perform the following actions: +For each new SPEC device found on the system, the driver performs the +following steps: @itemize @bullet -@item Program a @i{firmware} (or @i{gateware}) file in the FPGA -@item Load program code for the internal CPU (if any and if requested) -@item Load a sub-module for driving the FMC card (if any). +@item It enables MSI interrupts, the only kind supported in this package. +@item It loads the @code{spec-init.bin} ``golden'' gateware file. +@item It checks that the content of the binary is as expected (using + a minimal @i{sdb}-based verification). +@item It reads the whole I2C EEPROM found on the mezzanine. +@item It allocates an @i{fmc_device} structure and registers as + a nee device in the @i{fmc} bus. @end itemize -If the first step fails, nothing more is tried; the second and third -steps are expected to fail (you may not run a CPU in the FPGA or you -may have its program already included in the @i{gateware} file); -similarly you may not need a sub-driver, because the card is -stand-alone and needs no Linux driver. - -Please note that the CPU program file is not loaded by default any -more: you must explicitely request it to be loaded with a module -parameter. If you pass ``lm32=1'' as a parameter, it will try to load -at the default address (0x80000). IF you you pass a different value -than 1, it will be used as the load address (for example: -``@code{insmod lm32=0xc0000}''). - -@b{Note:} all three files are looked-for in @i{/lib/firmware/fmc}, -because our own installations share firmware for COTS peripherals -but mount a host-specific NFS subdirectory. This is incompatible -with previous versions of this package, but the error message should -be clear enough for our users to sort the problem out. (Actually, I -tried to add a backward-compatible second choice, but it would have -messed up the code quite a lot). - -The three files being looked for are named like this, assuming here -the card is seen under PCIe bus number 2: - -@table @code - -@item fmc/spec-B0002.bin - - This is the name of the @i{gateware} file, which is looked-for - in @code{/lib/firmware} by the @i{hotplug} or @i{udev} program. - -@item fmc/spec-B0002-cpu.bin - - This is the program code for the embedded CPU. Again, it must - live in @code{/lib/firmware}. - -@item fmc/spec-B0002.ko - - The kernel sub-module. It is loaded using @i{modprobe}, so - it should live in the same place where @code{spec.ko} is found. - Also, please run ``@code{depmod -a}'' after copying or renaming - the file. - -@end table - -The central part of actual names of the three files is -customizable. You can use the @code{name=} module parameter to change -it. The default value, ``@code{%b}'' expands to the bus number, as -shown. This allows to differentiate between cards, and have different -bitstreams and sub-drivers for each of them, according to the -geographical placement in the host computer. - -The names being passed are using literally, with the exception -of the following three formats: +Failure of any of those steps is fatal. -@table @code - -@item %b - - The bus number, expanded as @code{B%04x"}. - -@item %s - - The @i{slot} and @i{function} composite number, expanded as - @code{S%04x}. Note that usually every PCIe mechanical slot is - seen as a separate bus number, so using @code{%s} is not generally - useful. - -@item %% - - A literal @code{%}. - -@end table +Any mezzanine-specific action must be performed by the driver for the +specific FMC card, including reprograming the FPGA with the final +gateware file. Similarly, the @i{spec} driver is not concerned with +programming the LM32 image, when it makes sense to. This is different +from the role splitting in previous versions of the driver. -For example, I have two cards, fit in bus number 2 and 3. I prepared -the files for the card in slot 3 but not for the one in slot 2; I -receive the following messages when loading with the default name -(@code{%b}). Here the @code{0} and @code{-2} items are error codes (0 -means success): - -@smallexample - spec_probe (device 0002:0000) - spec 0000:02:00.0: Can't load firwmare "spec-B0002.bin" - -2 - spec_probe (device 0003:0000) - spec_load_fpga: got binary file "spec-B0003.bin", 1484404 (0x16a674) bytes - spec_load_lm32: got program file "spec-B0003-cpu.bin", 49044 (0xbf94) bytes - LM32 has been restarted - spec_load_submodule: load "spec-B0003": 0 -@end smallexample - -The following messages are printed if I pass @code{name=pippo} (Italian for -@i{foo}): - -@smallexample - spec_probe (device 0002:0000) - spec 0000:02:00.0: Can't load firwmare "spec-pippo.bin" - -2 - spec_probe (device 0003:0000) - spec 0000:03:00.0: Can't load firwmare "spec-pippo.bin" - -2 -@end smallexample - -Please note that when a sub-module is loaded, it registers in the kernel -with its real name, not the one you renamed it to. For example, -the output of @i{lsmod} in the former case shown above is as follows, -even if the module has been loaded as @code{spec-B0003.ko}: +@b{Note:} the gateware binary is looked-for in @i{/lib/firmware/fmc}, +which is where all fmc-related external files are expected to live. +That's because our own installations share firmware for COTS peripherals +but mount a host-specific NFS subdirectory. -@example - Module Size Used by - spec 3835 0 - spec_fine_delay 421 0 -@end example - - -@c ========================================================================== -@node How sub-modules work -@section How sub-modules work - -@b{Warning:} this is still in flush, and I'm open to suggestions. The -current implementation is really hackish and has a number of weak -points if several boards are in use. +Please refer to the @i{fmc-bus} document for details about the overall +design of the interactions of carriers and mezzanines. -When a sub-module is loaded, it can access the list of PCI devices -managed by the spec. Such a list is really list, in the Linux sense. - -The sub-module is expected to identify its own cards and act -accordingly. The current implementation trivial, and is on show in -@code{spec-wr-nic.o}. I won't document here because I hope it will -change soon. - -This are the outstanding bugs that are begging to be fixed: - -@table @i -@item The sub-module is loaded only once +@b{Warning:} currently the @i{match} function of the bus always +returns success: the mezzanines I currently have for testing have no +ID records written in their internal EEPROM, so I'm not able to setup +the associated data structures and code. For this reason there is no +@i{module_alias} support nor autoprobe of drivers: any @i{fmc} driver +you load will drive all SPEC cards found on the system. - If two cards request the same module, one only is loaded when - the first card is probed, and it may find an incomplete list - of cards. Adding a delay is a hack I won't do. - -@item Renaming doesn't really work - - Modules register by name (the real name, not the file name), - so even if the module's file name is changed it won't re-load - (you get ``@code{File exists}'' error). - -@item Sub-modules are not told which spec to use - - Even if there's one sub-module for each spec, you won't know - what card to use, because the module's own name may be different - from the file name. Even reading @code{dev->names[SPEC_NAME_SUBMOD]} - won't help - -@end table - -But this is the current status, and it works perfectly with one card -and one sub-module. I'll think more about it -- again, suggestions -are welcome. - -@c ========================================================================== -@node Future Plans -@section Future Plans +@c ########################################################################## +@node fmc-trivial.ko +@chapter fmc-trivial.ko -The SPEC driver should scan its own wishbone bus according to the -self-description specification, that is being finalized these times, -instead of using static file names. +The simple module @i{fmc-trivial} is just a simple client that +registers an interrupt handler. I use it to verify the basic mechanism +of the FMC bus and how interrupts work. It's worth reading, but it's not +worth describing here. +@c ########################################################################## +@node fmc-write-eeprom.ko +@chapter fmc-write-eeprom.ko +This module, not yet written, will be able to load a binary file from +@i{/lib/firmware/fmc} and write it to the internal EEPROM of the mezzanine. +Partial writing is allowed by filling a simple TLV array of binary areas. @c ########################################################################## @node The WR-NIC @chapter The WR-NIC -The @i{wr-nic} is going to be part of this package, but it is not as of this -revision. +The @i{wr-nic} driver is being worked on. It should be a model of how +FMC drivers should behave, so it is hosted in this package. + +The driver will register an Network Interface Card on the host computer, +to allow exchanging data on the same fiber channel where White-Rabbit +synchronization is taking place. The driver relies on the 5-channel +digital I/O mezzanine to time-stamp incoming pulses and generate +precisely timed output signals. @c ########################################################################## -@node Tools -@chapter Tools +@node User-Space Tools +@chapter User-Space Tools The @i{tools} subdirectory of this package includes a few host-side programs that may be useful while working with the SPEC device. @@ -397,72 +271,6 @@ The tools currently available are: @end table -Use of some of these programs is exemplified in @ref{Troubleshooting}. - - -@c ########################################################################## -@node Troubleshooting -@chapter Troubleshooting - -When loading several binary files, sometimes there's something that -goes wrong. This section includes a few suggestion to identify what -step may have gone wrong. - -First of all, you may want to check whether the FPGA binary has been -properly loaded and stuff is running inside it. One test that -can be made is whether the LM32 soft-core is actually able to run -programs. - -To this aim, you can compile and run the @code{spec-test00-cpu} -program in the @code{test-lm32} directory of this package. - -@c FIXME: load a default firmware binary -The strange name of the program is due to my plan to allow -``@code{insmod spec name=test00}'' in order to run this test and later -other ones. - -Currently, the LM32 programs can be re-loaded to a running FPGA using -@i{spec-cl} (cpu loader), described in @ref{Tools}. Thus, to load -the test program, you can run this command: - -@example - spec-cl spec-test00-cpu.bin -@end example - -The loader still sometimes fails (usually the first time), with a -message like ``@code{programming error at 0 (expected 98000000, found -0000c56a)}''. Please retry: the second time it works (this has to -be fixed, I apologize meanwhile). - -The test program si simply incrementing an integer variable over -and over. The variable lives at address 0x8000 in LM32 space, which -is address 0x88000 in SPEC space. Thus, you can test it like this: - -@example - # spec-cl spec-test00-cpu.bin; specmem 88000; sleep 1; specmem 88000 - 00037cda - 0025bfa2 -@end example - -If you @code{export VERBOSE=1} beforehand you'll get a little more -information from both programs (please note that any value in VERBOSE -will work, the program only checks the variable exists): - -@example - # spec-cl spec-test00-cpu.bin; specmem 88000; sleep 1; specmem 88000 - spec-cl: Wrote 1700 bytes at offset 0x8000 - 00088000 == 0003872c - 00088000 == 0025c8e0 -@end example - -As you see a running LM32 core increments its memory location slightly -more than 2.2million times per second (0x220000). If you can verify -this, it means the FPGA is properly programmed and the CPU core is -running. - -More troubleshooting tests will be added. - - @c ########################################################################## @bye