Skip to content

S
SDB - Self-describing Bus
Commit c9aa3651 authored by Alessandro Rubini's avatar Alessandro Rubini
Browse files
Options

specification: removed interrupt section, minor fixes 

The interrupt discussion has been moved to the projects' wiki.
Besides that, some typos, external references and nothing else.
And today is not yesterday: new date is there.
Signed-off-by: Alessandro Rubini's avatarAlessandro Rubini <rubini@gnudd.com>
parent 3958079a
Show whitespace changes
Inline Side-by-side
Showing with 29 additions and 162 deletions
docs/specification/sdb.tex
View file @ c9aa3651
......@@ -35,7 +35,7 @@ for Logic Cores -- proposed Version 1.1}
\author{Alessandro Rubini (Consultant for CERN)\\
Wesley Terpstra (GSI),\\
Manohar Vanga (CERN, BE/CO/HT)}
\date{March 27th 2013}
\date{March 28th 2013}
\begin{document}
\maketitle
......@@ -144,7 +144,7 @@ have already been used.
All multi-byte values are stored in \textit{big endian} order. We need
a well-defined endianness to allow generic scanning of the target bus;
we picked bigendian because most embedded devices are big endian and
we picked big-endian because most embedded devices are big endian and
because it is the format usually chosen by existing standards documents.
All data structures are 64 byte in size and they are all similar in their
......@@ -181,8 +181,7 @@ computer or network.
\subsection{Interrupts}
This version of SDB doesn't describe interrupts. Lack of interrupt
description is a design choice (that may change in the future,
though), and here we try to explain it.
description is a design choice.
SDB was born as a description of address spaces. Access to address
spaces is usually done through \textit{bus} signals, according to the
......@@ -192,162 +191,24 @@ strictly-speaking are not part of the bus: they are a sort of a
``interrupt controller'' core. The same applies to the out-of-band
DMA channels used in some designs, where two devices are connected via
a path not part of the normal bus interconnect fabric.
In that case the DMA controller can act as a master on behalf of the slave.
We currently don't offer such descriptions because we really think
designs should be converted to the concept of MSI interrupts, and
this section summarizes the reasoning.
% From now on this is Wesley's message to the list, only slightly edited.
A bus lets you control slave
devices from master devices. This communication is always initiated by
the master, a request-response messaging pattern. Then one day you have
a slave device which needs to initiate communication.
The traditional answer has been interrupt lines, that
let you reuse the bus lines. Your
slave card just needs to "wake-up" the master, who can then use the
usual bus lines to find out what's the matter and take appropriate
action. So, for the cost of a single line, they enable two-way
communication.
Unfortunately, this simple solution comes at a significant cost. The
core problem is that you've actually introduced a completely new (albeit
superficially simple) bus to your system. All of the following problems
stem from this change:
\begin{itemize}
\item like in a normal bus protocol, an interrupt needs to be
acknowledged. This is typically done by writing some register on the
slave device which causes it lower the interrupt line when you've dealt
with all the things that needed processing;
\item like in a normal bus protocol, you need flow control. Maybe the
master is busy and wants to deal with some critical section code before
it gets around to dealing with your interrupt. This is why interrupt
systems need enable flags, mask registers, and so on;
\item like in a normal bus protocol, you need addressing. The master
should be able to tell what the interrupt was about. Usually you use
multiple interrupt lines, a one-hot addressing scheme. Interrupt 1 means
NIC is ready and interrupt 2 means the printer is burning;
\item like in a normal bus protocol, you need routing/arbitration.
With interrupts, the master/slave relationship is inverted. Here the
slaves send requests to the masters. So you need to decide which master
receives the interrupt from a given slave. Unlike the preceding points,
this doesn't really have a standard solution. Every system is different.
If you have one master, it's simple. Once you have multiple masters you
need to decide how the interrupts are wired to the masters. Often you
need this to be reconfigurable at runtime. For example, the NIC might be
handled by the LM32 in-chip or the host CPU off-chip;
\item like in a normal bus protocol, you need a description language.
When your software starts up it needs to determine which slave is
connected to which master
\item you lose plugability with the original bus protocol: interrupts
are a completely distinct bus and they cannot be plugged together with
devices that just use the vanilla bus protocol.
\end{itemize}
Recently, the PCI SIG decided to obsolete "legacy interrupts" and
replace them with "message-signalled interrupts". MSI is a procedure for
using the existing PCIe bus for sending interrupts. It essentially says:
if you need a slave to talk to a master just send requests as a
master yourself.
This approach has a few advantages: it allows an unlimited
number of interrupt lines (addressing), without the need to share lines. It
makes possible for all participants on the PCIe bus to receive
interrupts if they desired (routing). There is a performance
benefit since it requires less messaging than legacy interrupts
(acknowledgement). Indeed, now that PCIe uses a message-oriented
protocol, the pin count advantage of legacy interrupts is gone.
Finally, legacy interrupts can be implemented on top of the
MSI scheme.
FPGA designers face exactly the same decision that the PCI SIG faced:
they are designing a SoC. Pin count is not an issue because synthesis
eliminates any pins that are not used, and inside the chip there is plenty of
freedom.
In the core that is expected to handle the ``interrupt'',
we can add a bus slave interface instead of a "raise
interrupt" pin. At first this may seem just a clever hack that
saves some time. However, we realized this brings in more and more
advantages:
\begin{itemize}
\item acknowledgements are handled just like normal bus acknowledgment.
For Wishbone, for example, when the host
system is done processing the interrupt, it raises the ack line. This
means interrupts are discrete, like in MSI. You can say "I have 5 things
for you to do" and not have to squish them into a "I raise this line
until you do everything I want". This means you can choose to handle
some of events that generated interrupts, but delay processing the
others. If the master stops handling interrupts, acks stop flowing
and no code is needed.
\item there's no need for special interrupt masking etc. If the
master doesn't
want to process an interrupt right now, it just lets the stall line go high.
From the software side, this requires nothing special at all. If the master
is not reading, the queue fills and the line goes high. Again no node
is needed;
\item there is an unlimited number of interrupts. Each address+data
pair can be interpreted differently;
\item nothing special at all is needed to route interrupts. They
are just another master on the crossbar. If a master wants to receive
interrupts from a particular slave, it just writes its address to a
register on the device. When that slave generates an interrupt, it is a
write to the specified address and the master gets the interrupt. This completely
solves the problem of determining which of multiple slaves
raises which interrupts;
\item by using the same protocol for interrupts as for the normal bus,
modularity is improved. For example we can make software bus slaves and
hardware masters can read/write
memory from software programs in userspace on the host system;
\item since interrupts are generated by bus writes, they are compatible
with a remote protocol like Etherbone or other ones.
You can remotely trigger an interrupt on any device in the
network.
\end{itemize}
The only argument we can see in favour of legacy interrupts on a SoC is
that they are "simpler". A slave just needs to write a '1' or a '0' to
an output as opposed to adding a bus master interface. However,
this ``simplicity'' argument ignores the costs that happen once you have
enough of these devices, once the
interrupt bus becomes non-trivial. To keep the code simple, one could
easily imagine a small HDL component that takes an address register and
a generates a write upon request.
To use MSI and achieve compatibility with a legacy master like
a soft-core with an ``interrupt'' input line, you just need an
"interrupt unit" which is a bus slave that raises different pins when it
is written to at different offsets. In other words, just like in PCIe,
implement legacy interrupts using the MSI approach is pretty simple.
Neither of these is strictly in the scope of SDB; but especially as
far as interrupts are concerned, we really think FPGA designs should
be converted to the concept of MSI interrupts. Message-signalled
interrupts have a number of advantages over legacy interrupts, and
don't even need any description besides what SDB already offers.
A more complete discussion of this is in the wiki page of the project:
% This ends the text Wesley sent to the list.
To summarize: interrupt lines are not strictly in the scope of SDB and
we currently don't offer such description because we really think FPGA
designs should be converted to the concept of MSI interrupts -- which need
no description support within SDB as a side effect.
\texttt{http://www.ohwr.org/projects/fpga-config-space/wiki}.
On the other hand, it may make sense to define SDB structures to
describe this special wiring, to help existing ``legacy'' projects to
benefit from SDB and avoid doping the software source code with static
information about device wiring. Maybe future releases of this specification
will allow description of legacy interrupts.
describe this special wiring, when it exists, to help existing
``legacy'' projects to benefit from SDB and avoid doping the software
source code with static information about device wiring. Thus,
it may happen that future
releases of this specification include description of legacy
interrupts.
\subsection{The Overall System Structure}
......@@ -509,7 +370,7 @@ Another issue is that the device drivers may either be concerned with the FPGA
binary as a whole or be interested in each and every individual logic block;
thus, the same
GPIO logic block can be either driven by the host or ignored by it -- because
is is directly used by the soft-core within the synthesized binary.
is is directly used by the soft-core within the synthesised binary.
\subsection{Wb-core and Enumeration}
......@@ -637,7 +498,7 @@ soft-core CPU with very little overhead.
Use cases for such a storage file-system are almost read-only, but not completely:
sometimes users need to update FPGA binary images or some
calibration parameters. Such use doesn't require a complex
filesystem with wear-leveling capabilities.
filesystem with wear-levelling capabilities.
In our search for the state of the art, we didn't find small and
simple filesystems that allow in-place replacement of files. We thus
......@@ -662,7 +523,7 @@ then register a filesystem that only spans the needed part of the
flash, as defined at manufacture time (or by device-specific
software).
A user-space program creates the SDB filesytem as an image file, it
A user-space program creates the SDB filesystem as an image file, it
writes it using normal char-device MTD operations or other tools
to move binary data. The filesystem can then be accessed on the storage
medium by either user code or kernel code. Or by the microcontroller
......@@ -931,7 +792,7 @@ number.
\item[sdb\_records] \hfill \\
This field specifies the number of records in the table. It \textbf{must} include
this very record in the count, and the whole address range (this number multplied by 64 bytes)
this very record in the count, and the whole address range (this number multiplied by 64 bytes)
\textbf{must} be accessible. Note that the array \textbf{may} include empty records at any position.
\item[sdb\_version] \hfill \\
......@@ -1379,13 +1240,19 @@ byte-swap each 32-bit word before using the structure in a
little-endian host. After such swapping, the data fields live at the correct
offsets and must be accessed as big endian.
\subsection{Existing Tools}
\subsection{References}
As part of the \textit{Etherbone} project, you already find a number
The \textit{Etherbone} project, is an early adopter of SDB;
it includes also a number
of tools that work with SDB structures (including \texttt{eb-ls} that
printed the table of devices shown above). The project is at
\texttt{http://www.ohwr.org/projects/etherbone-core} .
The \textit{sdbfs} work started as a separate project but it is now
part of the \texttt{fpga-config-space} project. It is currently work
in progress, but the \textit{gensdbfs} and other non-kernel parts are already
being used in production.
\end{document}
% LocalWords: metadata FPGA gateware CERN
% LocalWords: metadata FPGA gateware CERN endian etherbone filesystem
Markdown is supported
0% or
You are about to add 0 people to the discussion. Proceed with caution.
Finish editing this message first!
Please register or to comment