From 73103a72bf142bb28eb8fd72a9b47ca8aba1f781 Mon Sep 17 00:00:00 2001
From: Alessandro Rubini <rubini@gnudd.com>
Date: Thu, 28 Mar 2013 13:39:36 +0100
Subject: [PATCH] specification: removed interrupt section, minor fixes
The interrupt discussion has been moved to the projects' wiki.
Besides that, some typos, external references and nothing else.
And today is not yesterday: new date is there.
Signed-off-by: Alessandro Rubini <rubini@gnudd.com>
---
docs/specification/sdb.tex | 191 ++++++-------------------------------
1 file changed, 29 insertions(+), 162 deletions(-)
diff --git a/docs/specification/sdb.tex b/docs/specification/sdb.tex
index 6193fa3..7883bbc 100644
--- a/docs/specification/sdb.tex
+++ b/docs/specification/sdb.tex
@@ -35,7 +35,7 @@ for Logic Cores -- proposed Version 1.1}
\author{Alessandro Rubini (Consultant for CERN)\\
Wesley Terpstra (GSI),\\
Manohar Vanga (CERN, BE/CO/HT)}
-\date{March 27th 2013}
+\date{March 28th 2013}
\begin{document}
\maketitle
@@ -144,7 +144,7 @@ have already been used.
All multi-byte values are stored in \textit{big endian} order. We need
a well-defined endianness to allow generic scanning of the target bus;
-we picked bigendian because most embedded devices are big endian and
+we picked big-endian because most embedded devices are big endian and
because it is the format usually chosen by existing standards documents.
All data structures are 64 byte in size and they are all similar in their
@@ -181,8 +181,7 @@ computer or network.
\subsection{Interrupts}
This version of SDB doesn't describe interrupts. Lack of interrupt
-description is a design choice (that may change in the future,
-though), and here we try to explain it.
+description is a design choice.
SDB was born as a description of address spaces. Access to address
spaces is usually done through \textit{bus} signals, according to the
@@ -192,162 +191,24 @@ strictly-speaking are not part of the bus: they are a sort of a
``interrupt controller'' core. The same applies to the out-of-band
DMA channels used in some designs, where two devices are connected via
a path not part of the normal bus interconnect fabric.
-In that case the DMA controller can act as a master on behalf of the slave.
-
-We currently don't offer such descriptions because we really think
-designs should be converted to the concept of MSI interrupts, and
-this section summarizes the reasoning.
-
-% From now on this is Wesley's message to the list, only slightly edited.
-
-A bus lets you control slave
-devices from master devices. This communication is always initiated by
-the master, a request-response messaging pattern. Then one day you have
-a slave device which needs to initiate communication.
-The traditional answer has been interrupt lines, that
-let you reuse the bus lines. Your
-slave card just needs to "wake-up" the master, who can then use the
-usual bus lines to find out what's the matter and take appropriate
-action. So, for the cost of a single line, they enable two-way
-communication.
-
-Unfortunately, this simple solution comes at a significant cost. The
-core problem is that you've actually introduced a completely new (albeit
-superficially simple) bus to your system. All of the following problems
-stem from this change:
-\begin{itemize}
-\item like in a normal bus protocol, an interrupt needs to be
-acknowledged. This is typically done by writing some register on the
-slave device which causes it lower the interrupt line when you've dealt
-with all the things that needed processing;
-
-\item like in a normal bus protocol, you need flow control. Maybe the
-master is busy and wants to deal with some critical section code before
-it gets around to dealing with your interrupt. This is why interrupt
-systems need enable flags, mask registers, and so on;
-
-\item like in a normal bus protocol, you need addressing. The master
-should be able to tell what the interrupt was about. Usually you use
-multiple interrupt lines, a one-hot addressing scheme. Interrupt 1 means
-NIC is ready and interrupt 2 means the printer is burning;
-
-\item like in a normal bus protocol, you need routing/arbitration.
-With interrupts, the master/slave relationship is inverted. Here the
-slaves send requests to the masters. So you need to decide which master
-receives the interrupt from a given slave. Unlike the preceding points,
-this doesn't really have a standard solution. Every system is different.
-If you have one master, it's simple. Once you have multiple masters you
-need to decide how the interrupts are wired to the masters. Often you
-need this to be reconfigurable at runtime. For example, the NIC might be
-handled by the LM32 in-chip or the host CPU off-chip;
-
-\item like in a normal bus protocol, you need a description language.
-When your software starts up it needs to determine which slave is
-connected to which master
-
-\item you lose plugability with the original bus protocol: interrupts
-are a completely distinct bus and they cannot be plugged together with
-devices that just use the vanilla bus protocol.
-\end{itemize}
-
-Recently, the PCI SIG decided to obsolete "legacy interrupts" and
-replace them with "message-signalled interrupts". MSI is a procedure for
-using the existing PCIe bus for sending interrupts. It essentially says:
-if you need a slave to talk to a master just send requests as a
-master yourself.
-
-This approach has a few advantages: it allows an unlimited
-number of interrupt lines (addressing), without the need to share lines. It
-makes possible for all participants on the PCIe bus to receive
-interrupts if they desired (routing). There is a performance
-benefit since it requires less messaging than legacy interrupts
-(acknowledgement). Indeed, now that PCIe uses a message-oriented
-protocol, the pin count advantage of legacy interrupts is gone.
-Finally, legacy interrupts can be implemented on top of the
-MSI scheme.
-
-FPGA designers face exactly the same decision that the PCI SIG faced:
-they are designing a SoC. Pin count is not an issue because synthesis
-eliminates any pins that are not used, and inside the chip there is plenty of
-freedom.
-
-In the core that is expected to handle the ``interrupt'',
-we can add a bus slave interface instead of a "raise
-interrupt" pin. At first this may seem just a clever hack that
-saves some time. However, we realized this brings in more and more
-advantages:
-
-\begin{itemize}
-
-\item acknowledgements are handled just like normal bus acknowledgment.
-For Wishbone, for example, when the host
-system is done processing the interrupt, it raises the ack line. This
-means interrupts are discrete, like in MSI. You can say "I have 5 things
-for you to do" and not have to squish them into a "I raise this line
-until you do everything I want". This means you can choose to handle
-some of events that generated interrupts, but delay processing the
-others. If the master stops handling interrupts, acks stop flowing
-and no code is needed.
-
-\item there's no need for special interrupt masking etc. If the
-master doesn't
-want to process an interrupt right now, it just lets the stall line go high.
-From the software side, this requires nothing special at all. If the master
-is not reading, the queue fills and the line goes high. Again no node
-is needed;
-
-\item there is an unlimited number of interrupts. Each address+data
-pair can be interpreted differently;
-
-\item nothing special at all is needed to route interrupts. They
-are just another master on the crossbar. If a master wants to receive
-interrupts from a particular slave, it just writes its address to a
-register on the device. When that slave generates an interrupt, it is a
-write to the specified address and the master gets the interrupt. This completely
-solves the problem of determining which of multiple slaves
-raises which interrupts;
-
-\item by using the same protocol for interrupts as for the normal bus,
-modularity is improved. For example we can make software bus slaves and
-hardware masters can read/write
-memory from software programs in userspace on the host system;
-
-\item since interrupts are generated by bus writes, they are compatible
-with a remote protocol like Etherbone or other ones.
-You can remotely trigger an interrupt on any device in the
-network.
-\end{itemize}
-
-The only argument we can see in favour of legacy interrupts on a SoC is
-that they are "simpler". A slave just needs to write a '1' or a '0' to
-an output as opposed to adding a bus master interface. However,
-this ``simplicity'' argument ignores the costs that happen once you have
-enough of these devices, once the
-interrupt bus becomes non-trivial. To keep the code simple, one could
-easily imagine a small HDL component that takes an address register and
-a generates a write upon request.
-
-To use MSI and achieve compatibility with a legacy master like
-a soft-core with an ``interrupt'' input line, you just need an
-"interrupt unit" which is a bus slave that raises different pins when it
-is written to at different offsets. In other words, just like in PCIe,
-implement legacy interrupts using the MSI approach is pretty simple.
+Neither of these is strictly in the scope of SDB; but especially as
+far as interrupts are concerned, we really think FPGA designs should
+be converted to the concept of MSI interrupts. Message-signalled
+interrupts have a number of advantages over legacy interrupts, and
+don't even need any description besides what SDB already offers.
+A more complete discussion of this is in the wiki page of the project:
-% This ends the text Wesley sent to the list.
-
-To summarize: interrupt lines are not strictly in the scope of SDB and
-we currently don't offer such description because we really think FPGA
-designs should be converted to the concept of MSI interrupts -- which need
-no description support within SDB as a side effect.
+\texttt{http://www.ohwr.org/projects/fpga-config-space/wiki}.
On the other hand, it may make sense to define SDB structures to
-describe this special wiring, to help existing ``legacy'' projects to
-benefit from SDB and avoid doping the software source code with static
-information about device wiring. Maybe future releases of this specification
-will allow description of legacy interrupts.
-
+describe this special wiring, when it exists, to help existing
+``legacy'' projects to benefit from SDB and avoid doping the software
+source code with static information about device wiring. Thus,
+it may happen that future
+releases of this specification include description of legacy
+interrupts.
\subsection{The Overall System Structure}
@@ -509,7 +370,7 @@ Another issue is that the device drivers may either be concerned with the FPGA
binary as a whole or be interested in each and every individual logic block;
thus, the same
GPIO logic block can be either driven by the host or ignored by it -- because
-is is directly used by the soft-core within the synthesized binary.
+is is directly used by the soft-core within the synthesised binary.
\subsection{Wb-core and Enumeration}
@@ -637,7 +498,7 @@ soft-core CPU with very little overhead.
Use cases for such a storage file-system are almost read-only, but not completely:
sometimes users need to update FPGA binary images or some
calibration parameters. Such use doesn't require a complex
-filesystem with wear-leveling capabilities.
+filesystem with wear-levelling capabilities.
In our search for the state of the art, we didn't find small and
simple filesystems that allow in-place replacement of files. We thus
@@ -662,7 +523,7 @@ then register a filesystem that only spans the needed part of the
flash, as defined at manufacture time (or by device-specific
software).
-A user-space program creates the SDB filesytem as an image file, it
+A user-space program creates the SDB filesystem as an image file, it
writes it using normal char-device MTD operations or other tools
to move binary data. The filesystem can then be accessed on the storage
medium by either user code or kernel code. Or by the microcontroller
@@ -931,7 +792,7 @@ number.
\item[sdb\_records] \hfill \\
This field specifies the number of records in the table. It \textbf{must} include
-this very record in the count, and the whole address range (this number multplied by 64 bytes)
+this very record in the count, and the whole address range (this number multiplied by 64 bytes)
\textbf{must} be accessible. Note that the array \textbf{may} include empty records at any position.
\item[sdb\_version] \hfill \\
@@ -1379,13 +1240,19 @@ byte-swap each 32-bit word before using the structure in a
little-endian host. After such swapping, the data fields live at the correct
offsets and must be accessed as big endian.
-\subsection{Existing Tools}
+\subsection{References}
-As part of the \textit{Etherbone} project, you already find a number
+The \textit{Etherbone} project, is an early adopter of SDB;
+it includes also a number
of tools that work with SDB structures (including \texttt{eb-ls} that
printed the table of devices shown above). The project is at
\texttt{http://www.ohwr.org/projects/etherbone-core} .
+The \textit{sdbfs} work started as a separate project but it is now
+part of the \texttt{fpga-config-space} project. It is currently work
+in progress, but the \textit{gensdbfs} and other non-kernel parts are already
+being used in production.
+
\end{document}
-% LocalWords: metadata FPGA gateware CERN
+% LocalWords: metadata FPGA gateware CERN endian etherbone filesystem
--
GitLab