Hardware Software Interface (HSI)
Introduction
The Hardware Software Interface (HSI) provides access to low level hardware information from exported Vivado designs so that it can be used in software development.
The Hardware Software Interface was the tool used for BSP and application creation in older releases of the Xilinx toolchain for software development. In recent releases, HSI has been progressively replaced for software development by subsequent versions of the Xilinx Software Command Line Tool (XSCT), that provides higher levels of abstraction.
Despite the fact the Hardware Software Interface (HSI) is not an independent tool anymore, the HSI API is used internally by XSCT to access the hardware information and the main HSI commands have been integrated into XSCT. In this way, if the user requires a deeper knowledge on the hardware design, a whole range of HSI commands can be used from XSCT to access the hardware information.
As an example of how the HSI is still a must-have tool in the skilled Xilinx software programmer toolbox, we will use it to generate all of the hardware-dependent components of a Linux runtime:
- An alternative and straightforward way to create standalone software components:
- First Stage Boot Loader (FSBL)
- Platform Management Unit Firmware.
- A mandatory tool to automate the creation of Linux Device Trees for custom hardware.
NOTE: the HSI API commands in this section are available in Xilinx Vitis 2019.2 and 2020.1
Setting the Environment
Before starting to work, we create a folder for the components of the Linux runtime we plan to build.
mkdir -p ~/soc-course/linux
Then, we need to get the exported Xilinx Shell Archive (XSA) from the Vivado project we want to support in our Linux runtime. As a complete example, we will use the design with custom peripherals in the programmable logic. For an easier path management, we will copy the file into a new hardware directory inside the Linux related one:
mkdir -p ~/soc-course/linux/hardware
cp ~/soc-course/vivado/ultra96v2_custom/ultra96v2_custom.xsa ~/soc-course/linux/hardware/
For easier path handling, we move into the linux directory:
cd ~/soc-course/linux/
Now, we need to load the Xilinx Vitis environment script:
source /tools/Xilinx/Vitis/2019.2/settings64.sh
export LC_ALL="C"
Standalone Software with HSI
In this section, we will explore how we can use the HSI to create standalone software, focusing on how to generate:
- First Stage Boot Loader (FSBL)
- Platform Management Unit Firmware.
Note that not all of the software related functions of the Hardware Software Interface are supported now, but we can use it the provided ones as a straightforward way of creating standalone software apps.
In order to access the HSI API, we open the Xilinx Software Command-line Tool (XSCT):
source /tools/Xilinx/Vitis/2019.2/settings64.sh
export LC_ALL="C"
xsct
Opening an exported Hardware Design
As a first step, we need to open the desired hardware project that we previously exported from Vivado. In order to do this, we will use the following HSI command available from the XSCT shell:
hsi open_hw_design hardware/ultra96v2_custom.xsa
Once the hardware design is open, we can check the contents of the hardware that are going to be used have been extracted to the hardware folder by sending the ls command to the OS shell using the XSCT exec command:
exec ls hardware
We can check that we have the following contents there:
- all of the psu_init files
- the ultra96v2_custom.bit bitstream for the Programmable Logic
- a drivers folder that contains empty wrappers for a potential driver for our custom AXI IP peripheral.
In addition, we can check the name of the hardware design that we have opened by running the following HSI command:
hsi current_hw_design
We will see that the hardware design name matches the name of the Vivado HDL wrapper that we created on top of the Block Design, in this example design_1_wrapper.
Creating the FSBL and PMU Firmware applications
Once the hardware design is opened, we can use the generate_app command in the HSI to generate a software template for both the First Stage Boot Loader (FSBL) and the PMU Firmware.
First, we start by checking the available application templates for the psu_cortexa53_0 processor:
hsi generate_app -sapp -proc psu_cortexa53_0
From the available list, we must select the zynqmp_fsbl template and generate the source code for the First Stage Boot Loader (FSBL) in the my_fsbl folder:
hsi generate_app -proc psu_cortexa53_0 -app zynqmp_fsbl -dir my_fsbl
Note: if we plan to use the Cortex-R5 processor to execute the FSBL, we just need to change the psu_cortexa53_0 identifier by the psu_cortexr5_0 one.
Now, we can repeat but checking the available templates for the psu_pmu_0 processor:
hsi generate_app -sapp -proc psu_pmu_0
From the available list, we must select the zynqmp_pmufw template and generate the source code for the PMU Firmware in the my_pmufw folder:
hsi generate_app -proc psu_pmu_0 -app zynqmp_pmufw -dir my_pmufw
NOTE: there is an -os flag that is set to standalone as the default Operating System, but there is no any other available choice actually in the HSI.
Finally, after the template generation has finished, we can close the hardware design and exit the XSCT:
hsi close_hw_design design_1_wrapper
exit
Fixing the FSBL and PMU Firmware BSPs
When we have finished the template generation for our apps, we can check that a Board Support Package has been generated for each of them.
In the my_fsbl/zynqmp_fsbl_bsp/system.mss file, we have all of the Board Support Package settings for the auto-generated BSP for the FSBL, including:
- OS: parameters for the standalone Operating System.
- Processor: parameters for the target processor.
- Driver: an entry for each required driver in our hardware, including Xilinx provided and custom ones.
-
Library: an entry for each supported library, including those required for a FSBL, i.e:
- xilffs
- xilsecure
- xilpm
In the my_pmufw/zynqmp_pmufw_bsp/system.mss file, we have all of the Board Support Package settings for the auto-generated BSP for the PMU Firmware, including:
- OS: parameters for the standalone Operating System.
- Processor: parameters for the target processor.
- Driver: an entry for each required driver in our hardware, including Xilinx provided and custom ones.
-
Library: an entry for each supported library, including those required for a PMU Firmware, i.e.:
- xilfpga
- xilsecure
- xilskey
If we take a look to the stdin and stdout parameters inside the OS settings for both BSPs, we will see that they are set to psu_uart_0 by default:
PARAMETER stdin = psu_uart_0
PARAMETER stdout = psu_uart_0
In order to adapt this to our Ultra96-V2 board using the UART 1 in the USB adapter, we might be tempted about changing the stdin and stdout parameters to psu_uart_1 in the system.mss files, but this won't have any effect. In order to use this file, we should use the BSP related elements from the HSI API that have been deprecated or are just not working anymore.
In order to change the used UART, we need to take a look to the xparameters.h files in both BSP, i.e.:
- my_fsbl/zynqmp_fsbl_bsp/psu_cortexa53_0/include/xparameters.h
- my_pmufw/zynqmp_pmufw_bsp/psu_pmu_0/include/xparameters.h
In both of the files, we find a declaration for the base address of the stdin and stdout UARTs:
#define STDIN_BASEADDRESS 0xFF000000
#define STDOUT_BASEADDRESS 0xFF000000
If we search inside the xparameters.h files, we will see that, in the same way that it's done for every other peripheral in our design, we find a section related with the UART peripherals:
/******************************************************************/
/* Definitions for driver UARTPS */
#define XPAR_XUARTPS_NUM_INSTANCES 2
/* Definitions for peripheral PSU_UART_0 */
#define XPAR_PSU_UART_0_DEVICE_ID 0
#define XPAR_PSU_UART_0_BASEADDR 0xFF000000
#define XPAR_PSU_UART_0_HIGHADDR 0xFF00FFFF
#define XPAR_PSU_UART_0_UART_CLK_FREQ_HZ 100000000
#define XPAR_PSU_UART_0_HAS_MODEM 0
/* Definitions for peripheral PSU_UART_1 */
#define XPAR_PSU_UART_1_DEVICE_ID 1
#define XPAR_PSU_UART_1_BASEADDR 0xFF010000
#define XPAR_PSU_UART_1_HIGHADDR 0xFF01FFFF
#define XPAR_PSU_UART_1_UART_CLK_FREQ_HZ 100000000
#define XPAR_PSU_UART_1_HAS_MODEM 0
/******************************************************************/
We can easily identify that the base address for stdin and stdout is pointing to psu_uart_0, so we need to change the values in both of the xparameters.h files to the psu_uart_1 base address, i.e.:
#define STDIN_BASEADDRESS 0xFF010000
#define STDOUT_BASEADDRESS 0xFF010000
Once this is done and the xparameters.h are saved, our BSPs for FSBL and PMU Firmware will be using the UART 1 that we need for communicating with the Ultra96-V2.
Building the FSBL and PMU Firmware applications
When we have fixed the BSPs for the generated templates, we are ready to build the applications from the Linux O.S.. Note that we need to load the Vitis environment settings before building, as they include the environment to use the cross-compiler toolchains we are going to require.
As a first step, we start with building the First Stage Boot Loader, so we get into the template folder:
cd ~/soc-course/linux/my_fsbl
Here, we will find a Makefile driven application. If we take a look to the autogenerated Makefile file, we will see that several targets and ARM 64-bit architecture cross-compilation flags are defined inside. Note that the generated executable binary is named executable.elf.
In this way, we can build the FSBL by using make:
make
As a quick check, we can check the ARM 64-bit architecture in our executable.elf by using the file command:
file executable.elf
This should produce an output similar to:
executable.elf: ELF 64-bit LSB executable, ARM aarch64, version 1 (SYSV), statically linked, BuildID[sha1]=99cc0f012f1687eff86a71c575cc102cb7aae36b, with debug_info, not stripped
Now, we can proceed with the PMU Firmware by following the same process, so we get into the template folder:
cd ~/soc-course/linux/my_pmufw
Again, we will find a Makefile driven application. If we take a look to the autogenerated Makefile file, we will see that several targets and MicroBlaze 32-bit architecture cross-compilation flags are defined inside. Note that the generated executable binary is named executable.elf.
Now, we can build the PMU Firmware by using make:
make
As a quick check, we can check the MicroBlaze architecture in our executable.elf by using the file command:
file executable.elf
This should produce an output similar to:
executable.elf: ELF 32-bit LSB executable, Xilinx MicroBlaze 32-bit RISC, version 1 (SYSV), statically linked, with debug_info, not stripped
Creating and Testing a Boot Image
Now that we have build the first of the components we need for booting a Linux runtime, we can create a boot image containing the following partitions:
- First Stage Boot Loader for Cortex-A53 0
- PMU Firmware for Platform Management Unit
- Bitstream for Programmable Logic
This can be done by creating a hsi_flow.bif file inside the linux folder (this is just for relative path convenience, modify as per your setup):
cd ~/soc-course/linux
vi hsi_flow.bif
And copy this content (note that we are using FSBL to load the PMU Firmware in order to see its messages in stdout):
/* Linux */
the_ROM_image:
{
[bootloader, destination_cpu = a53-0] my_fsbl/executable.elf
[destination_cpu = pmu] my_pmufw/executable.elf
[destination_device = pl] hardware/ultra96v2_custom.bit
}
Now, we can generate the Boot Image with the bootgen command line tool:
bootgen -image hsi_flow.bif -arch zynqmp -w -o BOOT.bin
Then, we can copy the BOOT.bin image to the FAT partition of the microSD and configure the Ultra96-V2 Boot Mode to SD:
- Switch 1: OFF
- Switch 2: ON
Now, we need to open a serial terminal in /dev/ttyUSB1 and then we power-up the board.
If everything goes well, we should see:
- In the Ultra96-V2 board, the blue LED named DONE (close to the microSD) is ON, indicating that the bitstream has been successfully programmed.
- In the serial terminal, we will get the messages coming from the First Stage Boot Loader and the PMU Firmware, e.g.:
Xilinx Zynq MP First Stage Boot Loader
Release 2019.2 Jun 15 2020 - 20:11:43
PMU Firmware 2019.2 Jun 15 2020 20:12:03
PMU_ROM Version: xpbr-v8.1.0-0
Customized Device Tree with HSI
The Device Tree in Linux provides a way to describe hardware elements that cannot be discovered automatically by the Kernel and that can be specific for System-on-Chip architectures or custom board designs. This information was previously included in the Kernel source code, but the explosion of available custom ARM based boards made this approach very hard to maintain as it was producing a fragmentation in the Kernel board related stuff.
In order to avoid this, the Device Tree was introduced in Linux as a way of providing a mechanism that acts as a container for SoC and board specific information that is used by generic processor code in the Kernel source.
The Device Tree data supports two representation formats:
-
Device Tree Source (DTS): human readable format in plain text and stored as:
- .dts: mandatory top source files
- .dtsi: optional include source files.
-
Device Tree Blob (DTB): binary format used by Linux to identify and register the hardware devices. It has two formats:
- Flattened Device Tree (FDT): the raw format for the DTB itself that the Kernel reads for using the information at early booting stages.
- Expanded Device Tree (EDT): the data structure built into the Kernel from the DTB for more convenient access in later booting stages and after booting.
Because the Zynq UltraScale+ MPSoC is a highly flexible system that allows not only for multiple configurations in the Processing System, but for custom architectures implemented in the Programmable Logic, different hardware designs in Vivado require a different Device Tree to be properly supported by Linux. By using the Hardware Software Interface (HSI), the task of generating a customized Device Tree for a given Vivado Hardware Design can be greatly automatized.
Getting the sources
As a first step, we need to get the sources of the device tree generator repository maintained by Xilinx. This is basically a TCL based tool that acts as an extension for the HSI interface to allow for automatic generation of a custom device tree for a specific hardware design exported by Vivado.
In order to do this, we go to the Linux folder we have previously created:
cd ~/soc-course/linux
Then, we clone the repository and checkout the release that matches our Xilinx toolchain version:
git clone https://github.com/Xilinx/device-tree-xlnx
cd device-tree-xlnx
git checkout xilinx-v2019.2
Once done,we go back to the Linux folder:
cd ~/soc-course/linux
Generating the Device Tree Source (DTS)
Once we have the sources, we open the XSCT to access the HSI commands:
xsct
From the XSCT shell, we open the Vivado exported hardware design by using the HSI API-- ultra96v2_custom.xsa in this example:
hsi open_hw_design hardware/ultra96v2_custom.xsa
Once the hardware design is open, we add the device tree generator sources as an additional template repository for HSI:
hsi set_repo_path device-tree-xlnx
After doing this, we will have a ** new kind of Operating System** supported in the HSI interface, the device_tree one.
Now we can create a new software design named device-tree that makes use of the device_tree OS and targets the psu_cortexa53_0 processor:
hsi create_sw_design device-tree -os device_tree -proc psu_cortexa53_0
Once we have done this, we can generate the Device Tree Sources (DTS) for our customized hardware design into a new my_dts directory:
hsi generate_target -dir my_dts
Finally, we can close the current hardware design and exit the XSCT shell to go back to the Linux O.S. terminal:
hsi close_hw_design [hsi current_hw_design]
exit
Hacking the Device Tree Source (DTS)
Now that we have the generated DTS, we can take a look to the contents of the my_dts folder.
First, we have the hierarchy of device tree source files, i.e. the sources we are actually interested in:
- system-top.dts: This is the top file that contains the memory information, early console and the boot arguments.
- zynqmp.dtsi: This the included file that contains the nodes for CPU and Processing System peripherals.
- zynqmp-clk-ccf.dtsi: This is the included file that contains the clock information for the peripherals.
- pcw.dtsi: This is the included file that contains dynamic properties for the Processing System peripheral nodes.
- pl.dtsi: This is the included file that contains the nodes for peripherals implemented in Programmable Logic.
In addition, we have a device-tree.mss file that contains the information for this system as a conventional software project. Note that we are not going to use this file at all, but it's just an artifact generated by the HSI API that may contains some useful information, e.g.:
- the stdin and stdout are using psu_uart_0.
- the peripherals and their associated Linux device drivers.
We will introduce the following changes into system-top.dts to allow for using the Ultra96-V2 UART 1 as the serial console.
--- my_dts/system-top.dts 2020-06-17 17:52:39.053379439 +0200
+++ my_dts/system-top.dts 2020-06-17 17:53:12.117920151 +0200
@@ -17,8 +17,8 @@
};
aliases {
i2c0 = &i2c1;
- serial0 = &uart0;
- serial1 = &uart1;
+ serial0 = &uart1;
+ serial1 = &uart0;
spi0 = &spi0;
spi1 = &spi1;
};
Now, we need to fix several issues with the SD card by modifying the corresponding sdhci0 node device node to match the Ultra96-V2. Note that *sdhci1 node is not actually used in the Ultra96-V2.
The microSD card has no write protect pin, so the card will be recognized as Read-Only. In the kernel device tree, you can add disable-wp or wp-inverted to the mmc0 sdhci0 node:
- disable-wp: disables the write protection detection logic.
- wp-inverted: inverts the write protection detection logic.
In addition, there is a problem with the voltage levels in the microSD that may lead to data corruption or not accessible partitions. More specifically, the MMC voltage is set to 1.8V and this is not appropriated for the Ultra96-V2 board, so we need to add the no-1-8-v parameter into the sdhci0 node.
In order to introduce these changes, we modify the zynqmp.dtsi file:
--- my_dts/zynqmp.dtsi 2020-06-17 18:00:19.229865424 +0200
+++ my_dts/zynqmp.dtsi 2020-06-17 18:00:54.925731674 +0200
@@ -881,6 +881,8 @@
power-domains = <&zynqmp_firmware 39>;
nvmem-cells = <&soc_revision>;
nvmem-cell-names = "soc_revision";
+ disable-wp;
+ no-1-8-v;
};
sdhci1: mmc@ff170000 {
Finally, if we take a look to the pl.dtsi file, we will find the nodes corresponding to the peripherals in the Programmable Logic and its associated Linux drivers:
- axi_timer_0: it is using a Kernel space timer drivers supplied by Xilinx, i.e. xlnx,axi-timer-2.0 (for PL), xlnx,xps-timer-1.00.a (for PS).
- my_led_driver_0: it is using an hypothetical Kernel space driver supplied by Xilinx, i.e. xlnx,my-led-driver-1.0.
The problem is that this driver doesn't actually exist. In this point, we have two options:
- Write a Kernel space driver and include it into the supplied Kernel sources.
- Write a User space driver by employing the already supplied generic Userspace I/O (UIO) driver.
The Userspace I/O (UIO) driver supports several advanced features (register access, DMA, interruptions...) that makes this the most common selection for custom devices in the Programmable Logic. In order to make use of the UIO driver, we just modify the declared driver for our peripheral in pl.dtsi:
--- my_dts/pl.dtsi 2020-06-17 18:15:17.331872994 +0200
+++ my_dts/pl.dtsi 2020-06-17 18:15:55.248483497 +0200
@@ -31,7 +31,7 @@
/* This is a place holder node for a custom IP, user may need to update the entries */
clock-names = "s00_axi_aclk";
clocks = <&zynqmp_clk 71>;
- compatible = "xlnx,my-led-driver-1.0";
+ compatible = "generic-uio";
reg = <0x0 0xa0010000 0x0 0x10000>;
xlnx,s00-axi-addr-width = <0x4>;
xlnx,s00-axi-data-width = <0x20>;
Building the Device Tree Blob (DTB)
Once we have generated and patched the Device Tree Sources, we need to use it to generate the Device Tree Blob (DTB). The DTB is just a binary container for the device tree information that is read by the Kernel to configure itself at boot time and adapt its behavior to the specific hardware at a processor and board level.
NOTE: For our convenience, all of the following commands assume that we are the linux folder:
cd ~/soc-course/linux
Before going forward, we need to get the Device Tree Compiler (DTC), a tool that allows to generate DTB files from DTS ones and vice versa.
There are several ways to get the DTC. First, it's included in the Kernel source code, so it can be built at the same time we build the Kernel binary in a posterior step.
Second, you can download the DTC sources from the official Kernel GIT, build it and add it to your Operating System PATH, e.g.:
git clone https://git.kernel.org/pub/scm/utils/dtc/dtc.git
cd dtc
make
export PATH=`pwd`:$PATH
cd ..
Third, if you are using a Debian/Ubuntu based distribution, the DTC is provided as an application that can be installed using the package manager:
sudo apt-get install device-tree-compiler
Now that we have the Device Tree Compiler, note that DTC works on single files, so we need to merge all of the device tree source files into a single one. In order to do this, we can just use the GCC compiler to create a single system.dts file with the content of all of the previously generated ones:
gcc -I my_dts -E -nostdinc -undef -D__DTS__ -x assembler-with-cpp -o my_dts/system.dts my_dts/system-top.dts
Finally, now that we have a single system.dts file, we can use the DTC to build the DTB from the DTS:
dtc -I dts -O dtb -o my_dts/system.dtb my_dts/system.dts
Is interesting to note that we can go the way forward and use the DTC to generate a DTS from a DTB, something very useful for forensic and debugging Linux runtimes, e.g.:
dtc -I dtb -O dts -o my_dts/system.dts my_dts/system.dtb
The DTB can be included in the the Boot Image binary in some booting approaches, but is a most common practice to leave them as an independent binary and make use of them as an independent component. In this way, we reserve the generated system.dtb for being used in posterior steps.