ARM Trusted Firmware (ATF)
- Introduction
- ARM Trusted Firmware Boot Flow
- Building the ARM Trusted Firmware
- Creating and Testing a Boot Image
Introduction
The Zynq UltraScale+ MPSoC APU is an ARMv8-A architecture and as such supports four Exception Levels (EL). These Exception Levels are used to control privileges at the application level, i.e., each application has its own exception level.
The Exception Levels are numbered and ordered from 0 to 3, being EL0 the lower level of privileges and EL3 the highest level of privilege. Despite the fact the following scheme for the software model is not mandatory, it's the convention recommended by ARM and standard software should adapt to it:
- EL0: User applications.
- EL1: Operating System.
- EL2: Hypervisor.
- EL3: Low-level Firmware
The ARM Trusted Firmware (ATF) for the Zynq UltraScale+ MPSoC implements the firmware for EL3. More specifically, it is the Trusted Firmware-A (TF-A), the standard and layered implementation ARM Cortex Application processors ARMv7-A and ARMv8-A. This is the reason why the ARMv8 architecture exception levels (ELs) are exclusively for applications running on the APU, that is based on Cortex-A53 ARMv8-A architecture. The RPU is a Cortex-R5 ARMv7-R architecture and PMU is a Microblaze based one, hence they do not support them.
In the Zynq UltraScale+ MPSoC, the ARM Trusted Firmware acts as a proxy to modify system-critical settings on behalf of the Operating System running at EL1. As an example, this is how power management functionality works for the Linux Operating System:
- The Linux O.S. sends issues power management commands (CPU stand-by, CPU suspend, power ON/OFF, system reset...) that are passed to the ATF.
- The ATF communicates the requested operations to the Platform Management Unit (PMU) Firmware via the Inter Processor Interrupt (IPI).
- The PMU Firmware performs the requested low-level operation.
- The PMU Firmware reports the outcome of the operation to the ATF via the Inter Processor Interrupt (IPI).
- The ATF sends the received information to the upper level software, i.e. the Linux Operating System.
Finally, the exception levels are related with ARM TrustZone technology, a mechanism that allows for hardware resources isolation in the ARM SoC. Generally speaking, the ARM architectures supports two TrustZone modes for the cores, Secure and Non-Secure, and this is the relation with the different Exception Levels:
- EL0, EL1, EL2: the processor can be in Secure or Non-Secure mode.
- EL3: the processor is only allowed to be in Secure mode.
NOTE: More detailed and functional information about this topic can be found in the Xilinx ARM Trusted Firmware Wiki
ARM Trusted Firmware Boot Flow
The ARM Trusted Firmware cold boot flow comprises up to five stages or boot levels running at different exception levels:
Stage | Level | Description |
---|---|---|
BL1 | EL3 | Trusted bootstrap, boot detection |
BL2 | EL1S | Trusted bootloader |
BL31 | EL3 | Resident runtime firmware |
BL32 | EL1S | Optional Secure Payload (e.g. trusted O.S. |
BL33 | EL2 | Normal world bootloader |
NOTE: in this table, EL1S stands for Exception Level 1, TrustZone in Secure mode. Remember that EL3 is always in TrustZone Secure mode.
The ARM Trusted Firmware includes in its source code the BL1, BL2, BL31 and BL32 elements, but in the specific case of the Zynq UltraScale+ MPSoC we have that:
- BL1 is the Boot ROM for CSU and PMU.
- BL2 is the First Stage Boot Loader (FSBL).
In addition, when using Linux as the Operating System for the MPSoC, BL33 will be the u-Boot program that we will build later.
In this way, we only need to build the BL31 component from the ARM Trusted Firmware source code.
Building the ARM Trusted Firmware
Setting the Environment
Load the Vitis environment so that the cross-toolchain tools included in the Xilinx toolchain are available:
source /tools/Xilinx/Vitis/2019.2/settings64.sh
Then export the CROSS_COMPILE and ARCH environment variables for the aarch64 APU in MPSoC:
export CROSS_COMPILE=aarch64-linux-gnu-
export ARCH=aarch64
Finally, for an easier path handling, we move into the linux directory we created in previous steps:
cd ~/soc-course/linux/
Getting the Sources
Then, we need to check the sources for ARM Trusted Firmware from the associated Xilinx GitHub repository:
git clone https://github.com/Xilinx/arm-trusted-firmware.git
cd arm-trusted-firmware
git checkout xilinx-v2019.2
Building the ATF BL31 Boot Stage
Before building, we should note that there are several specific build options for the ARM Trusted Firmware
- ZYNQMP_ATF_MEM_BASE: Specifies the base address of the BL31 binary.
- ZYNQMP_ATF_MEM_SIZE: Specifies the size of the memory region of the BL31 binary.
- ZYNQMP_BL32_MEM_BASE: Specifies the base address of the BL32 binary.
- ZYNQMP_BL32_MEM_SIZE: Specifies the size of the memory region of the BL32 binary.
-
ZYNQMP_CONSOLE: Select the console driver. Options:
- cadence, cadence0: Cadence UART 0
- cadence1: Cadence UART 1
Regarding the ZYNQMP_ATF_MEM_BASE and ZYNQMP_ATF_MEM_SIZE, these allow us to specify in which memory region we want to place and execute the ATF. By default, the ARM Trusted Firmware builds for On-Chip Memory (OCM) space at address 0xFFFEA000. We should only move this if the BL31 would be too big to fit in the reserved space, as it happens when we build it for debug. As this is not the case, we can safely ignore them.
About the ZYNQMP_BL32_MEM_BASE and ZYNQMP_BL32_MEM_SIZE, we do not plan to include the optional BL32 Secure Payload so we can safely ignore them.
Finally, the ZYNQMP_CONSOLE is the flag that selects the UART in the MPSoC that will be used to print information in the standard output. As the Ultra96-V2 board is using the UART 1 to connect to the USB adapter, we will need to select cadence1 as the console at build time.
NOTE: in some software for Xilinx devices, you will sometimes find that Cadence brand is used to denote specific devices or drivers. This is due to the fact that some of the Hard IP-Cores in the Xilinx SoCs are built from Cadence layout IP libraries.
Now, we can build the BL31 component of ARM Trusted Firmware for Zynq MPSoC:
make PLAT=zynqmp ZYNQMP_CONSOLE=cadence1 bl31
The generated binary for BL31 is located in /build/zynqmp/release/bl31
build/zynqmp/release/bl31/bl31.elf
Creating and Testing a Boot Image
Now that we have a working ARM Trusted Firmware, we can test it in conjunction with the previously built FSBL and PMU Firmware. In order to do this, we can create a boot image containing the following partitions:
- First Stage Boot Loader for Cortex-A53 0
- PMU Firmware for Platform Management Unit
- Bitstream for Programmable Logic
- ARM Trusted Firmware (BL31)
IMPORTANT: Bitstream must be loaded before ATF is loaded. The reason is FSBL employs the OCM region which is reserved for ATF for holding a temporary buffer used for Programmable Logic programming in the case the bitstream is present in the boot image. For this reason, if bitstream is loaded after the ATF, the FSBL will overwrite the ATF image with its temporary buffer, leading to ARM Trusted Firmware corruption.
As an example, we will do this by updating the hsi_flow.bif file inside the linux folder (this is just for relative path convenience, modify as per your setup):
cd ~/soc-course/linux
vi hsi_flow.bif
And now we add an additional partition for the ARM Trusted Firmware (note that we are using FSBL to load the PMU Firmware in order to see its messages in stdout):
/* Linux */
the_ROM_image:
{
[bootloader, destination_cpu = a53-0] my_fsbl/executable.elf
[destination_cpu = pmu] my_pmufw/executable.elf
[destination_device = pl] hardware/ultra96v2_custom.bit
[destination_cpu = a53-0, exception_level = el-3, trustzone = secure] arm-trusted-firmware/build/zynqmp/release/bl31/bl31.elf
}
In the new partition for ARM Trusted Firmware we are specifying these flags:
- destination_cpu = a53-0: the APU Core 0 is the only one that can be used to execute the ATF.
- exception_level = 3: this flag is used to specify the EL3 for the ATF application.
-
trustzone = secure: this flag indicates that the core is set to TrustZone Secure mode. note that:
- the value for this flag can be secure or nonsecure, being the default nonsecure.
- as this is running on EL3, secure mode would be set automatically.
As a side comment, if we are using the Boot Image creation tool from Vitis, the Partition Editor includes an Advanced control section to specify the Exception Level and TrustZone mode:
Now, we can generate the Boot Image with the bootgen command line tool:
bootgen -image hsi_flow.bif -arch zynqmp -w -o BOOT.bin
Then, we can copy the BOOT.bin image to the FAT partition of the microSD and configure the Ultra96-V2 Boot Mode to SD:
- Switch 1: OFF
- Switch 2: ON
Now, we need to open a serial terminal in /dev/ttyUSB1 and then we power-up the board.
If everything goes well, we should see:
- In the Ultra96-V2 board, the blue LED named DONE (close to the microSD) is ON, indicating that the bitstream has been successfully programmed.
- In the serial terminal, we will get the messages coming from the First Stage Boot Loader, the PMU Firmware and the ATF, e.g.:
Xilinx Zynq MP First Stage Boot Loader
Release 2019.2 Jun 15 2020 - 20:11:43
PMU Firmware 2019.2 Jun 15 2020 20:12:03
PMU_ROM Version: xpbr-v8.1.0-0
NOTICE: ATF running on XCZU3EG/silicon v4/RTL5.1 at 0xfffea000
NOTICE: BL31: Secure code at 0x0
NOTICE: BL31: Non secure code at 0x0
NOTICE: BL31: v2.0(release):xilinx-v2019.2
NOTICE: BL31: Built : 13:35:54, Jun 16 2020
In these messages, we can check that the ATF is running at the 0xFFFEA000 address in On-Chip Memory (OCM) as expected.