Raspberry Pi 4 Model B Bare Metal Debugging with GDB and OpenOCD

October 20, 2024

Instruction for setting up an efficient workflow for bare metal development for the Raspberry Pi 4 Model B including full system reset from GDB - no more power cycling!

I created this as an offshoot of my AArch64OS project. There are a lot of great resources for bare metal development on the Raspberry Pi 4 Model B (RPi4B from here on), but I found that there were a few gaps that I wanted to fill. In particular this article and the corresponding repository strives to provide a concise but complete set of scripts and code with the following features to get started quickly with GDB, OpenOCD, and a debug probe:

• minimal dependencies - all tools are installed with a single script
• minimal assumptions about the build system and development tools you will ultimately use - e.g., does not even use make or C
• no unnecessary complexity - e.g., config.txt is pretty minimal despite there being a few things I’d do differently if building out a more complex bare metal project - kernel_old=1, disable_commandline_atags=1, etc.
• as close to the metal as practical - e.g., uses assembly instead of C so linking and linker scripts are not needed
• a complete solution for resetting the RPi4B directly from gdb - no more physical power cycling!

This last feature is something I could not find in any other resources I came across. I found lots of hints and discussions, but no complete solution. Maybe I just missed it, but regardless I hope this helps others where I struggled.

Hardware

• Serial to USB adapter. Most any FTDI based adapter with 3.3V logic levels will work (RPi4B UARTs do not support 5V) I used one from Adafruit.
• Raspberry Pi 4 Model B. I used the 4GB model, but any model should work.
• USB-C power supply compatible with the RPi4B.
• JTAG debug probe. I have tested both a Segger J-Link BASE and a FT232H adapter. Both work equally well for this project, but the FT232H is a lot cheaper so I setup the serial.sh script to use it by default, but left the J-Link as an option commented out.
• SD card. Any size will work because we are only creating a 100MB partition.
• SD card reader. I used this one.
• Host computer. I am using a Debian 12 “bookworm” system with a x86_64 architecture.
• Female/female jumper wires. I used these from Adafruit.
• Soldering iron and solder. Most any will work, but I used a Hakko FX-888D.
• Breakaway header pins. I had some laying around, but they are similar to these from Adafruit.

Serial Connection

A serial connection is used for monitoring the RPi4B bootloader messages and output from the bare metal program. Only three wires are needed for a basic serial connection: GND, TXD, and RXD. The TXD and RXD pins are used to send and receive data to and from the RPi4B. The GND pin is used as a reference voltage for the serial communication.

There isn’t a standard for the colors of the wires that is universally followed, but for my adapter relevant pins are:

• Black: GND
• Orange: TXD (3.3V)
• Yellow: RXD (3.3V)

If you are using a different adapter, you will need to look up the pinout.

Connect these to the RPi4B as follows (TXD connects to RXD and RXD connects to TXD):

• Black wire to J8 pin 6 (GND)
• Yellow wire to J8 pin 8 (TXD)
• Orange wire to J8 pin 10 (RXD)

When looking at the GPIO header (J8) on the RPi4B with the USB/ethernet ports at the bottom, the top left pin is 1 and the bottom right pin is 40.

JTAG Connection

The table below shows the connections between the JTAG debug probe and the RPi4B. The colors are just the colors of the wires I used. The tables shows the appropriate pins for both the J-Link and the FT232H; only one of those two columns are applicable depending on which probe you are using.

Note that for the FT232H I show both the generic FT232H pin name and in parentheses the names as labeled on the Adafruit breakout board.

J-Link pin mappings come from Segger.

The FT232H pin mappings depend on the OpenOCD configuration file you use as multiple mappings are possible for the reset pins depending on the ftdi layout_init and ftdi layout_signal settings in the file. I copied the interface/ftdi/um232h.cfg file that comes with OpenOCD so these pin mappings are specific to that config. There are a lot of non-working config files in various forums and blog posts for this debug probe. The UM232H is a development board from FTDI that has a FT232H chip on it, but is functionally equivalent to the Adafruit FT232H breakout board for our purposes. I copied it into the repo so I could add my own comments explaining the config and to avoid problems with it changing with future releases of OpenOCD.

Note, RTCK signal is not required on either debug probe per Arm. The VTREF signal is only required on the J-Link.

For the Adafruit FT232H, make sure the I2C mode switch is off (it connects D1 and D2).

When looking at the J-Link with the notch for the ribbon cable lock on the left, the top left pin is 1 and the bottom right pin is 20. When looking at the RPi4B GPIO header (J8) with the USB/ethernet ports at the bottom, the top left pin is 1 and the bottom right pin is 40.

The Tricky Part: The RPi4B does not expose a nRESET pin for JTAG to allow resetting the entire system. For efficient bare metal development, this is critical otherwise everytime you reload a new kernel to test, you have to physically power cycle the Raspberry Pi. To work around this, I connected the nRESET pin on the debug probe to the RUN pin on the J2 header of the Raspberry Pi. The J2 header does not have any header pins soldered to it by default, so I solder three breakaway header pins.

Software

Arm AAarch64 Toolchain

For basic assembly programming all you really need is a verion of binutils targeting the right architecture, but the most convenient way to get this along with a complete GCC toolchain is to use the Arm GNU Toolchain Downloads. Specifically, I used this download for AArch64 bare-metal target (aarch64-none-elf) because we are targeting a bare metal system with no pre-existing OS like Linux.

OpenOCD

OpenOCD is software for talking to various debug adapters and target boards/chips attached to them. I used the version from the Debian repository (v0.12.0). To use OpenOCD you need two config files: one for the adapter and one for the target. I used the interface/ftdi/um232h.cfg (or interface/jlink.cfg) that comes with OpenOCD but I slightly modified the board/rpi4b.cfg file that comes with OpenOCD to support resetting the board using the nRESET pin on the debug probe. Specifically, I changed reset_config trst_only to reset_config trst_and_srst.

OpenOCD once started acts as a server. You can connect to it with GDB (e.g., target remote localhost:3333) or you can connect to it with telnet (e.g., telnet localhost 4444) to issue commands directly.

Raspberry Pi 4 Model B Firmware

We need a few “firmware” files from Raspberry Pi bootloader. I used the latest version from the Raspberry Pi GitHub repository.

• boot/bcm2711-rpi-4-b.dtb and boot/overlays/disable-bt.dtbo: These are the device tree files used by the bootloader and and the kernels such as Linux. We are not using a Linux kernel, but the bootloader still needs these files to initialize the board. The Bluetooth module uses the first PL011 (UART0) by default which makes the primary UART on pins 8 and 10 of the J8 header disabled. The easiest way to enable a reliable serial connection is to disable the Bluetooth module with dtoverlay=disable-bt in config.txt. This loads the overlays/disable-bt.dtbo file and make the first PL011 (UART0) the primary UART exposed on pins 8 and 10 of the J8 header. See the Rasperry Pi docs for more details or see my discussion here.
• boot/start4.elf and boot/fixup4.dat: These are the second stage bootloader files. The first stage bootloader is stored on an onboard EEPROM. Both of these bootloader stages run on the VideoCore GPU and initialize the board before loading any code to run on the ARM CPU.

Picocom

There are multiple options for serial terminals, but I used picocom because it is simple and easy to use. I installed it from the Debian repository. The main thing you need to know about picocom is ctrl-a ctrl-x is used to exit it. minicom is another popular option that could be used if you are more familiar with it.

The Code

My rpi4b-bare-metal-debugging repository has a set of basic scripts and code to easily get started.

• get-tools.sh: Downloads and installs the Arm GNU Toolchain and Raspberry Pi 4 Model B firmware and installs a few system dependencies. wget and xz-utils are just required for downloading and extracting the toolchain. openocd and picocom have already been discussed. dosfstools and fdisk are required for sdcard.sh to create the bootable SD card partition.
• build.sh: Compiles the kernel.s and loop.s assembly files and extracts raw binary files and debug symbol files. It also builds a boot directory to be copied to the SD card with sdcard.sh.
• sdcard.sh: Creates a bootable SD card from the boot directory created by build.sh. It takes one argument which is the device name of the SD card. This script must be run as root (or using sudo). Make sure you specify the correct device or it could result in data loss. You’ve been warned. To check which device is the SD card, run sudo dmesg | tail immediately after inserting the SD card. You should see something like:

[1841835.047410] sd 6:0:0:3: [sdd] 124735488 512-byte logical blocks: (63.9 GB/59.5 GiB)
[1841835.049479] sdd: detected capacity change from 0 to 124735488
[1841835.049993]  sdd: sdd1

which would indicate the device name is /dev/sdd.

• serial.sh: Starts picocom to connect to the serial port of the Raspberry Pi. It takes one argument, the device name of the serial port (typically /dev/ttyUSB0). Similar to the SD card, you can verify the device name by running sudo dmesg|tail immediately after connecting the serial adapter. You should see something like:

  [1842054.065450] usb 3-2: FTDI USB Serial Device converter now attached to ttyUSB0
  

  which would indicate the device name is /dev/ttyUSB0.
• openocd.sh: Starts the openocd daemon with the rpi4b.cfg and um232h.cfg (or jlink.cfg) files.
• gdb.sh: Starts the aarch64-none-elf-gdb debugger with the gdbinit file loaded.
• gdbinit: The gdb initialization file connects to OpenOCD and defines a command, resetload that does the following:
  • reset the RPi4B by toggling the srst signal (connected to J2 RUN pin)
  • wait 15 seconds for the RPi4B to boot and OpenOCD to reconnect
  • halt the CPU
  • build the kernel
  • load the kernel to the RPi4B memory at 0x80000
  • load the corresponding debug symbols in gdb
  • set the program counter to the kernel entry point of 0x80000
• config.txt: The RPi4B boot configuration file. The rational for each setting is well commented in the file.
• kernel.s: The kernel assembly file for our “hello world” bare metal program. The binary generated from this is loaded with gdb.
• loop.s: A dummy kernel that does nothing but loop forever waiting for gdb to load a new kernel. This is what we copy to the SD card.
• rpi4b.cfg: Our modified OpenOCD configuration file for the Raspberry Pi 4 Model B.

First Time Setup

1. Clone the repository

git clone git@github.com:dsmcfarl/rpi4b-bare-metal-debugging.git
cd rpi4b-bare-metal-debugging

2. Install tools

./get-tools.sh

3. Initial build (required to create the boot directory for sdcard.sh)

./build.sh

4. Create a bootable SD card (insert an SD card into a card reader first)

sudo dmesg | tail # to see the device name
sudo ./sdcard.sh /dev/sdX # where /dev/sdX is the device name of the sd card

5. Boot (insert the SD card into the RPi4B and power it on)

The RPi4B is now running the loop.s program as its kernel. It will just loop forever doing nothing.

Development Workflow

1. Start a serial terminal (plug in the serial adapter first)

./serial.sh /dev/ttyUSB0 # where /dev/ttyUSB0 is the device name of the serial adapter, change as required

2. Start openocd in another terminal

./openocd.sh

3. Start gdb in another terminal

./gdb.sh

4. Reset and load the kernel. In gdb:

resetload

5. Test the kernel. In gdb:

continue

6. Make changes to kernel.s (for example, edit the message “hello world”)

7. Reload the kernel and test. In gdb:

resetload
continue

8. Press ctrl-c in gdb terminal to interrupt the kernel.

9. Repeat steps 6-8 as needed.

Its nice if you have multiple monitors or are using a multiplexer like tmux so you can see all three terminals (serial, openocd, and gdb) at the same time.

The nice part about the way this is setup is that you don’t even need physical access to the RPi4B after the initial setup. For example, I have the RPi4B/JTAG debug probe connected to a machine in my basement, but I can ssh into that machine from my laptop and do bare metal development from anywhere without lugging around a debug probe, serial adapter, power supply, and Raspberry Pi.

Conclusion

I hope this helps others get started with bare metal development on the Raspberry Pi 4 Model B. Hopefully the code in the repository in particular, provides a good starting point for you to expand on for your own projects. Please leave me a comment or submit an issue on github if you have any questions or suggestions for improvement.

Postscript

Why not QEMU? Why use real hardware instead of the convenience of an emulator? Emulators work great for rapidly iterating on development, but for bare metal development, an emulator cannot accurately simulate all of the quirks of the hardware. As a result, when you spend too much time developing on an emulator, you can end up with a system that works great in the emulator but not on the real hardware and when you try to migrate it to hardware, you end up backtracking a lot trying to find the source of the bugs. I find it is best to develop on real hardware from the beginning to avoid this problem.