Cross Compiler CMake Usage Guide with rsynced Raspberry Pi 64 bit OS

Cross-Compiler CMake Usage Guide : Rsynced Raspberry Pi 64-bit OSes Sysroot

This guide documents the complete steps to create rootfs/sysroot so that you can create a cross compile environment for any Raspberry Pi running 64-bit OS (such as Raspberry Pi OS (64-bit)) on a x64 Linux machine using rsync from the Raspberry Pi hardware you're already running. After that we will cross-compile a working Software Binaries (Hello-World CMAKE example in this case) with CMAKE using only the Raspberry Pi GCC Toolchains available within our project.

Important

Supported Pi Models: These instructions are meant only for Raspberry Pi Models/Versions with ARMV8-A CPU Architecture that supports 64-Bit execution state. You can find complete all supported Pi's list here.

Note

Our Cross-Compiler toolchains also works out-of-the-box on any Linux distro via WSL2 on Windows 10 Machines. 💯

Prerequisites:

A. Hardware:

Host [PC/Laptop]: Any x86/x86_64 AMD/Intel machine
Target [Raspberry Pi]: Raspberry Pi any variant/module

B. Software:

Host: Any Linux machine (💡 Our Cross-Compiler toolchains also works out-of-the-box on any Linux distro via WSL2 on Windows 10 Machines. 💯)
Target: Any Raspberry Pi 64-bit Buster (Debian 10) and above OS (Raspberry Pi OS 64-bit Bookworm Tested)

C. Others:

Networking: Your Target Machine (Raspberry Pi) and Host Machine (where you cross-compiling) both MUST have Internet Access, and MUST be on SAME Network to follow these instructions.

Steps/Settings for Target Machine (Raspberry Pi):

1. Start from Scratch (Optional)

Important

If you just brought a new Raspberry Pi or wanted to start from scratch just follow along. Otherwise, if you already has your Raspberry Pi setup, running, and Network Ready, then just skip to step 2

Note

This section assume you have atleast 10GB SDcard for installing Raspberry Pi OS Bookworm OS and a Laptop/PC for uploading it.

1.1. Download Softwares & Prepare the SD card

Download the latest version of Raspberry Pi 64-bit OS (Bookworm) from here on your laptop/pc.
You will be needing an image writer to write the downloaded OS into the SD card (micro SD card in our case). So download the open-source "win32 disk imager" from here, OR you can also use Balena Etcher instead.
Insert the SD card into the laptop/pc and run the image writer. Once open, browse and select the downloaded Raspberry Pi OS image file. Select the correct device, that is the drive representing the SD card.

Caution

If the drive (or device) selected is different from the SD card then the other selected drive will become corrupted. SO BE CAREFUL!

Once the write is complete, eject the SD card and insert it into the Raspberry Pi and turn it on. It should start booting up.
Please remember that after booting the Pi, there might be situations when the user credentials like the "username" and password will be asked. Raspberry Pi comes with a default username pi and password raspberry and so use it whenever it is being asked.

1.2 Set up Network

Now the you have your Raspberry Pi up and Running, its time to connect it your network with one of following ways:

If you have Monitor: Connect it to your raspberry pi along with a keyboard and mouse to navigate, and follow this guide.
If you don't have Monitor: Follow this guide
Any other way

2. Set up SSH

If you have Monitor: On the Raspberry Pi terminal, type: sudo raspi-config and menu should pop up on your terminal. To enable SSH, go to: Interfacing Options -> SSH -> Yes, and Click OK to enable it. Choose Finish finally and exit.
If you don't have Monitor: After setting up the network, if you don't have monitor or you operating it remotely. Then, enable SSH by just taking out your SD card, and hook it your computer, and simply create an empty file called ssh in the /boot/partition path inside SD card. Now insert back SD card into the Raspberry Pi.

3. Open Terminal

From another Laptop/PC using SSH: To connect to your Pi from a different computer, copy and paste the following command into the terminal window but replace 192.160.1.47 with the IP address of the Raspberry Pi. Use Ctrl + Shift + V to paste in the terminal.
```
ssh [email protected] 
```
It will ask for password, and if not changed, it is default (raspberry), and so use it whenever it is being asked.

Note

It is possible to configure your Raspberry Pi to allow access from another computer without needing to provide a password each time you connect. For more details, see here.

On Raspberry Pi directly with a Monitor: Just search "Terminal" and click on it.

4. Enable Development Sources

You need to edit your sources list to enable development sources. To do this, enter the following command into pi terminal:

sudo nano /etc/apt/sources.list

In the nano text editor, uncomment the following lines by removing the # character from following line:

deb http://deb.debian.org/debian bookworm main contrib non-free non-free-firmware
deb http://security.debian.org/debian-security bookworm-security main contrib non-free non-free-firmware
deb http://deb.debian.org/debian bookworm-updates main contrib non-free non-free-firmware

When done, press CTRL+O and then ENTER to quit.

5. Update the system

Run the following commands in terminal to update the system

sudo apt update
sudo apt dist-upgrade
sudo apt install rsync

6. Enable rsync with elevated rights

Later in this guide, we will be using the rsync command to sync files between the Host PC/Laptop and the Raspberry Pi. For some of these files, root rights (i.e. sudo) is required internally.

You can do this with a single terminal command as follows:

echo "$USER ALL=NOPASSWD:$(which rsync)" | sudo tee --append /etc/sudoers

That's it. Now rsync should be setup to run with sudo if needed.

7. Install the important Development Packages

Run the following commands in Raspberry Pi terminal to install the required packages:

sudo apt install build-essential cmake unzip pkg-config gfortran gcc g++ gperf flex texinfo gawk bison openssl pigz libncurses-dev autoconf automake tar figlet

Important

Make sure to install additional packages based on software you cross-compiling with CMAKE.

8. Setup Important Symlinks as follows: (Important)

Our toolchains requires few additional symbolic links to work properly. Therefore, to create all required symbolic link reliably, we need to download SSymlinker bash script as follows:

wget https://raw.githubusercontent.com/abhiTronix/raspberry-pi-cross-compilers/master/utils/SSymlinker

Once it is downloaded, you just need to make it executable, and then run it for each path manually using the following commands:

sudo chmod +x SSymlinker
./SSymlinker -s /usr/include/aarch64-linux-gnu/asm -d /usr/include
./SSymlinker -s /usr/include/aarch64-linux-gnu/gnu -d /usr/include
./SSymlinker -s /usr/include/aarch64-linux-gnu/bits -d /usr/include
./SSymlinker -s /usr/include/aarch64-linux-gnu/sys -d /usr/include
./SSymlinker -s /usr/include/aarch64-linux-gnu/openssl -d /usr/include
./SSymlinker -s /usr/lib/aarch64-linux-gnu/crtn.o -d /usr/lib/crtn.o
./SSymlinker -s /usr/lib/aarch64-linux-gnu/crt1.o -d /usr/lib/crt1.o
./SSymlinker -s /usr/lib/aarch64-linux-gnu/crti.o -d /usr/lib/crti.o

That's it for Raspberry Pi setup.

Steps/Settings for Host Machine (PC/Laptop)

Now Raspberry Pi Side all setup, Let's focus on commands for our Host Machine, i.e. PC/Laptop, where you going to cross-compile software binaries (Hello world application in this case) for your Raspberry Pi.

Important

Make sure your Raspberry Pi and this Host machine (where you cross-compiling) MUST be on the SAME Network.

1. Update the Host Machine

First of all, Run the following commands to update your system and install important dependencies:

sudo apt update
sudo apt dist-upgrade
sudo apt install build-essential cmake unzip gfortran
sudo apt install gcc g++ git bison python gperf pkg-config gdb-multiarch wget rsync
sudo apt install gperf flex texinfo gawk bison openssl pigz libncurses-dev autoconf automake tar figlet

2. Setting up the directory structure

You can use these following commands to create "cmake-test" to use as workspace for the project:

sudo mkdir ~/cmake-test
sudo mkdir ~/cmake-test/tools
sudo mkdir ~/cmake-test/build
sudo mkdir ~/cmake-test/rootfs
sudo mkdir ~/cmake-test/rootfs/usr
sudo mkdir ~/cmake-test/rootfs/opt
sudo chown -R 1000:1000 ~/cmake-test
cd ~/cmake-test

Note

Ensure the last command should have changed your current directory to ~/cmake-test. If not, run the last line again to make sure you are inside it, as the next steps assume you're running your commands from this directory.

3. Download & Extract the Precompiled Cross-Compiler

Let's first change into tools directory for downloading our Precompiled Cross-compiler with the following command:

cd ~/cmake-test/tools

Caution

Ensure the last command should have changed your current directory to ~/cmake-test/tools now. If not, run it again.

A. Copy Binary URL:

Copy URL from one of following Precompiled Compressed Base (or Pre-installed) Toolchain version (for maximum compatibility) based on your Raspberry Pi Variant and OS you installed on it from below:

Warning

The Stretch (Debian Version 9) 32-bit/64-bit toolchains are no longer supported!

Toolchains	Host OS	Target OS	Current Status	Precompiled GCC versions available
Raspberry Pi GCC 64-Bit Cross-Compiler Toolchains (Buster)	any x64/x86 Linux machine	Buster 64-bit OS (Debian Version 10) only	Stable/Production	8.3.0
Raspberry Pi GCC 64-Bit Cross-Compiler Toolchains (Bullseye)	any x64/x86 Linux machine	Bullseye 64-bit OS (Debian Version 11) only	Stable/Production	10.2.0
Raspberry Pi GCC 64-Bit Cross-Compiler Toolchains (Bookworm)	any x64/x86 Linux machine	Bookworm 64-bit OS (Debian Version 12) only	Stable/Production	12.2.0

Tip

You can also use the latest cross-compiler binaries instead. But they are not tested or recommended.

B. Download Binary

After that, paste your copied URL and run the following command to download the Cross-compiler:

wget <Copied Binary URL goes here> #for e.g. => wget https://sourceforge.net/projects/raspberry-pi-cross-compilers/files/Bonus%20Raspberry%20Pi%20GCC%2064-Bit%20Toolchains/Raspberry%20Pi%20GCC%2064-Bit%20Cross-Compiler%20Toolchains/Bookworm/GCC%2012.2.0/Raspberry%20Pi%201%2C%20Zero/cross-gcc-12.2.0-pi_64.tar.gz

C. Extract Binary:

Once it is downloaded, we can extract it using the following command:

tar xf cross-gcc-*.tar.gz

4. Sync Raspberry Pi rootfs (Most Important)

First, let's move back into the cmake-test folder as needed for the next sections:

cd ~/cmake-test

Now, we need to sync up our rootfs folder with the system files from the Raspberry Pi. We will be using rsync that let us sync (i.e. copy) files from the Raspberry Pi with appropriate permission onto your Host Machine, potentially saving you a lot of time.

To do this, enter the following commands one by one into your terminal (change 192.168.1.47 with the IP address of your Raspberry Pi present on the same network):

Command 1: rsync -avz --rsync-path="sudo rsync" --delete [email protected]:/lib rootfs
Command 2: rsync -avz --rsync-path="sudo rsync" --delete [email protected]:/usr/include rootfs/usr
Command 3: rsync -avz --rsync-path="sudo rsync" --delete [email protected]:/usr/lib rootfs/usr
Command 4: rsync -avz --rsync-path="sudo rsync" --delete [email protected]:/opt/vc rootfs/opt

Caution

Double check after each of the above commands that all the files have been copied to ~/cmake-test/rootfs folder. There will be an information message if there were any issues.

5. Fix symbolic links

The files we copied in the previous step still have symbolic links pointing to the file system on the Raspberry Pi. We need to alter this so that they become relative links from the new rootfs directory on the host machine. We can do this with a python script, which we can download as follows:

wget https://raw.githubusercontent.com/abhiTronix/rpi_rootfs/master/scripts/sysroot-relativelinks.py

Once it is downloaded, you just need to make it executable and run it, using the following commands:

sudo chmod +x sysroot-relativelinks.py
./sysroot-relativelinks.py rootfs

Build Dummy Hello-World Project

First, let's move back into the cmake-test folder as needed for the next sections:

cd ~/cmake-test

Now we need to create suitable files for our CMAKE project.

1. Create C++ `main.cpp` file:

Let’s try to compile and run a C++17 code that uses an if block with init-statement (the example is a bit simple, but will show you how to compile C++17 programs):

#include <iostream>

int main() {
  // if block with init-statement:
  if (int a = 5; a < 8) {
    std::cout << "Local variable a is < 8\n";
  } else {
    std::cout << "Local variable a is >= 8\n";
  }
  return 0;
}

2. Create `PI.cmake` CMAKE File

Let's create a typical cross-compiling toolchain that has content such as:

Warning

Remember to replace Cross-Compiler Toolchain absolute path with yours in following CMAKE file.

set(CMAKE_VERBOSE_MAKEFILE ON)
set(CMAKE_SYSTEM_NAME Linux)
set(CMAKE_SYSTEM_VERSION 1)
set(CMAKE_SYSTEM_PROCESSOR aarch64)

set(tools <path to toolchain. for e.g $ENV{HOME}/cmake-test/tools/cross-pi-gcc-12.2.0-64>) # warning change toolchain path here.
set(rootfs_dir $ENV{HOME}/cmake-test/rootfs)

set(CMAKE_FIND_ROOT_PATH ${rootfs_dir})
set(CMAKE_SYSROOT ${rootfs_dir})

set(CMAKE_LIBRARY_ARCHITECTURE aarch64-linux-gnu)
set(CMAKE_EXE_LINKER_FLAGS "${CMAKE_EXE_LINKER_FLAGS} -fPIC -Wl,-rpath-link,${CMAKE_SYSROOT}/usr/lib/${CMAKE_LIBRARY_ARCHITECTURE} -L${CMAKE_SYSROOT}/usr/lib/${CMAKE_LIBRARY_ARCHITECTURE}")
set(CMAKE_C_FLAGS "${CMAKE_CXX_FLAGS} -fPIC -Wl,-rpath-link,${CMAKE_SYSROOT}/usr/lib/${CMAKE_LIBRARY_ARCHITECTURE} -L${CMAKE_SYSROOT}/usr/lib/${CMAKE_LIBRARY_ARCHITECTURE}")
set(CMAKE_CXX_FLAGS "${CMAKE_CXX_FLAGS} -fPIC -Wl,-rpath-link,${CMAKE_SYSROOT}/usr/lib/${CMAKE_LIBRARY_ARCHITECTURE} -L${CMAKE_SYSROOT}/usr/lib/${CMAKE_LIBRARY_ARCHITECTURE}")

## Compiler Binary 
SET(BIN_PREFIX ${tools}/bin/aarch64-linux-gnu)

SET (CMAKE_C_COMPILER ${BIN_PREFIX}-gcc)
SET (CMAKE_CXX_COMPILER ${BIN_PREFIX}-g++ )
SET (CMAKE_LINKER ${BIN_PREFIX}-ld 
            CACHE STRING "Set the cross-compiler tool LD" FORCE)
SET (CMAKE_AR ${BIN_PREFIX}-ar 
            CACHE STRING "Set the cross-compiler tool AR" FORCE)
SET (CMAKE_NM {BIN_PREFIX}-nm 
            CACHE STRING "Set the cross-compiler tool NM" FORCE)
SET (CMAKE_OBJCOPY ${BIN_PREFIX}-objcopy 
            CACHE STRING "Set the cross-compiler tool OBJCOPY" FORCE)
SET (CMAKE_OBJDUMP ${BIN_PREFIX}-objdump 
            CACHE STRING "Set the cross-compiler tool OBJDUMP" FORCE)
SET (CMAKE_RANLIB ${BIN_PREFIX}-ranlib 
            CACHE STRING "Set the cross-compiler tool RANLIB" FORCE)
SET (CMAKE_STRIP {BIN_PREFIX}-strip 
            CACHE STRING "Set the cross-compiler tool RANLIB" FORCE)

set(CMAKE_FIND_ROOT_PATH_MODE_PROGRAM NEVER)
set(CMAKE_FIND_ROOT_PATH_MODE_LIBRARY ONLY)
set(CMAKE_FIND_ROOT_PATH_MODE_INCLUDE ONLY)

3. Create `CMakeLists.txt` File

Finally create a suitable CMakeLists.txt file:

cmake_minimum_required(VERSION 3.10)
project(cmake_test)
set(CMAKE_VERBOSE_MAKEFILE ON)
set(CMAKE_CXX_STANDARD 17)
add_executable(cmake_test main.cpp)

4. Configure CMAKE Project

Let's move into the build directory for further steps, as we don't want to build within that source directory as its crowded, so we will access it from within this this directory:

cd ~/cmake-test/build

Caution

Ensure you are still in the ~/cmake-test/build directory.

Finally, Now we can configure our Hello-World CMAKE Project as follows:

cmake -DCMAKE_TOOLCHAIN_FILE=~/cmake-test/PI.cmake  -DCMAKE_BUILD_TYPE=Debug ..

5. Finally Build Project

Our build has been configured now, and it is time to actually build the source files, and run the following command:

make -j$(nproc)

Note

The -j$(nproc) option indicates that the job should be spread into multiple threads and run in parallel on available cores.

and your compiled files will be available at ~/cmake-test/build.

Support This Project ❤️

If these binaries helped you big time, please consider supporting it. Thank you.

Cross Compiler CMake Usage Guide with rsynced Raspberry Pi 64 bit OS

Cross-Compiler CMake Usage Guide : Rsynced Raspberry Pi 64-bit OSes Sysroot

Prerequisites:

A. Hardware:

B. Software:

C. Others:

Steps/Settings for Target Machine (Raspberry Pi):

1. Start from Scratch (Optional)

1.1. Download Softwares & Prepare the SD card

1.2 Set up Network

2. Set up SSH

3. Open Terminal

4. Enable Development Sources

5. Update the system

6. Enable rsync with elevated rights

7. Install the important Development Packages

8. Setup Important Symlinks as follows: (Important)

Steps/Settings for Host Machine (PC/Laptop)

1. Update the Host Machine

2. Setting up the directory structure

3. Download & Extract the Precompiled Cross-Compiler

A. Copy Binary URL:

B. Download Binary

C. Extract Binary:

4. Sync Raspberry Pi rootfs (Most Important)

5. Fix symbolic links

Build Dummy Hello-World Project

1. Create C++ main.cpp file:

2. Create PI.cmake CMAKE File

3. Create CMakeLists.txt File

4. Configure CMAKE Project

5. Finally Build Project

Support This Project ❤️

Home 🏠

Toolchains Binaries Setup Guide

Beta/Experimental

Advanced Topics for Devs

Miscellaneous

Clone this wiki locally

1. Create C++ `main.cpp` file:

2. Create `PI.cmake` CMAKE File

3. Create `CMakeLists.txt` File