This guide serves as a basic introduction to using the Yocto project for building a custom Linux image for the Raspberry Pi. The Singularity containerisation engine is bundled within this image as an example, and because it is a nifty piece of software.
In writing this, I assume the reader has some working knowledge of using the Linux command-line. I strongly recommend keeping your work inside a git repository: being able to revert back to a known-working version was a lifesaver on multiple occasions.
What is Singularity?
In a nutshell, Singularity is a container platform built on the principle of mobility of compute. It is designed to be used on HPC clusters and, unlike Docker, it does not require root access to mount an image. In addition, it can use Docker images out-of-the-box and it can pull them from the Docker Hub. For more information, see the Singularity website.
Containers are a solution to the ever-present dependency problem: how do you make sure that the user has all of the software needed to run the program you are shipping? In general terms, containers work by bundling a specific operating system, alongside other necessary software, and running the target program using them.
While Docker has become the most-used containerisation platform, Singularity is interesting for a couple of reasons:
- It does not require root access for the most common workflows. This means that users can simply add their image to a high-performance computing cluster and do the necessary work without needing an administrator password.
- Singularity uses the Docker image format, which means that it works with the myriad of available bundles on the Docker Hub.
Therefore, a computer running Singularity on top of a minimal Linux build can host virtually any software, regardless of its dependencies or even the operating system the software was built for, as long as you can make or obtain a working image.
What is the Yocto Project?
The Yocto Project is an open-source, mostly MIT-licensed collaboration that makes it easier to create and build Linux-based embedded systems. It offers a mature, fully automated Linux build system, supporting all major embedded architectures, alongside recipes for various commonly-used software.
Unifying the world of embedded Linux is a daunting task. The Yocto Project has done a stellar job of it, but it is not user-friendly. Things can and oftentimes do go wrong. Errors in a file can create errors of an apparently unrelated origin. Thankfully, the Yocto user manual provides immense detail concerning common tasks, and the Yocto Project mailing lists are publicly available and can be of great help.
Getting started with Yocto
If you are new to the Yocto Project, I strongly recommend reading the entirety of the Yocto Project Quickstart and following its steps until you can successfully emulate a basic image using QEMU. By doing this, you will become familiar with some of the basic tasks, and if something goes wrong it will be significantly easier to find help online. Once this have succeeded, you will have a known working state onto which you can apply the steps within this guide.
Note: I prefer keeping the build directory and my layers outside the poky directory, in order to make changes easier to track using git submodules. This is my directory structure:
I initialise the build environment by running:
Contrast with the approach in the quickstart guide, where the build directory resides within poky and layers are cloned within the latter. I recommend following my approach, for consistency with the rest of this guide.
Creating your own image
In this section I will detail the process of writing an image using software from within the Yocto project. The end product will be a fully-featured Linux build containing the Singularity containerisation engine.
Obtaining the necessary layers
Since our target platform is the Raspberry Pi, you must have the Raspberry Pi BSP in our layer directory. Obtain it by cloning the repository:
The link to the remote used in the command can be found within the webpage linked above. For reference, here is the command used:
git clone git://git.yoctoproject.org/meta-raspberrypi</code]>
Once you have cloned a layer, you need to tell the build system to use it. You can do that using:
bitbake-layers add-layer ../meta-raspberrypi/
Here it is in action:
WARNING: Running the bitbake-layers tool outside of the build directory makes it fail and print out a massive stack trace. Make sure you are within the build directory and that you reference the layers from this directory.
Click here for further information about bitbake-layers.
Inspecting the README.md file of meta-raspberrypi reveals that some extra layers are required: the meta-oe, meta-multimedia, meta-python and meta-networking from within meta-openembedded. Clone the meta-openembedded layer using the remote in README.md and add the necessary layers using bitbake-layers add-layer. The order is important: Adding meta-networking before meta-python causes an error:
Thankfully, bitbake-layers told us exactly what is wrong. The error is fixed by adding meta-python before you add meta-networking:
We can see what layers we are currently using by running bitbake-layers show-layers . At this stage, you should have the same layers as shown below. Make sure you have all these layers before proceeding.
meta-virtualisation
The recipe for Singularity resides within meta-virtualisation. Clone this layer and add it to the build system as shown above. The README.md file reveals that this layer has further dependencies: openembedded-core, meta-filesystems, oe-meta-go and meta-selinux. The README.md file also provides links to the remote repositories of these dependencies, therefore adding them to your build should be straightforward.
NOTE: The dependency on openembedded-core is fulfilled by the layers automatically added from within poky. You should skip cloning and adding this layer. Poky can replace openembedded-core because it was developed off of OpenEmbedded.
Once you have added the layers required by meta-virtualisation, you should be left with this: 
Once you are at this stage, you are ready to create your own image. Important takeaways:
- Make sure you fully read the README files of any layer you plan on using, and that you obtain the layers it depends upon.
- Do not forget to add any layer you want to use by using bitbake-layers add-layer.
Writing the custom image recipe
In order to keep everything organised, and to make it easy to share the work, I like to keep my images inside a layer, even though you can obtain the same result by modifying local.conf. The easiest way to create a layer is by using the yocto-layer script:
yocto-layer create generates the minimally required files to create a layer, leaving it to the user to populate it with recipes.
Make sure you make the build system aware of the layer by using bitbake-layers add . Remember that you must be inside the build directory to successfully run the bitbake-layers script. Additionally, give a sensible name to your layer! The OpenEmbedded build system does not support file or directory names that contain spaces.
It is conventional to store image recipes under a recipes-core/images directory, inside the layer folder. Therefore, we will follow this convention. The image itself should be a .bb file. I have named mine ‘computenodes-image.bb’ . Here is what the directory structure should look like:
computenodes-image.bb must contain the following lines:
SUMMARY = "Basic image, containing Singularity"
LICENSE = "MIT"
# Base the image upon a mostly complete Linux build
include recipes-extended/images/core-image-full-cmdline.bb
# Install the Singularity containerization platform
# Also install kernel modules, as detailed in the BSP that is being used
IMAGE_INSTALL_append = " \
singularity \
kernel-modules \
"
# Allocate ~1 extra GB to pull containers
IMAGE_ROOTFS_EXTRA_SPACE = "1000000"
This recipe is based upon rpi-hwup-image from meta-raspberrypi. Files preceded by a ‘#’ are comments. The important changes are the fact that it includes core-image-full-cmdline, as opposed to core-image-minimal. This change provides a more complete Linux environment, as opposed to a system which is not capable of doing much more than booting. We are additionally installing Singularity and allocating more SD card space for the containers.
The SUMMARY variable is meant to succinctly describe the product of the recipe. The LICENSE variable specifies the license under which the source code / recipe is licensed. It must be included inside any recipe. Here is some more general information about how OpenEmbedded handles licensing.
The include keyword includes all of the text of the mentioned recipe inside the file. BitBake has multiple ways of sharing functionality between files. Additionally, IMAGE_INSTALL must be used with care. While IMAGE_ROOTFS_EXTRA_SPACE is more or less self-explanatory, the Yocto documentation has a section on it.
Building the image
To inform the build system that you want to target a Raspberry Pi, you must add the following line to build/conf/local.conf:
MACHINE = "raspberrypi3"
Now, you are finally ready to build. Run bitbake computenodes-image and, after waiting for the build to be complete, you can find the end product under build/tmp/deploy/images/raspberrypi3 . In my case, the image is called computenodes-image-raspberrypi3.rpi-sdimg . You can then flash the image using Etcher. If Etcher does not recognise the file as a valid OS image, change .rpi-sdimg to .img inside the filename. Alternatively, you can achieve the same result with a symbolic link to computenodes-image-raspberrypi3.rpi-sdimg if the link’s filename ends with .img .
Testing the image
Obtain access to the terminal by plugging in a monitor and keyboard or, if you know the IP address of the Pi, you can SSH into it. Once you’re there, you can test Singularity:
If the selftest succeeds, try pulling a container from the Docker hub:
It works! Let us open a shell inside the container.
Another example:
It is clear that the environment inside the container is different from the one outside.
Conclusion
We have looked at creating a basic image using the Yocto project. The process involves cloning the necessary layers and their dependencies, creating a new layer, writing the image recipe and, finally, building it. Our test image successfully booted, and the Singularity containerisation engine works without a flaw.