Yocto - read-only rootfs (squashfs) + ext4 overlay (qemu)

Experimenting with a qemu yocto build that models an Embedded system with a read-only rootfs, and writeable overlay.

Basic yocto setup

Clone git://git.yoctoproject.org/poky

Checkout dunfell branch

Clone https://github.com/marcusfolkesson/meta-readonly-rootfs-overlay.git

Update the compatibility in meta-readonly-rootfs-overlay/conf/layer.conf to match your poky branch

LAYERSERIES_COMPAT_readonly-rootfs-overlay = "honister kirkstone dunfell"

source oe-init-build-env

Add meta-readonly-rootfs-overlay to build/conf/bblayers.conf

Create own layer meta-mylayer (git@github.com:bugwhine/meta-mylayer.git)

and add to build/conf/bblayers.conf

Setup meta-mylayer/conf/layer.conf containing the boilerplate -

# We have a conf and classes directory, add to BBPATH

BBPATH .= ":${LAYERDIR}"

# We have recipes-* directories, add to BBFILES

BBFILES += "${LAYERDIR}/recipes-*/*/*.bb \

            ${LAYERDIR}/recipes-*/*/*.bbappend"

BBFILE_COLLECTIONS += "mylayer"

BBFILE_PATTERN_mylayer = "^${LAYERDIR}/"

BBFILE_PRIORITY_mylayer = "5"

LAYERVERSION_mylayer = "4"

LAYERSERIES_COMPAT_mylayer = "dunfell"

Add squashfs support to the yocto kernel

$ bitbake linux-yocto -c kernel_configme -f

$ bitbake linux-yocto -c menuconfig

Use menuconfig to add squashfs support, and customize options as desired

$ bitbake linux-yocto -c diffconfig
$ mkdir -p meta-mylayer/recipes-kernel/linux
$ mv fragment.cfg ../meta-mylayer/recipes-kernel/linux/linux-yocto/squashfs.cfg

Create a bbappend file meta-mylayer/recipes-kernel/linux/linux-yocto_%.bbappend referencing the fragment

FILESEXTRAPATHS:prepend := "${THISDIR}/${PN}:"

SRC_URI += "file://squashfs.cfg"

Build the kernel 

$ bitbake linux-yocto

And confirm that the configuration is present

$ grep SQUASHFS `find tmp/work/qemux86_64-poky-linux/linux-yocto/ -name '.config'`

CONFIG_SQUASHFS=y

...

CONFIG_SQUASHFS_FRAGMENT_CACHE_SIZE=3

Setup an image (wic)

Create an image recipe meta-mylayer/recipes-core/images/my-image-wic.bb with content (based on core-image-full-cmdline)

DESCRIPTION = "A console-only image with more full-featured Linux system \

functionality installed."

SRC_URI = "file://${FILE_DIRNAME}/${BPN}.wks"

IMAGE_FEATURES += "splash ssh-server-openssh"

IMAGE_INSTALL = "\

    packagegroup-core-boot \

    packagegroup-core-full-cmdline \

    ${CORE_IMAGE_EXTRA_INSTALL} \

    initscripts-readonly-rootfs-overlay \

    "

IMAGE_FSTYPES = "wic"

inherit core-image

WKS_FILE = "my-image-wic.wks"

WKS_FILE_DEPENDS = "dosfstools-native mtools-native gptfdisk-native squashfs-tools-native"

WKS_FILE_DEPENDS_append_x86 = " syslinux-native syslinux"

WKS_FILE_DEPENDS_append_x86-64 = " syslinux-native syslinux"

WKS_FILE_DEPENDS_append_x86-x32 = " syslinux-native syslinux"

QB_KERNEL_CMDLINE_APPEND += "root=/dev/vda1 rootrw=/dev/vda2 rootrwoptions=rw,noatime init=/init"

QB_DEFAULT_FSTYPE = "wic"

QB_FSINFO = "wic:no-kernel-in-fs"

#QB_KERNEL_ROOT = "/dev/vda1"

Create the wks (wick kickstart script) meta-mylayer/recipes-core/images/my-image-wic.wks with content - 

part / --source rootfs --ondisk sda --fstype=squashfs --align 1024

part /media/rfs/rw --ondisk sda --fstype=ext4 --label rwoverlay --size 100M

Build and run

$ bitbake my-image-wic
...

$ runqemu my-image-wic nographic kvm

...

Booting from ROM...

[    0.000000] Linux version 5.4.230-yocto-standard (oe-user@oe-host) (gcc version 9.5.0 (GCC)) #1 SMP PREEMPT Wed Feb 1 15:40:31 UTC 2023

[    0.000000] Command line: root=/dev/vda rw  console=ttyS0 mem=256M ip=192.168.7.2::192.168.7.1:255.255.255.0 oprofile.timer=1 root=/dev/vda1 rootrw=/dev/vda2 rootrwoptions=rw,noatime init=/ini

...

Poky (Yocto Project Reference Distro) 3.1.24 qemux86-64 ttyS0

qemux86-64 login: root

Check the rootfs mounts  

root@qemux86-64:~# lsblk

NAME   MAJ:MIN RM   SIZE RO TYPE MOUNTPOINT

hdc     22:0    1     4G  0 disk

vda    253:0    0 126.3M  0 disk

|-vda1 253:1    0  25.3M  0 part /media/rfs/ro

`-vda2 253:2    0   100M  0 part /media/rfs/rw

root@qemux86-64:~# cat /proc/mounts

...

/dev/root /media/rfs/ro squashfs ro,relatime 0 0

/dev/vda2 /media/rfs/rw ext4 rw,noatime 0 0

overlay / overlay rw,relatime,lowerdir=/media/rfs/ro,upperdir=/media/rfs/rw/upperdir,workdir=/media/rfs/rw/work 0 0

...

Confirm that the overlay is working

root@qemux86-64:~# touch /etc/aaa

root@qemux86-64:~# ls /media/rfs/rw/upperdir/etc/aaa

/media/rfs/rw/upperdir/etc/aaa

root@qemux86-64:~# ls /media/rfs/ro/etc/aaa

ls: cannot access '/media/rfs/ro/etc/aaa': No such file or directory