Friday, May 4, 2018

Yocto kernel in-tree development work-flows

The Yocto Project Linux Kernel Development Manual, covers at least two flows for kernel development - 'Traditional Kernel Development' and 'devtool'. Here, I'll present some experiences with both these methods, along with a third method that I found convenient.

Traditional Kernel Development
Preparation
- create a layer for holding kernel patches, and configuration fragments
- create an append recipe (eg. meta-mylayer/recipes-kernel/linux/linux-yocto_4.12.bbappend) that identifies the patches and configuration fragments that you wish to apply
- create a local clone of the Yocto Linux Kernel

Development Loop
- Make code and/or configuration changes in local clone of Yocto Linux Kernel
- Stage and commit the changes (note this is necessary to have them included in the build)
- Adjust conf/local.conf to point to your local kernel clone
- Build and test the changes
- Generate a patch/config fragment
- Move the patch file to your layer, and update the .bbappend file to use it

Reflecting on the flow
Despite the efficiency of git's 'amend commit' function, the loop is quite heavy/time consuming.
For example, if you find the need to add a few printk debug statements, for troubleshooting, committing them each time before building seems an unnecessary burden.

Devtool Kernel Development
Preparation
- create a layer for holding kernel patches
- create an append recipe (eg. meta-mylayer/recipes-kernel/linux/linux-yocto_4.12.bbappend) that identifies the patches that you wish to apply
- (build and install an extensible SDK)
- build a clean image
- checkout the kernel source using 'devtool modify linux-yocto'. This step creates a local copy of the kernel source, in the (SDK) workspace, as well a recipe to include it in the build

Development Loop
- Make code changes in the SDK workspace clone of the kernel source tree
- Build and test the changes
- Stage and commit the changes
- Use 'devtool finish' to generate patches, and include them in your layer

Reflecting on the flow
This flow address the burdens of the traditional flow, as the stage/commit/patch steps are moved outside the change-build-test loop. I found this works well for source code changes.

A limitation of the flow, is that configuration fragments and patches are 'locked' during the preparation step of the flow. Ie. if, whilst in the development loop, you wish to add a configuration fragment ('from the shelf'), you should exit the flow, add the fragment, and re-enter the flow by again completing the preparation steps.

Alternatively, one can manually apply the fragments/patches in the workspace - but doing so conflates these changes with those being made in the development loop.

External Source Kernel Development
Preparation
- create a local clone of the Yocto Linux Kernel
- create a layer for holding kernel patches, and configuration fragments
- create an append recipe (eg. meta-mylayer/recipes-kernel/linux/linux-yocto_4.12.bbappend) that identifies the patches and configuration fragments that you wish to apply
- in the .bbappend recipe add the following -
inherit externalsrc
EXTERNALSRC = "/repo/linux-custom"
SRCTREECOVEREDTASKS := "do_validate_branches do_kernel_checkout do_fetch"

Development Loop
- Make code and/or configuration changes in the kernel source tree
- Build and test the changes
- Stage and commit the changes
- Generate a patch/config fragment
- Move the patch file to your layer, and update the .bbappend file to use it
- Remove the references to the local clone of the Yocto Linux Kernel when done

Reflecting on the flow
This flow attempts to combine the best parts of the other two flows.

It 'gives back' the capability to add/remove patches/configuration fragments via the meta layer (aka. recipe), by overriding this definition in the kernel-yocto.bbclass -
SRCTREECOVEREDTASKS += "do_kernel_configme do_validate_branches do_kernel_configcheck do_kernel_checkout do_fetch do_unpack do_patch"

Thus allowing patches and config fragements to be applied by running the 'unpack' task when building. ie.
bitbake -C unpack linux-yocto

It retains the simple change-build-test loop of the devtool flow.
It sacrifices the automated update of the meta data, provided by 'devtool finish'.