*

Yocto and OpenEmbedded at Collabora

Posted on 15/04/2016 by Andrew Shadura

How the use of Yocto and OpenEmbedded helps corporations migrate to free software

There’s a certain confusion existing even among people closely working with Yocto Project, on what exactly Yocto is. First of all, Yocto isn’t a Linux distribution. In fact, Yocto Project is an umbrella organisation that takes care of a bunch of embedded Linux technologies, including OpenEmbedded Core, BitBake, Poky and others. These and others technologies Yocto Project provides allow users to build custom Linux distributions suited to their own needs.

One might ask, if Yocto isn’t a distribution, how does one make a distribution using its technologies?

Figure 1. OpenEmbedded Architecture Workflow.


The current Yocto technology stack has evolved from its roots in previously separate OpenEmbedded Project. Since the merger of OpenEmbedded and Yocto, OpenEmbedded has introduced a layers system allowing vendors and users to have their bits separate yet plugging into each other. There’s a number of layers Yocto Project provides (oe-core, meta-yocto, meta-yocto-bsp) which form so-called ‘reference distribution’, Poky. Poky contains foundation package recipes (from OpenEmbedded Core), distribution policy configuration, reference BSPs, build tools and documentation. Normally, Poky is what you start from when creating your own distribution.

Figure 2. Layered architecture of Poky reference distribution.


Package recipes are written in a language similar to both Make and shell, and to certain extent resemble Gentoo’s ebuilds. These recipes are being used by BitBake to build binary packages, and provide necessary information on where sources are found, and how to build them. BitBake recipes support includes, inheritance and overrides making it easy to change or extend the behaviour of an existing recipe without touching its code.

SUMMARY = "The canonical example of init scripts"
SECTION = "base"
LICENSE = "GPLv2"
LIC_FILES_CHKSUM = "file://${WORKDIR}/COPYRIGHT;md5=349c872e0066155e1818b786938876a4"

SRC_URI = "file://skeleton \
           file://skeleton_test.c \
           file://COPYRIGHT\
           "

do_compile () {
        ${CC} ${WORKDIR}/skeleton_test.c -o ${WORKDIR}/skeleton-test
}

do_install () {
        install -d ${D}${sysconfdir}/init.d
        cat ${WORKDIR}/skeleton | \
          sed -e 's,/etc,${sysconfdir},g' \
              -e 's,/usr/sbin,${sbindir},g' \
              -e 's,/var,${localstatedir},g' \
              -e 's,/usr/bin,${bindir},g' \
              -e 's,/usr,${prefix},g' > ${D}${sysconfdir}/init.d/skeleton
        chmod a+x ${D}${sysconfdir}/init.d/skeleton
 
        install -d ${D}${sbindir}
        install -m 0755 ${WORKDIR}/skeleton-test ${D}${sbindir}/
}

RDEPENDS_${PN} = "initscripts"
CONFFILES_${PN} += "${sysconfdir}/init.d/skeleton"
Figure 3. Example of a BitBake recipe.

 

SUMMARY = "Example of how to build an external Linux kernel module"
LICENSE = "GPLv2"
LIC_FILES_CHKSUM = "file://COPYING;md5=12f884d2ae1ff87c09e5b7ccc2c4ca7e"

inherit module

SRC_URI = "file://Makefile \
           file://hello.c \
           file://COPYING \
          "

S = "${WORKDIR}"
Figure 4. Example of a BitBake recipe with inheritance.

 

RDEPENDS_${PN}_append = " systemd"
Figure 5. Example of .bbappend.


Collabora is a company that provides consultancy to companies who are deploying open source technologies in their products, by providing its own open source based products and through knowledge sharing activities such as training. Collabora employs many free software developers who are experts or major developers in such areas as multimedia (GStreamer), graphics (Wayland, Weston), Linux kernel, productivity software (LibreOffice) and others.

At Collabora, we use Yocto on a project for a manufacturer of medical equipment, who use a Linux-based operating system in their products. Currently, their production devices are using a very custom Buildroot-based system, which runs quite an old (3.x-something) version of Linux kernel with lots of custom proprietary daemons and APIs. At some point they realised that it’s quite a difficult task to support that system, and they decided they need help of experts, us.

For the project, the customer have decided to eliminate as much as possible custom proprietary libraries, take as much work as possible upstream, migrate to and integrate systemd as the init system. They also wanted us to help them with development methodology.

Thanks to the layer system Yocto Project uses, building your own distribution based on it is quite an easy job, once you know what you need from it. It all started as a layer with three files: the layer definition itself, distribution configuration file, and a machine definition.

# We have a conf and classes directory, add to BBPATH
BBPATH .= ":${LAYERDIR}"

# We have recipes-* directories, add to BBFILES
BBFILES += "${LAYERDIR}/recipes-*/*/*.bb \
       ${LAYERDIR}/recipes-*/*/*.bbappend"

BBFILE_COLLECTIONS += "distro-core"
BBFILE_PATTERN_distro-core = "^${LAYERDIR}/"
BBFILE_PRIORITY_distro-core = "10"
Figure 6. Layer definition file (conf/layer.conf).

 

require conf/distro/poky.conf

DISTRO = "distro-core"
DISTRO_NAME = "Core Distro Platform"
DISTRO_VERSION = "2.0"
DISTRO_CODENAME = "badger"

DISTRO_FEATURES_append = " systemd wayland xwayland xattr pam apparmor"
DISTRO_FEATURES_remove = "x11"

# Enable systemd as init
VIRTUAL-RUNTIME_init_manager = "systemd"

# Disable sysv init and prevent any init scripts in the images
DISTRO_FEATURES_BACKFILL_CONSIDERED = "sysvinit"
VIRTUAL-RUNTIME_initscripts = ""

PREFERRED_PROVIDER_jpeg = "jpeg"
PREFERRED_PROVIDER_jpeg-native = "jpeg-native"
Figure 7. Distribution definition file (conf/distro/badger.conf).

 

#@NAME: vexpress-a9 machine
#@DESCRIPTION: Machine configuration for the vexpress a9 board 

PREFERRED_PROVIDER_virtual/xserver = "xserver-xorg"

# Ship all kernel modules by default
MACHINE_EXTRA_RRECOMMENDS = " kernel-modules"

# Allow for MMC booting (required by the NAND-less)
EXTRA_IMAGEDEPENDS += ""

# Uncomment the following line to enable the hard floating point abi. Note that
# this breaks some binary libraries and 3D (neither of which ship with
# meta-yocto). For maximum compatibility, leave this disabled.
#DEFAULTTUNE ?= "cortexa8hf-neon"
include conf/machine/include/tune-cortexa9.inc

#IMAGE_CLASSES += "sdcard_image"

#IMAGE_FSTYPES += "tar.bz2 ext3 vexpressa9-sdimg"
IMAGE_FSTYPES += "tar.bz2 ext3"
#EXTRA_IMAGECMD_jffs2 = "-lnp "

# 2.6.37 and later kernels use OMAP_SERIAL, ttyO2
# earlier kernels use ttyS2
SERIAL_CONSOLE = "115200 ttyO2"

PREFERRED_PROVIDER_virtual/kernel ?= "linux-yocto"

KERNEL_IMAGETYPE = "zImage"

UBOOT_MACHINE = "ca9x4_ct_vxp_config"
UBOOT_ENTRYPOINT = "0x80008000"
UBOOT_LOADADDRESS = "0x80008000"
KERNEL_EXTRA_ARGS += "LOADADDR=${UBOOT_ENTRYPOINT}"

MACHINE_FEATURES = "kernel26 apm usbgadget usbhost vfat alsa"
Figure 8. Machine definition file (conf/machine/vespressa9.conf).


These files define where recipe files are to be found, exactly what features (systemd, wayland, apparmor) we’re using and what we don’t (x11, sysvinit), and what processor architectures we’re building for, what types of images we need to generate and so on.

For architecture-dependent parts, we first used a layer Freescale provided (meta-fsl-arm, meta-fsl-arm-extra), so we wouldn’t need to write our own image generation routines, or tuning the cross-compiler features by hand. Later, we removed that dependency by bundling greatly simplified and customised versions of Freescale’s recipes, but until we needed that it was a great help that we could reuse already working code.

When the initial phase of the project was completed, and we had a working image booting on a hardware prototype with Weston shell running, it was decided to split our layer in two, thus separating the platform itself and the application layer. Into the application layer went customer’s proprietary software that will remain proprietary — at least, for now, and supporting daemons and libraries it depends on. The platform layer is almost entirely free software, with the exception of a few legacy hardware-related daemons which will be at some point replaced with their free software counterparts.

One might ask, why do we need anything in the platform layer apart from the distro configuration, if it’s free software anyway? The answer is that, we’re using quite some bleeding edge technology packages, but at the same time we want our platform to be based on the stable branch of Poky, the Yocto meta-distribution. That means, from time to time we need to import recipes for newer versions of software from the development branch. This is especially true when recipes coming from Yocto need to be improved: while BitBake allows extending existing recipes with use of .bbappend files, we mostly use that for distribution-specific things only. If we need some generic change that can be upstreamed, like user sessions support in systemd and dbus, we copy the latest version of the recipe from upstream, and patch it locally, so that the fixes can be easily submitted upstream.

This brings a benefit of needing minimal edits to the patches before they’re submitted; otherwise we’d need a complete rewrite of the feature we need.

Apart from software updates and patches we also carry recipes for some free software which isn’t release-ready, some custom configuration and some temporary workarounds for kernel bugs we don’t currently have capacity to fix properly.

So far, since the project began, our team has contributed to the community at least the following:

  • many fixes to various OE Core package recipes, including patches enabling systemd integration, merged /usr;
  • patches to Weston and Linux kernel enabling accelerated graphics on our hardware;
  • patches to ifupdown adding inheritance feature similar to what BitBake has;
  • and much more.

Kernel support for the customer’s hardware is all being upstreamed by the customer, as the customer believes their hardware should run mainline kernel.

Even though the project is still in progress, even at its current stage it clearly demonstrates how is possible to migrate a big project to an open platform, reducing the maintenance cost and at the same time helping everyone else, and to a great extent that is possible thanks to Yocto.

Licensed under Creative Commons Attribution-ShareAlike 3.0 license.

 

Original post

Comments (0)


Add a Comment





Allowed tags: <b><i><br>Add a new comment:


Latest Blog Posts

ipcpipeline: Splitting a GStreamer pipeline into multiple processes

17/11/2017

Earlier this year I worked on a certain GStreamer plugin that is called “ipcpipeline”. This plugin provides elements that make it possible…

Quick hack: Experiments with crosvm

09/11/2017

Running crosvm outside Chromium OS is quite easy, with the only complication being that minijail isn't widely packaged in distros. In these…

Tracing memory leaks in the NFC Digital Protocol stack

06/11/2017

Kmemleak allows you to track possible memory leaks inside the Linux kernel. Basically, it tracks dynamically allocated memory blocks in…

Who knew we still had low-hanging fruit?

17/10/2017

Earlier this month I had the pleasure of attending the Web Engines Hackfest, hosted by Igalia at their offices in A Coruña, and also sponsored…

Performance analysis in Linux (continued)

06/10/2017

In this post, I will show one more example of how easy it is to disrupt performance of a modern CPU, and also run a quick discussion on…

XDC 2017 - Links to recorded presentations (videos)

23/09/2017

Many thanks to Google for recording all the XDC2017 talks. To make them easier to watch, here are direct links to each talk recorded at…

Open Since 2005

We use cookies on this website to ensure that you get the best experience. By continuing to use this website you are consenting to the use of these cookies. To find out more please follow this link.

Collabora Ltd © 2005-2017. All rights reserved. Website sitemap.