We're hiring!
*

Introducing debos, a versatile images generator

Ana Guerrero López avatar

Ana Guerrero López
June 27, 2018

Share this post:

In Debian and derivative systems, there are many ways to build images. The simplest tool of choice is often debootstrap. It works by downloading the .deb files from a mirror and unpacking them into a directory which can eventually be chrooted into.

More often than not, we want to make some customization on this image, install some extra packages, run a script, add some files, etc

debos is a tool to make thiese kinds of trivial tasks easier. debos works using recipe files in YAML listing the actions you want to perform in your image sequentially and finally, choosing the output formats.

As opposite to debootstrap and other tools, debos doesn't need to be run as root for making actions that require root privileges in the images. debos uses fakemachine a library that setups qemu-system allowing you to work in the image with root privileges and to create images for all the architectures supported by qemu user. However, for this to work, make sure your user has permission to use /dev/kvm.

Let's see how debos works with a simple example. If we wanted to create an arm64 image for Debian Stretch customized, we would follow these steps:

  • debootstrap the image
  • install the packages we need
  • create an user
  • setup our preferred hostname
  • run a script creating an user
  • copy a file adding the user to sudoers
  • creating a tarball with the final image

This would translate into a debos recipe like this one:

{{- $architecture := or .architecture "arm64" -}}
{{- $suite := or .suite "stretch" -}}
{{ $image := or .image (printf "debian-%s-%s.tgz" $suite $architecture) }}

architecture: {{ $architecture }}

actions:
  - action: debootstrap
    suite: {{ $suite }}
    components:
      - main
    mirror: http://deb.debian.org/debian
    variant: minbase

  - action: apt
    recommends: false
    packages:
      - adduser
      - sudo

  - action: run
    description: Set hostname
    chroot: true
    command: echo debian-{{ $suite }}-{{ $architecture }} > /etc/hostname

  - action: run
    chroot: true
    script: scripts/setup-user.sh

  - action: overlay
    description: Add sudo configuration
    source: overlays/sudo

  - action: pack
    file: {{ $image }}
    compression: gz

(The files used in this example are available from this git repository)

We run debos on the recipe file:

$ debos simple.yaml

The result will be a tarball named debian-stretch-arm64.tar.gz. If you check the top two lines of the recipe, you can see that the recipe defaults to architecture arm64 and Debian stretch. We can override these defaults when running debos:

$ debos -t suite:"buster" -t architecture:"amd64" simple.yaml

This time the result will be a tarball debian-buster-amd64.tar.gz.

The recipe allows some customization depending on the parameters. We could install packages depending on the target architecture, for example, installing python-libsoc in armhf and arm64:

- action: apt
  recommends: false
  packages:
    - adduser
    - sudo
{{- if eq $architecture "armhf" "arm64" }}
    - python-libsoc
{{- end }}

What happens if in addition to a tarball we would like to create a filesystem image? This could be done adding two more actions to our example, a first action creating the image partition with the selected filesystem and a second one deploying the image in the filesystem:

- action: image-partition
  imagename: {{ $ext4 }}
  imagesize: 1GB
  partitiontype: msdos
  mountpoints:
    - mountpoint: /
      partition: root
  partitions:
    - name: root
      fs: ext4
      start: 0%
      end: 100%
      flags: [ boot ]

- action: filesystem-deploy
  description: Deploying filesystem onto image

{{ $ext4 }} should be defined in the top of the file as follows:

{{ $ext4 := or .image (printf "debian-%s-%s.ext4" $suite $architecture) }}

We could even make this step optional and make the recipe by default to only create the tarball and add the filesystem image only adding an option to debos:

$ debos -t type:"full" full.yaml

The final debos recipe will look like this:

{{- $architecture := or .architecture "arm64" -}}
{{- $suite := or .suite "stretch" -}}
{{ $type := or .type "min" }}
{{ $image := or .image (printf "debian-%s-%s.tgz" $suite $architecture) }}
{{ $ext4 := or .image (printf "debian-%s-%s.ext4" $suite $architecture) }}

architecture: {{ $architecture }}

actions:
  - action: debootstrap
    suite: {{ $suite }}
    components:
      - main
    mirror: http://deb.debian.org/debian
    variant: minbase

  - action: apt
    recommends: false
    packages:
      - adduser
      - sudo
{{- if eq $architecture "armhf" "arm64" }}
      - python-libsoc
{{- end }}

  - action: run
    description: Set hostname
    chroot: true
    command: echo debian-{{ $suite }}-{{ $architecture }} > /etc/hostname

  - action: run
    chroot: true
    script: scripts/setup-user.sh

  - action: overlay
    description: Add sudo configuration
    source: overlays/sudo

  - action: pack
    file: {{ $image }}
    compression: gz

{{ if eq $type "full" }}
  - action: image-partition
    imagename: {{ $ext4 }}
    imagesize: 1GB
    partitiontype: msdos
    mountpoints:
      - mountpoint: /
        partition: root
    partitions:
      - name: root
        fs: ext4
        start: 0%
        end: 100%
        flags: [ boot ]

  - action: filesystem-deploy
    description: Deploying filesystem onto image
{{end}}

debos also provides some other actions that haven't been covered in the example above:

  • download allows to download a single file from the internet
  • raw can directly write a file to the output image at a given offset
  • unpack can be used to unpack files from archive in the filesystem
  • ostree-commit create an OSTree commit from rootfs
  • ostree-deploy deploy an OSTree branch to the image

The example in this blog post is simple and short on purpose. Combining the actions presented above, you could also include a kernel and install a bootloader to make a bootable image. Upstream is planning to add more examples soon to the debos recipes repository.

debos is a project from Sjoerd Simons at Collabora, it's still missing some features but it's actively being developed and there are big plans for the future!


Visit Ana's blog.

Comments (8)

  1. Osqui:
    Jun 28, 2018 at 10:29 AM

    Excellent.
    Do you plan extend this to create not Debian-like images?
    Thanks!

    Reply to this comment

    Reply to this comment

    1. Ana:
      Jul 06, 2018 at 01:52 PM

      Hi Osqui!

      As the name hints, we're not planning to work on creating images for other distributions. However we'll be happy to add this if somebody send a pull request :)

      Cheers,
      Ana

      Reply to this comment

      Reply to this comment

    2. Mrfai:
      Jul 08, 2018 at 11:58 PM

      The Debian cloud team is using the FAI software (https://fai-project.org) for different cloud images and has it's configuration at https://salsa.debian.org/cloud-team/fai-cloud-images. FAI can create installation images and bootable disk images for other distributions like Ubuntu, CentOS, Scientific Linux.

      Reply to this comment

      Reply to this comment

  2. Rich:
    Jun 28, 2018 at 11:57 AM

    Why didn't you use virt-builder? It does all of this already.

    Reply to this comment

    Reply to this comment

      1. Paul Menzel:
        Jun 29, 2018 at 07:45 AM

        To my knowledge, both of them require root privileges, don’t they?

        Reply to this comment

        Reply to this comment

        1. Rich:
          Jun 29, 2018 at 12:04 PM

          No, virt-builder does NOT require root privileges.

          Reply to this comment

          Reply to this comment

          1. Ana:
            Jul 06, 2018 at 01:57 PM

            Hi everybody,

            mkosi has focus on VM and container images rather than images for real hardware and doesn't support all the architectures debos suppots (via qemu-user)
            virt-builder itself doesn't create images, it's a frontend for the images you have already created.


            Cheers,
            Ana

            Reply to this comment

            Reply to this comment


Add a Comment






Allowed tags: <b><i><br>Add a new comment:


Search the newsroom

Latest Blog Posts

An easy to use MTP implementation for your next embedded Linux project

13/04/2021

Did you know you could run a permissively-licensed MTP implementation with minimal dependencies on an embedded device? Here's a step-by-step…

Rust: integrating LLVM source-base code coverage with GitLab

24/03/2021

Earlier this year, the Rust compiler gained support for LLVM source-base code coverage. In this post we'll explain how to setup a CI job…

Build and run GTK 4 applications with Visual Studio

18/03/2021

Over the past few months, I've been working on a side project to improve Meson sub-project support. The best stress test is to build projects…

New features, changes & improvements to KernelCI's UI

04/03/2021

The most complete automated testing and continuous integration tool for the Linux kernel continues to evolve at a rapid pace. Here's a look…

Asymmetric Multi Processing with Linux & Zephyr on the STM32MP1

03/03/2021

In the embedded world, many modern SoCs such as the ST Microelectronics STM32MP1 now include coprocessor cores which can be used for a wide…

Adding HEVC/H.265 support for NXP's i.MX 8M

17/02/2021

Our recent efforts on the Hantro kernel driver have resulted in the addition of H.264 decoding support and multiple performance improvements.…

Open Since 2005 logo

We use cookies on this website to ensure that you get the best experience. By continuing to use this website you are consenting to the use of these cookies. To find out more please follow this link.

Collabora Ltd © 2005-2021. All rights reserved. Privacy Notice. Sitemap.

Collabora Limited is registered in England and Wales. Company Registration number: 5513718. Registered office: The Platinum Building, St John's Innovation Park, Cambridge, CB4 0DS, United Kingdom. VAT number: 874 1630 19.