*

Introducing debos, a versatile images generator

Ana Guerrero López avatar

Posted on 27/06/2018 by Ana Guerrero López

In Debian and derivative systems, there are many ways to build images. The simplest tool of choice is often debootstrap. It works by downloading the .deb files from a mirror and unpacking them into a directory which can eventually be chrooted into.

More often than not, we want to make some customization on this image, install some extra packages, run a script, add some files, etc

debos is a tool to make thiese kinds of trivial tasks easier. debos works using recipe files in YAML listing the actions you want to perform in your image sequentially and finally, choosing the output formats.

As opposite to debootstrap and other tools, debos doesn't need to be run as root for making actions that require root privileges in the images. debos uses fakemachine a library that setups qemu-system allowing you to work in the image with root privileges and to create images for all the architectures supported by qemu user. However, for this to work, make sure your user has permission to use /dev/kvm.

Let's see how debos works with a simple example. If we wanted to create an arm64 image for Debian Stretch customized, we would follow these steps:

  • debootstrap the image
  • install the packages we need
  • create an user
  • setup our preferred hostname
  • run a script creating an user
  • copy a file adding the user to sudoers
  • creating a tarball with the final image

This would translate into a debos recipe like this one:

{{- $architecture := or .architecture "arm64" -}}
{{- $suite := or .suite "stretch" -}}
{{ $image := or .image (printf "debian-%s-%s.tgz" $suite $architecture) }}

architecture: {{ $architecture }}

actions:
  - action: debootstrap
    suite: {{ $suite }}
    components:
      - main
    mirror: http://deb.debian.org/debian
    variant: minbase

  - action: apt
    recommends: false
    packages:
      - adduser
      - sudo

  - action: run
    description: Set hostname
    chroot: true
    command: echo debian-{{ $suite }}-{{ $architecture }} > /etc/hostname

  - action: run
    chroot: true
    script: scripts/setup-user.sh

  - action: overlay
    description: Add sudo configuration
    source: overlays/sudo

  - action: pack
    file: {{ $image }}
    compression: gz

(The files used in this example are available from this git repository)

We run debos on the recipe file:

$ debos simple.yaml

The result will be a tarball named debian-stretch-arm64.tar.gz. If you check the top two lines of the recipe, you can see that the recipe defaults to architecture arm64 and Debian stretch. We can override these defaults when running debos:

$ debos -t suite:"buster" -t architecture:"amd64" simple.yaml

This time the result will be a tarball debian-buster-amd64.tar.gz.

The recipe allows some customization depending on the parameters. We could install packages depending on the target architecture, for example, installing python-libsoc in armhf and arm64:

- action: apt
  recommends: false
  packages:
    - adduser
    - sudo
{{- if eq $architecture "armhf" "arm64" }}
    - python-libsoc
{{- end }}

What happens if in addition to a tarball we would like to create a filesystem image? This could be done adding two more actions to our example, a first action creating the image partition with the selected filesystem and a second one deploying the image in the filesystem:

- action: image-partition
  imagename: {{ $ext4 }}
  imagesize: 1GB
  partitiontype: msdos
  mountpoints:
    - mountpoint: /
      partition: root
  partitions:
    - name: root
      fs: ext4
      start: 0%
      end: 100%
      flags: [ boot ]

- action: filesystem-deploy
  description: Deploying filesystem onto image

{{ $ext4 }} should be defined in the top of the file as follows:

{{ $ext4 := or .image (printf "debian-%s-%s.ext4" $suite $architecture) }}

We could even make this step optional and make the recipe by default to only create the tarball and add the filesystem image only adding an option to debos:

$ debos -t type:"full" full.yaml

The final debos recipe will look like this:

{{- $architecture := or .architecture "arm64" -}}
{{- $suite := or .suite "stretch" -}}
{{ $type := or .type "min" }}
{{ $image := or .image (printf "debian-%s-%s.tgz" $suite $architecture) }}
{{ $ext4 := or .image (printf "debian-%s-%s.ext4" $suite $architecture) }}

architecture: {{ $architecture }}

actions:
  - action: debootstrap
    suite: {{ $suite }}
    components:
      - main
    mirror: http://deb.debian.org/debian
    variant: minbase

  - action: apt
    recommends: false
    packages:
      - adduser
      - sudo
{{- if eq $architecture "armhf" "arm64" }}
      - python-libsoc
{{- end }}

  - action: run
    description: Set hostname
    chroot: true
    command: echo debian-{{ $suite }}-{{ $architecture }} > /etc/hostname

  - action: run
    chroot: true
    script: scripts/setup-user.sh

  - action: overlay
    description: Add sudo configuration
    source: overlays/sudo

  - action: pack
    file: {{ $image }}
    compression: gz

{{ if eq $type "full" }}
  - action: image-partition
    imagename: {{ $ext4 }}
    imagesize: 1GB
    partitiontype: msdos
    mountpoints:
      - mountpoint: /
        partition: root
    partitions:
      - name: root
        fs: ext4
        start: 0%
        end: 100%
        flags: [ boot ]

  - action: filesystem-deploy
    description: Deploying filesystem onto image
{{end}}

debos also provides some other actions that haven't been covered in the example above:

  • download allows to download a single file from the internet
  • raw can directly write a file to the output image at a given offset
  • unpack can be used to unpack files from archive in the filesystem
  • ostree-commit create an OSTree commit from rootfs
  • ostree-deploy deploy an OSTree branch to the image

The example in this blog post is simple and short on purpose. Combining the actions presented above, you could also include a kernel and install a bootloader to make a bootable image. Upstream is planning to add more examples soon to the debos recipes repository.

debos is a project from Sjoerd Simons at Collabora, it's still missing some features but it's actively being developed and there are big plans for the future!


Visit Ana's blog.

Comments (8)

  1. Osqui:
    Jun 28, 2018 at 10:29 AM

    Excellent.
    Do you plan extend this to create not Debian-like images?
    Thanks!

    Reply to this comment

    Reply to this comment

    1. Ana:
      Jul 06, 2018 at 01:52 PM

      Hi Osqui!

      As the name hints, we're not planning to work on creating images for other distributions. However we'll be happy to add this if somebody send a pull request :)

      Cheers,
      Ana

      Reply to this comment

      Reply to this comment

    2. Mrfai:
      Jul 08, 2018 at 11:58 PM

      The Debian cloud team is using the FAI software (https://fai-project.org) for different cloud images and has it's configuration at https://salsa.debian.org/cloud-team/fai-cloud-images. FAI can create installation images and bootable disk images for other distributions like Ubuntu, CentOS, Scientific Linux.

      Reply to this comment

      Reply to this comment

  2. Rich:
    Jun 28, 2018 at 11:57 AM

    Why didn't you use virt-builder? It does all of this already.

    Reply to this comment

    Reply to this comment

      1. Paul Menzel:
        Jun 29, 2018 at 07:45 AM

        To my knowledge, both of them require root privileges, don’t they?

        Reply to this comment

        Reply to this comment

        1. Rich:
          Jun 29, 2018 at 12:04 PM

          No, virt-builder does NOT require root privileges.

          Reply to this comment

          Reply to this comment

          1. Ana:
            Jul 06, 2018 at 01:57 PM

            Hi everybody,

            mkosi has focus on VM and container images rather than images for real hardware and doesn't support all the architectures debos suppots (via qemu-user)
            virt-builder itself doesn't create images, it's a frontend for the images you have already created.


            Cheers,
            Ana

            Reply to this comment

            Reply to this comment


Add a Comment






Allowed tags: <b><i><br>Add a new comment:


Latest Blog Posts

The docker.io Debian package is back to life

04/07/2018

Last week, a new version of docker.io, the Docker package provided by Debian, was uploaded to Debian Unstable. Quickly afterwards, the package…

Introducing debos, a versatile images generator

27/06/2018

In Debian and derivative systems, there are many ways to build images. The simplest tool of choice is often debootstrap. It works by downloading…

Secure video comes of age

25/06/2018

Launched by Haivision in 2017, and freely available on GitHub via the Mozilla Public License 2.0, SRT is an innovative UDP-based protocol…

GStreamer CI support for embedded devices

11/06/2018

Embedded devices are a popular deployment target for GStreamer yet they are not tested on the project's Continuous Integration (CI) system.…

Happy 20th, Open Source

05/06/2018

In late January 1998, Netscape surprised everyone by releasing the source for Communicator, its web browser, making it readily available…

Four open months at Collabora

29/05/2018

At the start of 2018 in January, I joined Collabora, an open source software consultancy, as a Software Engineer Intern with the Multimedia…

Open Since 2005 logo

We use cookies on this website to ensure that you get the best experience. By continuing to use this website you are consenting to the use of these cookies. To find out more please follow this link.

Collabora Ltd © 2005-2018. All rights reserved. Website sitemap.