Introduction

Before doing linux kernel development, I started by typing make in a kernel tree. After booting, I always had some non working peripherals. So my second step was to use a distribution specific build procedure. For example, the Ubuntu kernel build instructions can be found at https://wiki.ubuntu.com/KernelTeam/GitKernelBuild. It works, and one can easily build a kernel and install it, with all peripheral working. But this method will quickly reach its limitations to write new kernel code. On a decent computer (i7 5600U), the build/test cycle lasts about 30 minutes. It is possible to build only the needed module and insmod/rmmod, but in case of a crash followed by a rebooting, the developer loses its work environment.

The next step is to run the kernel inside a virtual machine.

Booting kernel in a virtual machine

VirtualBox is well known, very user friendly and supports a large amount of different OSes. Installation of a VirtualBox will be under the hour mark. But rebuilding the Ubuntu kernel is still a 30 minute cycle. Additionnaly exchanging files between Virtualbox and the host will involve some kind of networking or file sharing that have to be setup. For kernel development, Virtualbox Guest additions have to be rebuilt often when the kernel is updated.

Qemu is another virtual machine. A complete distro can be installed into it. But it has a very interesting option: -kernel. With that option, QEMU will boot the kernel binary provided as argument. Ubuntu users can try :

$ sudo qemu-system-x86_64 -kernel /boot/vmlinuz-`uname -r`

This will boot your kernel within QEMU, but an error occurs immedialy: There is no filesystem to boot. Also, since /boot is readable only by root, sudo permission is required. This is not needed with a user built kernel.

Adding a rootfs

debootstrap allows to install a debian distribution in a directory. Before going too fast, if your file system is mounted with the nodev option, it won’t be possible to create device nodes. Instead, we will mount a qemu image file on a directory and use debootstrap in the mount point we created as in the following:

IMG=qemu-image.img
DIR=mount-point.dir
qemu-img create $IMG 1g
mkfs.ext2 $IMG
mkdir $DIR
sudo mount -o loop $IMG $DIR
sudo debootstrap --arch amd64 jessie $DIR
sudo umount $DIR
rmdir $DIR

The target rootfs is a matter of taste. For learning purposes, using busybox would be very interesting too. But for development purposes, having all the debian development tools in the rootfs is very useful.

Boot as follow:

$ sudo qemu-system-x86_64 -kernel /boot/vmlinuz-`uname -r`\
                          -hda qemu-image.img\
                          -append "root=/dev/sda single"

To silence the warning about raw format, replace “-hda qemu-image.img” with “-drive file=qemu-image.img,index=0,media=disk,format=raw”

Boot as a single user to change root password and create a user.

Building and booting a kernel

Now it is time to build your own kernel. There exist a make KVM configuration target that tunes an existing configuration and makes it usable from QEMU. However, it will not create a .config file from scratch. So we will start from generic config file and kvmify it. It is still possible to build a dedicated config file that will allow for shorter build time, but that would require several iterations.

git clone --depth=1 git://git.kernel.org/pub/scm/linux/kernel/git/torvalds/linux.git
cd linux
make x86_64_defconfig
# if building an older kernel use 'make kvmconfig" instead if below command fails
make kvm_guest.config
make -j 8

Use the resulting file in the command line below. We can drop sudo.

$ qemu-system-x86_64 -kernel arch/x86/boot/bzImage
                     -hda qemu-image.img
                     -append "root=/dev/sda"

Reduce the boot time with kvm

KVM accelerates x86 virtualization in QEMU. It will only accelerate x86 platforms. It is as simple as adding a command line option.

$ qemu-system-x86_64 -kernel bzImage
                     -hda qemu-image.img
                     -append "root=/dev/sda"
                     --enable-kvm

Now your debootstrap image boots in less than two seconds. This can be checked in dmesg. Before –enable-kvm, systemd is started 5.9 seconds after boot. After enabling, systemd is started after 1.7 seconds. A more than 3 times shorter boot time. And it just can’t be compared to ubuntu. Note that we don’t have a full user interface up, so we cannot compare apple with peaches.

Connecting into QEMU

Initially, QEMU displays its own screen in a dedicated window. For a terminal use case, this is not really pratical as it gets into the ALT tab list, the keyboard and mouse capture are not suitable for this use either. Copying and pasting also aren’t very practical. It is much more convenient to remove the graphic interface and instruct the kernel to write to ttyS0 that qemu redirect to the terminal in –nographic mode.

$ qemu-system-x86_64 -kernel bzImage
                     -append "root=/dev/sda console=ttyS0"
                     -hda qemu-image.img
                     --enable-kvm
                     --nographic

Time to leave

Typing halt in qemu will stop the kernel, but the qemu process would continue running on the host and would have to be killed. The proper command to terminate the virtual machine is :

$ shutdown -h now

A second part to this post is planned, stay tuned!

Original post