January 16, 2017
Before doing linux kernel development, I started by typing
make in a kernel tree. After booting, I always had some non working peripherals. So my second step was to use a distribution specific build procedure. For example, the Ubuntu kernel build instructions can be found at https://wiki.ubuntu.com/KernelTeam/GitKernelBuild. It works, and one can easily build a kernel and install it, with all peripheral working. But this method will quickly reach its limitations to write new kernel code. On a decent computer (i7 5600U), the build/test cycle lasts about 30 minutes. It is possible to build only the needed module and
insmod/rmmod, but in case of a crash followed by a rebooting, the developer loses its work environment.
The next step is to run the kernel inside a virtual machine.
VirtualBox is well known, very user friendly and supports a large amount of different OSes. Installation of a VirtualBox will be under the hour mark. But rebuilding the Ubuntu kernel is still a 30 minute cycle. Additionnaly exchanging files between Virtualbox and the host will involve some kind of networking or file sharing that have to be setup. For kernel development, Virtualbox Guest additions have to be rebuilt often when the kernel is updated.
Qemu is another virtual machine. A complete distro can be installed into it. But it has a very interesting option: -kernel. With that option, QEMU will boot the kernel binary provided as argument. Ubuntu users can try :
$ sudo qemu-system-x86_64 -kernel /boot/vmlinuz-`uname -r`
This will boot your kernel within QEMU, but an error occurs immedialy: There is no filesystem to boot. Also, since
/boot is readable only by root, sudo permission is required. This is not needed with a user built kernel.
debootstrap allows to install a debian distribution in a directory. Before going too fast, if your file system is mounted with the nodev option, it won’t be possible to create device nodes. Instead, we will mount a qemu image file on a directory and use debootstrap in the mount point we created as in the following:
IMG=qemu-image.img DIR=mount-point.dir qemu-img create $IMG 1g mkfs.ext2 $IMG mkdir $DIR sudo mount -o loop $IMG $DIR sudo debootstrap --arch amd64 jessie $DIR sudo umount $DIR rmdir $DIR
The target rootfs is a matter of taste. For learning purposes, using busybox would be very interesting too. But for development purposes, having all the debian development tools in the rootfs is very useful.
Boot as follow:
$ sudo qemu-system-x86_64 -kernel /boot/vmlinuz-`uname -r`\ -hda qemu-image.img\ -append "root=/dev/sda single"
To silence the warning about raw format, replace “-hda qemu-image.img” with “-drive file=qemu-image.img,index=0,media=disk,format=raw”
Boot as a single user to change root password and create a user.
Now it is time to build your own kernel. There exist a make kvmconfig option that tunes an existing configuration and makes it usable from QEMU. However, it will not create a .config file from scratch. So we will start from generic config file and kvmify it. It is still possible to build a dedicated config file that will allow for shorter build time, but that would require several iterations.
git clone --depth=1 git://git.kernel.org/pub/scm/linux/kernel/git/torvalds/linux.git cd linux make x86_64_defconfig make kvmconfig make -j 8
Use the resulting file in the command line below. We can drop sudo.
$ qemu-system-x86_64 -kernel arch/x86/boot/bzImage -hda qemu-image.img -append "root=/dev/sda"
KVM accelerates x86 virtualization in QEMU. It will only accelerate x86 platforms. It is as simple as adding a command line option.
$ qemu-system-x86_64 -kernel bzImage -hda qemu-image.img -append "root=/dev/sda" --enable-kvm
Now your debootstrap image boots in less than two seconds. This can be checked in dmesg. Before –enable-kvm, systemd is started 5.9 seconds after boot. After enabling, systemd is started after 1.7 seconds. A more than 3 times shorter boot time. And it just can’t be compared to ubuntu. Note that we don’t have a full user interface up, so we cannot compare apple with peaches.
Initially, QEMU displays its own screen in a dedicated window. For a terminal use case, this is not really pratical as it gets into the ALT tab list, the keyboard and mouse capture are not suitable for this use either. Copying and pasting also aren’t very practical. It is much more convenient to remove the graphic interface and instruct the kernel to write to ttyS0 that qemu redirect to the terminal in –nographic mode.
$ qemu-system-x86_64 -kernel bzImage -append "root=/dev/sda console=ttyS0" -hda qemu-image.img --enable-kvm --nographic
halt in qemu will stop the kernel, but the qemu process would continue running on the host and would have to be killed. The proper command to terminate the virtual machine is :
$ shutdown -h now
A second part to this post is planned, stay tuned!
Introducing cmtp-responder - a permissively licensed Media Transfer Protocol (MTP) responder implementation which allows embedded devices…
Up until now, talking in-depth about userspace tracing was deliberately avoided because it merits special treatment, hence this part devoted…
After a successful team effort, the patch enabling the Chromium Embedded Framework (CEF) Ozone builds to run with different platform backends,…
Now that we've studied the mainstream way of developing and using eBPF programs on top of the low-level VM mechanisms, we'll look at projects…
A previous post introduced the SPURV Android compatibility layer for Wayland based Linux environment. In this post, we're going to dig into…
In part 1 and 2 of this series, we took a condensed in-depth look at the eBPF VM. In part 3, we define the high-level components of an eBPF…