Posted on 16/01/2017 by Frédéric Dalleau
Before doing linux kernel development, I started by typing
make in a kernel tree. After booting, I always had some non working peripherals. So my second step was to use a distribution specific build procedure. For example, the Ubuntu kernel build instructions can be found at https://wiki.ubuntu.com/KernelTeam/GitKernelBuild. It works, and one can easily build a kernel and install it, with all peripheral working. But this method will quickly reach its limitations to write new kernel code. On a decent computer (i7 5600U), the build/test cycle lasts about 30 minutes. It is possible to build only the needed module and
insmod/rmmod, but in case of a crash followed by a rebooting, the developer loses its work environment.
The next step is to run the kernel inside a virtual machine.
VirtualBox is well known, very user friendly and supports a large amount of different OSes. Installation of a VirtualBox will be under the hour mark. But rebuilding the Ubuntu kernel is still a 30 minute cycle. Additionnaly exchanging files between Virtualbox and the host will involve some kind of networking or file sharing that have to be setup. For kernel development, Virtualbox Guest additions have to be rebuilt often when the kernel is updated.
Qemu is another virtual machine. A complete distro can be installed into it. But it has a very interesting option: -kernel. With that option, QEMU will boot the kernel binary provided as argument. Ubuntu users can try :
$ sudo qemu-system-x86_64 -kernel /boot/vmlinuz-`uname -r`
This will boot your kernel within QEMU, but an error occurs immedialy: There is no filesystem to boot. Also, since
/boot is readable only by root, sudo permission is required. This is not needed with a user built kernel.
debootstrap allows to install a debian distribution in a directory. Before going too fast, if your file system is mounted with the nodev option, it won’t be possible to create device nodes. Instead, we will mount a qemu image file on a directory and use debootstrap in the mount point we created as in the following:
IMG=qemu-image.img DIR=mount-point.dir qemu-img create $IMG 1g mkfs.ext2 $IMG mkdir $DIR sudo mount -o loop $IMG $DIR sudo debootstrap --arch amd64 jessie $DIR sudo umount $DIR rmdir $DIR
The target rootfs is a matter of taste. For learning purposes, using busybox would be very interesting too. But for development purposes, having all the debian development tools in the rootfs is very useful.
Boot as follow:
$ sudo qemu-system-x86_64 -kernel /boot/vmlinuz-`uname -r`\ -hda qemu-image.img\ -append "root=/dev/sda single"
To silence the warning about raw format, replace “-hda qemu-image.img” with “-drive file=qemu-image.img,index=0,media=disk,format=raw”
Boot as a single user to change root password and create a user.
Now it is time to build your own kernel. There exist a make kvmconfig option that tunes an existing configuration and makes it usable from QEMU. However, it will not create a .config file from scratch. So we will start from generic config file and kvmify it. It is still possible to build a dedicated config file that will allow for shorter build time, but that would require several iterations.
git clone --depth=1 git://git.kernel.org/pub/scm/linux/kernel/git/torvalds/linux.git cd linux make x86_64_defconfig make kvmconfig make -j 8
Use the resulting file in the command line below. We can drop sudo.
$ qemu-system-x86_64 -kernel arch/x86/boot/bzImage -hda qemu-image.img -append "root=/dev/sda"
KVM accelerates x86 virtualization in QEMU. It will only accelerate x86 platforms. It is as simple as adding a command line option.
$ qemu-system-x86_64 -kernel bzImage -hda qemu-image.img -append "root=/dev/sda" --enable-kvm
Now your debootstrap image boots in less than two seconds. This can be checked in dmesg. Before –enable-kvm, systemd is started 5.9 seconds after boot. After enabling, systemd is started after 1.7 seconds. A more than 3 times shorter boot time. And it just can’t be compared to ubuntu. Note that we don’t have a full user interface up, so we cannot compare apple with peaches.
Initially, QEMU displays its own screen in a dedicated window. For a terminal use case, this is not really pratical as it gets into the ALT tab list, the keyboard and mouse capture are not suitable for this use either. Copying and pasting also aren’t very practical. It is much more convenient to remove the graphic interface and instruct the kernel to write to ttyS0 that qemu redirect to the terminal in –nographic mode.
$ qemu-system-x86_64 -kernel bzImage -append "root=/dev/sda console=ttyS0" -hda qemu-image.img --enable-kvm --nographic
halt in qemu will stop the kernel, but the qemu process would continue running on the host and would have to be killed. The proper command to terminate the virtual machine is :
$ shutdown -h now
A second part to this post is planned, stay tuned!
Earlier this year I worked on a certain GStreamer plugin that is called “ipcpipeline”. This plugin provides elements that make it possible…
Running crosvm outside Chromium OS is quite easy, with the only complication being that minijail isn't widely packaged in distros. In these…
Kmemleak allows you to track possible memory leaks inside the Linux kernel. Basically, it tracks dynamically allocated memory blocks in…
Earlier this month I had the pleasure of attending the Web Engines Hackfest, hosted by Igalia at their offices in A Coruña, and also sponsored…
In this post, I will show one more example of how easy it is to disrupt performance of a modern CPU, and also run a quick discussion on…
Many thanks to Google for recording all the XDC2017 talks. To make them easier to watch, here are direct links to each talk recorded at…