We're hiring!

Using syzkaller, part 2: Detecting programming bugs in the Linux kernel

Andre Almeida avatar

Andre Almeida
April 17, 2020

Share this post:

In my previous blog post, we discussed the importance of testing, what is fuzzing, and how the syzkaller fuzzes the kernel in order to find bugs. Now, let’s install the tool and starting using it to improve our code base.

The kernel source will be expected to be found in the $KSRC directory. The syscall descriptions are based on linux-next, so if something fails or triggers a warning that a specific syscall isn’t defined, consider changing to the current linux-next/master branch.

Your kernel should be specifically configured to enhance the performance of syzkaller and to enable it to be properly fuzzed. The following configs are necessary:


To be able to navigate through the code coverage in the web interface, CONFIG_DEBUG_INFO must also be enabled. To get the most of the power of syzkaller, there are more configuration options that can be enabled, the more the better honorable mention for KASAN. After modifying the configuration, build the kernel again, since we will need a compiled kernel in next steps. Also, it’s required to have the following tools:

  • Go
  • QEMU
  • git
  • OpenSSH
  • A recent C compiler with coverage support (both current GCC and Clang versions should work)

To get the Go toolchain and syzkaller source, run:

go get golang.org/dl/go1.12
go get -u -d github.com/google/syzkaller/...

This doesn’t seem a very canonical way of downloading and installing a package, but since this project is created by Go developers, I believe that their suggestion is currently the best approach. Syzkaller should now be at ~/go/src/github.com/google/syzkaller. Not a very convenient path, but this can be solved with a symlink or the PATH variable. In this example, it will be defined by the following variable:

export SYZPATH=~/go/src/github.com/google/syzkaller

A basic rootfs for the virtual machines can be created using a script provided by syzkaller that requires the debootstrap package. Choose a directory in which to store rootfs images (which is stored as `$IMAGES` below), then run:

export IMAGES=$(pwd)
bash $SYZPATH/tools/create-image.sh

Use -h to explore the other options provided by this tool. In summary, it will create a basic Debian image with the tools that we need and a SSH server to copy and run commands across virtual machines.

Verifying your setup

Just to make sure everything is working fine and avoid some troubleshooting in the future, check if the ssh is properly working. First, boot the machine:

qemu-system-x86_64 \
  -kernel $KSRC/arch/x86/boot/bzImage \
  -append "console=ttyS0 root=/dev/sda debug earlyprintk=serial slub_debug=QUZ"\
  -hda $IMAGES/stretch.img \
  -net user,hostfwd=tcp::10021-:22 -net nic \
  -enable-kvm \
  -nographic \
  -m 2G \
  -smp 2 \
  -pidfile vm.pid \
  2>&1 | tee vm.log

It’s possible to login as root if debugging is necessary.

At the host system, try to access it:

ssh -i $IMAGES/stretch.id_rsa -p 10021 -o "StrictHostKeyChecking no" root@localhost

When done, kill the VM with kill $(cat $IMAGES/vm.pid).

Build and configure syzkaller and its tools

Now we have a working rootfs, we are almost ready. Go ahead and build syzkaller and its tools:

make -j8

We need to create a work directory, where the tool will create a database and store results. Having a separate directory is a good way to organize information as it allows us to have different databases for different syscalls or for different configurations. To do that, just create different directories and set the one you want to work with in the configuration file. Any name works, but those starting with workdir* will be ignored by git, as per a rule in .gitignore file.

mkdir workdir

To define the behavior of the tool, create a configuration file somewhere, here called as config.cfg. This example should be enough for an initial run:

        "target": "linux/amd64",
        "http": "",
        "workdir": "$SYZPATH/workdir",
        "kernel_obj": "$KSRC",
        "image": "$IMAGES/stretch.img",
        "sshkey": “$IMAGES/stretch.id_rsa",
        "syzkaller": "$SYZPATH",
        "procs": 8,
        "type": "qemu",
        "vm": {
                "count": 2,
                "kernel": "$KSRC/arch/x86/boot/bzImage",
                "cpu": 2,
                "mem": 2048

The contents of the above file are as follows:

  • target: operating system/architecture to be fuzzed
  • http: IP address and port of where syzkaller’s web interface will be exposed
  • workdir: the work directory to be used, as explained above
  • kernel_obj: kernel source directory
  • image: bootstraped distro image
  • sshkey: SSH key to be used to access the VMs
  • syzkaller: path to syzkaller source
  • procs: number of parallel tests inside each VM
  • type: virtual machine hypervisor/device to be used
  • vm: virtual machine configuration:
    • count: number of VM’s to spawn
    • kernel: kernel image to fuzz
    • cpu: number of cores simulate in each VM (just as -smp at QEMU)
    • mem: RAM size of VM (just as -m at QEMU)

By default, all syscall are fuzzed (we will see how to change that later). We can now start the fuzzer, specifying the configuration file:

./bin/syz-manager -config=config.cfg

2020/02/17 14:22:01 loading corpus...
2020/02/17 14:22:01 serving http on
2020/02/17 14:22:01 serving rpc on tcp://[::]:46009
2020/02/17 14:22:01 booting test machines...
2020/02/17 14:22:01 wait for the connection from test machine...

Open the URL provided and you should be able to check the web interface. It’s a convenient way to navigate through the features, check coverage reports, crashes and corpus. Notice that the server may seem unresponsive for a few seconds while the tool is performing some heavy operations, so be patient.

This is what the main dashboard looks like. In the first section, you can read the current status fo the tool, like the uptime, number of syscalls enabled, crashes and fuzzer execution statistics. In the second one, you have a list of crashes, how many times it happened. It may also export a report and a C reproducer of the crash. The last section is the log, the same log that the tool outputs in the terminal with live information about the tool progress.

Clicking in corpus, you can see the set of syscalls that produces the current coverage. If a syscall can't increase the coverage, it will not be saved in this database.

The list shows a link for the coverage that this syscall had produce as well as the name of the syscall description. Clicking on the name, you can see the input of this corpus. For instance, this is an exemple of syz_mount_image$ext4:

syz_mount_image$ext4(0x0, &(0x7f0000000040)='./file0\x00', 0x0, 0x0, 0x0, 0x0, 0x0)

Clicking in a coverage report in the corpus page will show the coverage for this particular input. Clicking in the coverage link in the main dashboard will show the global coverage, for the current corpus.

This is the current coverage of the file usercopy.c for the current corpus. An explanation about the coverage colors can be found in the documentation.

And we are done! This is the basic usage of syzkaller. Dig into the documentation for more details and see this source file for more options for the configuration. I found the following ones to be very useful to add at config.cfg file:

  • "enable_syscalls": [ "eventfd", "read$eventfd", ... ]: only those syscalls will be fuzzed. If they depend on another syscall to work (e.g. open()) syzkaller will ask you to enable it. Using read will trigger all definitions for this syscalls, including the eventfd one. Using read$eventfd will trigger just the eventfd one.
  • "disable_syscalls": ["ioctl$int_int", “mmap”, ... ]: all syscalls will be fuzzed, except those defined here.

Those new options may be added anywhere at the config file, with the exception inside the vm curly braces.


Now you should be ready to starting using syzkaller. Remember that the tool needs some some time to starting showing some results. Join the mailing list to get in touch with developers, see new bugs found by the tool and to report the crashes that you found.

In the final part of this series, we are going to see how to modify syzkaller in order to stress your own changes and improve testing in our development process.

Continue reading (Using syzkaller, part 3: Fuzzing your changes)

Comments (0)

Add a Comment

Allowed tags: <b><i><br>Add a new comment:

Search the newsroom

Latest Blog Posts

An easy to use MTP implementation for your next embedded Linux project


Did you know you could run a permissively-licensed MTP implementation with minimal dependencies on an embedded device? Here's a step-by-step…

Rust: integrating LLVM source-base code coverage with GitLab


Earlier this year, the Rust compiler gained support for LLVM source-base code coverage. In this post we'll explain how to setup a CI job…

Build and run GTK 4 applications with Visual Studio


Over the past few months, I've been working on a side project to improve Meson sub-project support. The best stress test is to build projects…

New features, changes & improvements to KernelCI's UI


The most complete automated testing and continuous integration tool for the Linux kernel continues to evolve at a rapid pace. Here's a look…

Asymmetric Multi Processing with Linux & Zephyr on the STM32MP1


In the embedded world, many modern SoCs such as the ST Microelectronics STM32MP1 now include coprocessor cores which can be used for a wide…

Adding HEVC/H.265 support for NXP's i.MX 8M


Our recent efforts on the Hantro kernel driver have resulted in the addition of H.264 decoding support and multiple performance improvements.…

Open Since 2005 logo

We use cookies on this website to ensure that you get the best experience. By continuing to use this website you are consenting to the use of these cookies. To find out more please follow this link.

Collabora Ltd © 2005-2021. All rights reserved. Privacy Notice. Sitemap.

Collabora Limited is registered in England and Wales. Company Registration number: 5513718. Registered office: The Platinum Building, St John's Innovation Park, Cambridge, CB4 0DS, United Kingdom. VAT number: 874 1630 19.