KernelCI has been driving continuous integration for the Linux kernel with hundreds of commits every day since its creation in 2012 and as a Linux Foundation project since 2019. The platform has the ability to help you attempt to find and fix several problems, including regressions, build failures and merge conflicts from your patches with others.

Today, KernelCI is the most complete automated testing and continuous integration tool for the Linux kernel. It can test your code on many platforms performing automatic builds in kernel trees. Builds, tests information and more can be found on the KernelCI dashboard, that is, the main web interface used by the project.

Considering that the KernelCI dashboard has changed in many points, this blog post will share its new features, discuss changes and present the transition from checking basic boot reports to starting to check more advanced test reports. Opening the way to focusing more on tests, more specifically functional tests, and generally in quality, stability and long-term support than just about booting kernels.

Showing test results on main jobs page

The first big change was switching the "jobs" page to show test count results and drop the boot count there. So if you look at the jobs view, the fourth column is now showing the total number of tests that passed, always failed and regressed for each job in green, yellow and red respectively.

New jobs view showing builds and latest tests

If you want to see more detailed information and, for example, select a tree from the jobs page you will see a spinning animation that will show test results for this job, filtered by branch and kernel. Selecting next tree, you will see all branches that originate from it.

You can retrieve still more detailed information. Clicking on a branch, you go to a new page where it is possible to check test results for a branch and its kernel revisions as you can see in the following picture:

Jobs view presenting tests filtered by tree and branch

Selecting a kernel version from the jobs page, you will go to another new view that will show which test plans were run for a kernel revision. On this page you can also select a test plan. Selecting, for example, baseline, it is possible to find more details about tests from this plan with detailed information of configurations, SoC used by that build and its complete log. On the same page you can see in detail which test groups have passed or failed, by checking the test results or looking at their status icon. Going through, you can check the test path of each test group, and if it has caused some regressions you also have the option to check more information about it. By clicking on a regression there is a full results button that will redirected you to the page that shows test cases for the selected test plan.

The view that shows a test plan brings some useful information about a job filtered by branch, tree and one specific test plan with its cases, measurements and their status. If you wish to have more detail about the status and want to see more information about what could have happened, you can select a case and go for the next page. It will show more details about this test case, including log lines and its regression history, as you can check here for baseline.

Renewing Tests View

The test page was also updated at some points, now that the focus of KernelCI is going to be tests. If you take a look at the main tests page, results are now presented combining kernel revision and test plans. At first glance, you can see in the status column if any test has failed. Other direct information include test results for each combination of kernel and test plan.

Exploring new SoC details

Some changes were done for the SoC main page as well. Starting from the beginning and following the same approach as the jobs view, now on the main SoC page it is possible to see the total number of test results for each device.

If you select one row, you will be redirected to a page that shows trees, branches and a counter for latest test results from the selected SoC. Selecting a tree, you will see a list of kernel revisions of mainline tree.

Again, following the same approach from jobs, it is possible to see kernels from one SoC and one tree, as well as their test results. For each of those kernel versions, you can also check details about which test plans were run.

Taking the same principle from the jobs and tests view, the user is able to explore more information related to SoC. When selecting a kernel version in the last image, you will see which tests plan have run. Clicking on a test plan you will be redirected to a test page that will present results filtered by SoC, Kernel version and test plan, as illustrated by the image below:

Tests result filtered by SoC, Kernel and Test Plan

Then, it is possible to see full information about each specific combination of SoC, kernel and test plan. Click on full results and it will redirect you to the right test plan.

Good-bye legacy tests and boot views

All this work was done focusing on switching the way of presenting boots as results to showing results focused on tests and test results. After all those changes have been done, it was time to remove all the unnecessary views. The first to be dropped were pages responsible for showing some tests-related groups and boards.

All boots and their subdomains will not be necessary anymore, so they were removed from the main KernelCI website last June. In the same way, legacy test views were removed because the main idea for the next steps in KernelCI is to show results focused on tests.

Looking ahead

As work continues towards further improving user experience, discussions are now focused on enhancing KernelCI's dashboards, visualization and analytics. If you would like to take part in this work and share some ideas or feedback, please do so here.