We're hiring!
*

DRM-CI: A GitLab-CI pipeline for Linux kernel testing

Helen Koike avatar

Helen Koike
February 08, 2024

Share this post:

Reading time:

Continuing our Kernel Integration series, we're excited to introduce DRM-CI, a groundbreaking solution that enables developers to test their graphics subsystem patches across numerous devices within the community's shared infrastructure. Released with Linux kernel 6.7, DRM-CI is a GitLab-CI pipeline that runs on the Freedesktop infrastructure. This setup allows for efficient kernel compilation and dispatches tests across multiple devices available in various community labs.

Linus Torvalds' recent critique of the Xe driver's compilation issues in Linux 6.8 underscores the critical need for thorough testing in kernel development.

Present Capabilities of DRM-CI

Currently, DRM-CI runs the igt-gpu-tools on a wide range of devices, including amdgpu, i915, mediatek, meson, msm, rockchip, and virtio-gpu. Additionally, support for vkms is in the works. These devices are stationed in various labs such as those of Collabora, Google, and the newly included Qualcomm lab. We welcome the integration of more labs - just contribute a patch!


The roadmap for DRM-CI includes enabling other devices, incorporating additional tests like kselftests, adding support for vgem driver, and implementing further automations. DRM-CI builds upon the groundwork laid by Mesa3D CI, including its GitLab YAML files and most of its setup, fostering collaboration and mutual strengthening.

Quick Tutorial on Getting Started with DRM-CI

Running tests is straightforward – push your code, hit a button, and the tests begin. Here's a quick setup guide from kernel docs:

Setup

  1. Create or use your Linux tree at Freedesktop GitLab.
  2. In your kernel repo's settings, switch the CI/CD configuration file to drivers/gpu/drm/ci/gitlab-ci.yml.
  3. Join the drm/ci-ok group on Freedesktop GitLab for the necessary CI privileges.
  4. Pushing to your repository will now trigger a CI pipeline, with results accessible at your repository's pipeline page.

Triggering Pipelines from Command Line

DRM-CI inherits its structure from the Mesa3D project, allowing the use of similar tools. The ci_run_n_monitor.sh script facilitates pipeline triggering without needing to navigate the web interface.

  1. Clone Mesa: git clone --depth 1 https://gitlab.freedesktop.org/mesa/mesa.git
  2. Generate a personal access token with API scope at Freedesktop GitLab and save it to ~/.config/gitlab-token.
  3. From your Linux folder, run ../mesa/bin/ci/ci_run_n_monitor.sh with your specified target and project.

Example:

../mesa/bin/ci/ci_run_n_monitor.sh --target ".*" --force-manual --project linux # linux is the name of your repository on gitLab

ci_run_n_monitor.sh output

Targeted Pipeline Triggers

Optimize resource usage by triggering only the tests relevant to your patch. Use the --target option to specify jobs, such as --target "amd.*|i915.*|msm.*" for certain device types.

Using Hash or Pipeline URL

You can also give a specific hash or pipeline URL to the script:

./bin/ci/ci_run_n_monitor.sh --target "i915.*" --force-manual --pipeline-url https://gitlab.freedesktop.org/helen.fornazier/mesa/-/pipelines/1043677
./bin/ci/ci_run_n_monitor.sh --target "android-virgl-llvmpipe" --force-manual --rev HEAD^

Gantt Chart Creation

Generate a Gantt chart for an in-depth analysis of execution duration using the ci_gantt_chart.sh script.

../mesa/bin/ci/ci_gantt_chart.sh https://gitlab.freedesktop.org/janedoe/linux/-/pipelines/1075920

Addressing Fails and Flakes

When encountering test failures that aren't due to infrastructure issues (like a device being down), one can inspect the job artifacts.

Artifacts button

A key file, failures.csv, is generated listing all failing unit tests along with the reasons for their failures.

Example of failures.csv:

kms_addfb_basic@bad-pitch-65536,Fail
kms_addfb_basic@bo-too-small,Fail
kms_addfb_basic@too-high,Fail
kms_async_flips@async-flip-with-page-flip-events,Fail
kms_async_flips@crc,Fail
kms_async_flips@invalid-async-flip,Fail
kms_atomic_transition@plane-all-modeset-transition-internal-panels,Fail
kms_atomic_transition@plane-all-transition,Fail
kms_atomic_transition@plane-all-transition-nonblocking,Fail
kms_atomic_transition@plane-toggle-modeset-transition,Fail
kms_atomic_transition@plane-use-after-nonblocking-unbind,Fail

If your patch is the cause of a new failure, the first step should be to address this issue. However, if the failure isn't related to your patch, you can inform GitLab-CI about this expected failure to ensure your pipeline remains green.

Within the drivers/gpu/drm/ci/xfails/ directory, each job has associated files: -fails.txt, -flakes.txt, and -skips.txt. For a failure tagged as "Fail", add it to -fails.txt. If your patch unexpectedly fixes an issue, remove the respective test from -fails.txt.

In cases where a test is flaking (i.e., it inconsistently passes or fails), add it to -flakes.txt. GitLab-CI will then disregard its result.

Automating the Management of Fails and Flakes

The list of fails and flakes can be extensive, especially when enabling a new device. To assist with this, use the update-xfails.py script. It's advisable to retry failed jobs a few times to discern between fails and flakes. Run the script with the pipeline URL, and it will automatically update your xfails files based on the pipeline's results.

For example:

./drivers/gpu/drm/ci/xfails/update-xfails.py https://gitlab.freedesktop.org/janedoe/linux/-/pipelines/1017090

Tests Causing Device Crashes

If a particular test leads to an unrecoverable crash preventing the job from completing, you can list this test in the -skips.txt file (e.g., drivers/gpu/drm/ci/xfails/amdgpu-stoney-skips.txt). Tests mentioned here will not be executed, avoiding such crashes.

In addition, the skip list can be used to deliberately exclude tests that are irrelevant or unnecessary for certain drivers, thereby optimizing resource utilization.

Integrating GitLab-CI into Development Workflows

Developers can enhance their patch submission process by running pipelines and sharing the results link when submitting patches. This aids reviewers and maintainers in assessing the patch's impact, with all the advantages of pre-merge tests. However, to conserve resources, it's essential to coordinate with maintainers on their preferred testing protocols.

Expanding Beyond Graphics

Adapting the DRM-CI pipeline to other subsystems is feasible with a few modifications. The primary consideration is setting up dedicated GitLab-CI runners since Freedesktop's infrastructure is meant only for graphics.

In light of this, our team is developing a versatile and user-friendly GitLab-CI pipeline. This new pipeline is envisioned to function as a flexible interface for kernel maintainers and developers that can be evolved to connect with different test environments that can also be hooked with CI systems such as KernelCI. This approach aims to simplify the integration process, making GitLab-CI more accessible and beneficial to a broader range of developers.

Conclusion

A GitLab-CI workflow presents a significant stride in Linux kernel development, particularly for graphics subsystem patches. It not only streamlines the testing process but also fosters a culture of shared resources and community collaboration. As it continues to evolve, developers need to engage with these resources thoughtfully and responsibly, ensuring efficient and effective development cycles. Stay tuned for further updates and enhancements to DRM-CI, and don't hesitate to contribute to its growth! Happy coding!

Comments (0)


Add a Comment






Allowed tags: <b><i><br>Add a new comment:


Search the newsroom

Latest Blog Posts

Almost a fully open-source boot chain for Rockchip's RK3588!

21/02/2024

Now included in our Debian images & available via our GitLab, you can build a complete, working BL31 (Boot Loader stage 3.1), and replace…

What's the latest with WirePlumber?

19/02/2024

Back in 2022, after a series of issues were found in its design, I made the call to rework some of WirePlumber's fundamentals in order to…

DRM-CI: A GitLab-CI pipeline for Linux kernel testing

08/02/2024

Continuing our Kernel Integration series, we're excited to introduce DRM-CI, a groundbreaking solution that enables developers to test their…

Persian Rug, Part 4 - The limitations of proxies

23/01/2024

This is the fourth and final part in a series on persian-rug, a Rust crate for interconnected objects. We've touched on the two big limitations:…

How to share code between Vulkan and Gallium

16/01/2024

One of the key high-level challenges of building Mesa drivers these days is figuring out how to best share code between a Vulkan driver…

Google Open Source Peer Bonus 2023

19/12/2023

Google Open Source have chosen their second group of winners for the 2023 Google Open Source Peer Bonus Program, and Arnaud Ferraris, Senior…

Open Since 2005 logo

We use cookies on this website to ensure that you get the best experience. By continuing to use this website you are consenting to the use of these cookies. To find out more please follow this link.

Collabora Ltd © 2005-2024. All rights reserved. Privacy Notice. Sitemap.