We're hiring!

New year, new kernel: Collabora's contributions to Linux 5.11

Ezequiel Garcia avatar

Ezequiel Garcia
February 17, 2021

Share this post:

Reading time:

Time continues to fly by and 2021 is finally here. A year that could be known as the year of Rust in Linux kernel! Ok, maybe we're ahead of ourselves here, but it's fun to think about that. In the meantime though, Linus has released v5.11, so let's look at what has been contributed by Collabora.

We are certainly living in very exciting times, working on a kernel that has never been more stable. Or at least, that's how one feels reading the recent discussions related to kernel version overflow.

For v5.11, Collabora's team has merged exceptional patches, which were made possible with the contributions of the awesome kernel community.

Among the highlights in this release we have: the new Syscall User Dispatch mechanism, and the destaging of the both the H.264 stateless decoding interface and the Rockchip ISP driver. Let's dive in.

Hardware-accelerated decoding in Video4Linux2

Hardware-accelerated decoding with stateless IP blocks is available in quite a few SoCs (including platforms from NXP, Rockchip and Microchip) so the V4L2 stateless decoding support is an important area of work.

Continuing our work on CODECs, in this release the V4L2 community finally stabilized the H.264 decoding stateless API. This was a requirement for FFMPEG to support the V4L2 stateless hardware-accelerated backend, which is now in progress.

If you are curious about GStreamer's v4l2codec element, we've got you covered as well.

Why is stabilizing this H.264 API interface a big deal, given H.264 is nearly 20-years-old? Despite its age (note that many of us are about double or even triple this age and we are still very alive and kicking out patches), the H.264 codec continues to be relevant.

Of course, H.264 is only one piece in the grand scheme of CODECs. The next step is to complete the V4L2 stateless decoding support for HEVC, as well as VP8 and VP9.

Rockchip Image Signal Processor driver (rkisp1) out of staging

The rkisp1 drives the camera capture functionality of Rockchip SoCs (RK3399, RK3288, RK3326, PX30), which is present in devices such as Acer Chromebook Tab 10, ROCK Pi 4 and Pinebook Pro laptops. Note the rkisp1 driver only supports RK3399 at the moment, however the media community is working on RK3288 and PX30/RK3326 support.

Merged in v5.6 under staging, Collabora has kept working on rkisp1 with cleanups, bug fixing and, most importantly, stabilizing the uAPI, so we could safely move the driver out of staging in v5.11.

And it finally happened in v5.11 In this version we stabilized the device tree definitions, added support for capturing simultaneous streams, and contributed to the definition of an uAPI that is compatible with other Rockchip SoCs variants the driver may support in the future.

We also contributed to libcamera.org and worked on Chrome OS integration to complete the full camera stack.

libcamera, an open source camera stack, works out-of-the-box for basic capturing use cases, but not yet with 3A Algorithms. Please get in touch if you are interested in an Open Source 3A implementation for Rockchip SoCs.

For Chrome OS, with a big help from Google, the upstream driver was ported and the HAL was updated, so maybe your Chromebook will come with the upstream driver in the next kernel upgrade. :)

For more technical details on the driver, check rkisp1 documentation.

Syscall User Dispatch

One of our areas of work involves improving the Linux Kernel support for Wine. In this release, our team solved an interesting problem related to intercepting, filtering and emulating system calls efficiently.

Mechanisms for intercepting system calls exist and are broadly used elsewhere, but none fit the performance requirement of modern DRM-enabled Windows applications running with multi-personalities - that is, an application that is part native (the wine part) and part foreign (the windows code).

Our solution, which we called Syscall User Dispatch, introduces a new interface that, once configured, is entirely controlled from user space. This completely eliminated costs usually associated with the filtering of system calls on the native part.

With the kernel side of this work completed in v5.11, the long-term result should be more and more modern Windows games being supported by Wine and other compatibility layers for Linux.

Introducing input inhibition

The input subsystem in the kernel is a rather old one, but that doesn't mean nothing happens there. Quite the contrary, the development is as active as elsewhere. In particular, at Collabora we worked on a new input inhibition feature.

At a basic level, inhibiting an input device simply means ignoring input events from it. When an input device driver is inhibited, events are not passed to input handlers and thus have no effect. This can in turn be exploited by drivers, for instance turning off the device and save power. The kernel exposes the feature to applications, through a sysfs "inhibited" device property.

This feature is very useful in devices such as convertible laptops. One way of using such a laptop is tablet mode, where the physical keyboard ends up just under the touchscreen. While holding your device in this mode, you can't avoid accidentally pressing the keyboard keys, but the system is there to save the day, temporarily inhibiting your keyboard.

The feature was originally introduced by the Chrome OS team a few years ago and we are more than happy to see it landing upstream in v5.11, with the help of the input community.

Now that the feature is in place, it opens new possibilities for future development. Not all input device drivers currently support inhibition, so adding it (where feasible) is an area where we might expect some activity.

User Mode Linux - now powering Debos!

Debos is used throughout Collabora to create various custom OS images from simple recipe files. An important example is Apertis where the nightly image build pipeline consists of over 50 Debos jobs, most of which could run in parallel. Debos uses the fakemachine library under-the-hood to create throw-away virtual machines to sandbox the build from the host.

Until recently, Debos was only able to use KVM to create these ephemeral virtual machines which meant that each job in our GitLab CI/CD Pipelines had to run on a machine which supported KVM. Since most of the low-end cloud machine types don't allow you to run nested virtualisation, custom machines had to be setup for these specific jobs which has ended up being a bottleneck since the machines only allowed a limited number of jobs to run in parallel.

Last year Collabora integrated User Mode Linux (UML) support into Debos and Fakemachine which allows a standard cloud machine to build OS images, albeit a little slower than those running in KVM.

As a result of this effort, a very interesting data-corruption bug was uncovered in the user-mode-linux storage driver. The issue appeared when the communication pipe to the emulated disk was overloaded, and a multiple segment I/O request was submitted by a user application. In this case, there was a possibility of an I/O request being only partially executed.

The solution landed as a lenghty patch, rewritting most of the submission/completion code to provide atomicity guarantees when executing a segmented I/O request.

Additional miscellaneous patches were merged, the first to port the guest's random-number generator driver to the more generic hwrng framework (as the original implementation blocked random number generation for around three minutes after boot) and a second patch to allow virtualised disks to have a label attached to them.

We will continue to work on User Mode Linux support for Debos so remember to subscribe to the blog!

MediaTek support

Continuing our work on MediaTek SoCs, we've merged a new power domain driver in this release, which deprecates the old one. The new driver not only adds support for the MT8183 chip, but also paves the way for the upcoming Mediatek SoCs.

As Chromebook's popularity continues to rise, these new MediaTek devices are playing an important role in the premium Chromebook scene. We are very excited to be part of the effort: coming up next in our roadmap is mainlining display panel support for the Chromebook IdeaPad Duet!

Below is a full list of contributions for the 5.11 release, including suggestions, testing and reviews, as recorded in the git commit history:

Authored (99):

Adrian Ratiu (1):

Andrzej Pietrasiewicz (3):

Boris Brezillon (5):

Christopher Obbard (2):

Dafna Hirschfeld (14):

Enric Balletbo i Serra (12):

Ezequiel Garcia (20):

Gabriel Krisman Bertazi (31):

Guillaume Tucker (2):

Helen Koike (9):

Maintainer Committed (47):

Boris Brezillon (6):

Enric Balletbo i Serra (8):

Sebastian Reichel (33):

Signed-off-by (22):

Andrzej Pietrasiewicz (1):

Dafna Hirschfeld (4):

Enric Balletbo i Serra (13):

Ezequiel Garcia (1):

Helen Koike (3):

On behalf of (30):

Dave Stevenson (1):

Eddie Cai (1):

Heiko Stuebner (3):

Jitao Shi (1):

Jonas Karlman (1):

Matthias Brugger (8):

Mauro Carvalho Chehab (1):

Michael Klein (1):

Patrik Fimml (1):

Sebastian Krzyszkowiak (5):

Shunqian Zheng (2):

Tom Rix (1):

Weiyi Lu (3):

Yongqiang Niu (1):

Reviewed-by (16):

Alyssa Rosenzweig (1):

Boris Brezillon (3):

Dafna Hirschfeld (2):

Daniel Stone (2):

Ezequiel Garcia (4):

Gabriel Krisman Bertazi (2):

Pekka Paalanen (2):

Acked-by (45):

Christopher Obbard (1):

Daniel Stone (1):

Emil Velikov (1):

Enric Balletbo i Serra (3):

Helen Koike (17):

Pekka Paalanen (3):

Tomeu Vizoso (19):

Tested-by (3):

Christopher Obbard (1):

Enric Balletbo i Serra (1):

Guillaume Tucker (1):

Reported-by (2):

Andrzej Pietrasiewicz (1):

Martyn Welch (1):


Comments (2)

  1. Danqo:
    Feb 18, 2021 at 10:55 AM

    I would love to ask about diffuculty to implement stateles decoding support for apps supporting vaapi decoding. For example I would love to see firefox supporting hw acceleration on this SOCs. How difficult it would be?
    2 question: Do you have some rk3566/rk3568 board to play with?

    Reply to this comment

    Reply to this comment

    1. mfilion:
      Feb 19, 2021 at 04:03 PM

      Thanks for dropping by! VA-API is certainly possible, but it's not something we currently have on our to-do list. Of course, please get in touch if this is something you or your company would like us to look into. As for RK3566 boards, we don't have any at this moment.

      Reply to this comment

      Reply to this comment

Add a Comment

Allowed tags: <b><i><br>Add a new comment:


Search the newsroom

Latest News & Events

Monado stays ahead: Keeping pace with OpenXR 1.1 for cross-platform, open source XR


Monado, the cross-platform open source XR runtime, has recently received significant updates to align with the features and specifications…

Blast from the past at Embedded World: Atari plays for Linux


Adhering to the fundamentals of open source, the Atari VCS OS is based on Debian using the Apertis infrastructure, and the graphics rely…

Up close and personal with STMicroelectronics' STM32MP2 at Embedded World


Using TensorFlow Lite models optimized for the STM3MP2 NPU along with an upstream-ready H.264 encoder (Video4Linux2), this demo showcases…

Open Since 2005 logo

We use cookies on this website to ensure that you get the best experience. By continuing to use this website you are consenting to the use of these cookies. To find out more please follow this link.

Collabora Ltd © 2005-2024. All rights reserved. Privacy Notice. Sitemap.