This post is the first part of a series that will explain how Collabora is helping shape the video virtualization story for Chromebooks. It sets the stage by discussing the importance of video data in today's world and how hardware acceleration can tackle the problem from speed, power, and heat dissipation perspectives. It then describes VirtIO Video: the protocol that has been making fast video encoding and decoding possible on Chrome OS for Android applications through ARCVM.

Collabora’s involvement in VirtIO Video started with the development of a VA-API backend for CrosVM. It has since evolved to encompass two new stand-alone projects that came to be as a direct result of this work. Throughout this series, we aim to describe where we currently stand and document the future work needed to strengthen the codec stack in ARCVM. We will explore how VirtIO Video is used within the ARCVM stack and delve into the role of cros-codecs and cros-libva, enumerating the missing components in the current solution along the way. Finally, we hope to convey our shared vision for the video virtualization landscape on Linux and beyond.

On the growth and dominance of video data

To preface this conversation, one must note that we have entered the age of ubiquitous digital video. It brought with it the quest to develop ever more performant video compression algorithms and a plethora of hardware devices capable of doing accelerated encoding and decoding on demand, independent of the main CPU in the system.

Video data has grown to dominate the share of the end-user internet traffic by far. It was estimated that, by 2022, 82% of all consumer internet traffic would be video data, up from 73% in 2017. It is safe to say that a lot of the value in modern computer devices lies in the ability to record and play video in an efficient manner from a speed, power and heat dissipation perspective.

On Chromebooks and ARCVM

Modern Chromebooks are capable of running Android applications transparently through a virtualization layer known as ARCVM. In 2021, data released by Google pointed to a whopping 50% YoY increase in the amount of users.

Not only that, but the majority of the Chromebooks out there feature some kind of hardware codec accelerator device. Thus, as more people buy into ARCVM, it becomes ever more pressing to make this capability available to Android applications as well.

VirtIO Video

VirtIO Video is, as the name suggests, a virtualization protocol for codec devices. It came into existence as a result of the work of both Google and OpenSynergy to address the open question of efficiently virtualizing the hardware codec accelerators to a guest operating system.

The idea of VirtIO devices in general is to virtualize the host’s resources through a virtual queue abstraction. Said “Virtqueues” allow for the communication between guest and host by means of exchanging memory in batches that can be acted upon asynchronously by the other party. This is known as paravirtualization, a setup wherein the guest operating system knows it is being virtualized and can thus work in tandem with the other virtualization layers to obtain a better performance. This solution is much faster than the previous approach of emulating the host device in the virtual machine manager.

To operate said queues, usually a new protocol is put in place to translate from an existing API into virtIO commands. These commands are then acted upon by the parties to effectively implement the original API as if virtualization had not taken place.

In the case of VirtIO Video, the original API is the V4L2 Memory-to-Memory Stateful API. For the unaware, V4L2 Memory-to-Memory devices have an accompanying finite state machine that encodes the different states a device can be at, as well as the valid transitions between them. Userspace applications can extract useful work from the device by issuing IOCTLs against the device’s video and media nodes to change into the desired state and perform the intended task.

It is the responsibility of the VirtIO Video’s kernel driver to convert the original V4L2 IOCTLs into valid VirtIO Video commands that can be acted upon by the host device running inside the virtual machine manager. Once the device responds, it is also the driver’s responsibility to convert the response back into what the V4L2 API expects.

The driver does these conversions and communicates with the device by means of two Virtqueues. The Command Queue is in charge of driver-to-device communication, while the Event queue implements the other direction.

VirtIO Video architecture.

CrosVM: a virtual machine manager powered by Rust

CrosVM is a virtual machine manager written in Rust with a focus on simplicity, security, and performance. While it was initially designed to work with the Chrome OS stack, where it virtualizes Linux guests as a security boundary, it can also be run on other platforms.

To put it simply, CrosVM is a central piece in Crostini and ARCVM, which together makes it possible to run native Linux and Android applications transparently on a Chromebook. It does so while employing a great amount of sandboxing to prevent rogue programs from compromising the entire machine while focusing on paravirtualized devices to provide the necessary performance when running non-Chrome OS apps.

As VirtIO Video is precisely a paravirtualized device, it is used by CrosVM to share the video codec hardware available in the Chromebook to Android guests. This comes into play when the user is running any Android application that may want to perform video encoding or decoding, such as video-conferencing apps, streaming apps, Youtube, games, and so on.

It should be noted that CrosVM, being a virtual machine manager, does not offer any encode or decode capabilities per se. This job is delegated to another component, which we will cover rather soon.

CrosVM codec backends

As previously mentioned, CrosVM cannot encode or decode video by itself. It relies on backends instead.

A backend, in this context, is a separate library that performs the actual encode or decode of video data, and this was initially simply a layer on top of the Chrome browser. The reasoning was simple: if Chrome has a fully-fledged hardware-accelerated video stack that supports both VA-API and V4L2, why not use it?

The developed solution took the form of a library called libvda (not to be confused with libva) that forwards decoding/encoding requests from CrosVM to a running Chrome instance (and back again for the result of the requests) using Mojo message passing. This design directed how accelerated video was initially implemented on CrosVM.

This meant that a dependency between CrosVM (a virtual machine manager) and Chrome (a web browser) was created. This was deemed acceptable as there were no plans to run CrosVM outside of Chrome OS back then, but was quickly regarded as undesirable as further developments let it be run on other platforms as well.

Other options included popular open-source multimedia projects such as GStreamer and FFMPEG, but this came with yet another caveat: these were massive projects written in C, a language that simply does not offer Rust’s safety guarantees. Linking to these projects in production, therefore, would be detrimental to CrosVM’s core focus on security.

Although a FFMPEG backend exists, it is used for testing purposes and to make it possible to further develop VirtIO Video without a Chromebook device. It became clear that a comprehensive, open-source Rust library to do hardware-accelerated video encoding and decoding was missing, and this is how cros-codecs and cros-libva came to be.

Stay tuned for the next installment, which discusses cros-codecs and cros-libva in great detail. Meanwhile you can check the source code here:

• cros-codecs: https://github.com/chromeos/cros-codecs
• cros-libva: https://github.com/chromeos/cros-libva
• VA-API CrosVM backend: https://chromium.googlesource.com/crosvm/crosvm/+/refs/heads/main/devices/src/virtio/video/decoder/backend/vaapi.rs