We're hiring!
*

GStreamer gains V4L2 Mem-2-Mem support

Nicolas Dufresne avatar

Nicolas Dufresne
August 27, 2014

Share this post:

Two years since my last post, that seems a long time, but I was busy becoming a GStreamer developer. This story started late in 2009. A team at Samsung (and many core Linux contributors) started adding new type of drivers to the Linux Media Infrastructure API (also known as Video4Linux 2). They introduced video decoding, encoding and video post-processing support through a class of drivers called memory-to-memory.

At the end of 2012, my employer, Collabora was chosen to implement a proof of concept, enabling hardware decoding support to the Cotton Candy, a USB stick size computer based on Samsung Exynos 4412 and built by FXI. The new element has been developed by Sebastien Dröge and was called mfcdec. All this being demonstration code, it never got close to being useful in production..

At the end of 2013, we got contracted again, to bring the demonstration code toward production code. At this point, we took the decision that we where no longer going to build an Exynos specific decoder, but instead re-use the existing GStreamer V4L2 support and do it the “right” way.

It took nearly three months, but with the help of my colleague Julien Isorce, we managed to upstream and ship hardware decoding support for the Cotton Candy. The new element is calledv4l2videoNdec, where videoN is that name of the driver node (to allow having multiple decoder at the same time). The element was well suited for static pipeline and embedded applications, but not as flexible as software decoders for desktop.

At the beginning of 2014, we started a new project with Endless Mobile. This time, the goal was to do hardware accelerated decoding also on an Exynos 4412 platform, but in a desktop environment base on Gnome Shell. Two main issues had to be addressed. The buffer pool in GstV4l2 did not track it’s memory, and the color format produced by this decoder could not be color converter using GLES2 shader (not enough coordinate precision). We had to implement a custom memory allocator and rewrite most of the v4l2 buffer pool code. To handle the color format, we had to implement an element that wraps hardware video converter in order to obtain video frames in a format that can be uploaded to GLES2.

As of today, all this effort has landed into GStreamer and is now part of 1.4 GStreamer release. Some of my colleagues went even further by demonstrating during SIGGRAH the benefits of using V4L2 decoder when combining DMABUF and Wayland. Other team, including Pengutronix on Freescale CUDA and STE have started testing against this new promising decoder which finally brings a standard and low level way of decoding medias on Linux.

Original post

Related Posts

Related Posts

Comments (0)


Add a Comment






Allowed tags: <b><i><br>Add a new comment:


Search the newsroom

Latest Blog Posts

Using syzkaller, part 4: Driver fuzzing

26/06/2020

Syzkaller is much needed tool for Linux kernel testing and debugging. With some work, it can also be enhanced to find bugs in specific drivers,…

Cross building Rust GStreamer plugins for the Raspberry Pi

23/06/2020

Previously, we discussed about how Rust can be a great language for embedded programming. In this article, we'll explain an easy setup to…

Generating MPEG-DASH streams for Open Source adaptive streaming with GStreamer

12/06/2020

Adaptive streaming is a technique to provide flexibility and scalability by offering variable bit-rate streams to the client. Here's a quick…

Bifrost meets GNOME: Onward & upward to zero graphics blobs

05/06/2020

With only free software, a Mali G31 chip can now run Wayland compositors with zero-copy graphics, including GNOME 3. We can run every scene…

Using regmaps to make Linux drivers more generic

27/05/2020

Device drivers can support more revisions and SoC platforms by abstracting away specific hardware interface layouts. Let's examine a specific…

Cross-compiling with gst-build and GStreamer

15/05/2020

gst-build is one of the main build systems used by the community to develop the GStreamer platform. In my last blog post, I presented gst-build…

Open Since 2005 logo

We use cookies on this website to ensure that you get the best experience. By continuing to use this website you are consenting to the use of these cookies. To find out more please follow this link.

Collabora Ltd © 2005-2020. All rights reserved. Privacy Notice. Sitemap.