*

GStreamer gains V4L2 Mem-2-Mem support

Posted on 27/08/2014 by Nicolas Dufresne

Two years since my last post, that seems a long time, but I was busy becoming a GStreamer developer. This story started late in 2009. A team at Samsung (and many core Linux contributors) started adding new type of drivers to the Linux Media Infrastructure API (also known as Video4Linux 2). They introduced video decoding, encoding and video post-processing support through a class of drivers called memory-to-memory.

At the end of 2012, my employer, Collabora was chosen to implement a proof of concept, enabling hardware decoding support to the Cotton Candy, a USB stick size computer based on Samsung Exynos 4412 and built by FXI. The new element has been developed by Sebastien Dröge and was called mfcdec. All this being demonstration code, it never got close to being useful in production..

At the end of 2013, we got contracted again, to bring the demonstration code toward production code. At this point, we took the decision that we where no longer going to build an Exynos specific decoder, but instead re-use the existing GStreamer V4L2 support and do it the “right” way.

It took nearly three months, but with the help of my colleague Julien Isorce, we managed to upstream and ship hardware decoding support for the Cotton Candy. The new element is calledv4l2videoNdec, where videoN is that name of the driver node (to allow having multiple decoder at the same time). The element was well suited for static pipeline and embedded applications, but not as flexible as software decoders for desktop.

At the beginning of 2014, we started a new project with Endless Mobile. This time, the goal was to do hardware accelerated decoding also on an Exynos 4412 platform, but in a desktop environment base on Gnome Shell. Two main issues had to be addressed. The buffer pool in GstV4l2 did not track it’s memory, and the color format produced by this decoder could not be color converter using GLES2 shader (not enough coordinate precision). We had to implement a custom memory allocator and rewrite most of the v4l2 buffer pool code. To handle the color format, we had to implement an element that wraps hardware video converter in order to obtain video frames in a format that can be uploaded to GLES2.

As of today, all this effort has landed into GStreamer and is now part of 1.4 GStreamer release. Some of my colleagues went even further by demonstrating during SIGGRAH the benefits of using V4L2 decoder when combining DMABUF and Wayland. Other team, including Pengutronix on Freescale CUDA and STE have started testing against this new promising decoder which finally brings a standard and low level way of decoding medias on Linux.

Original post

Related Posts

Related Posts

Comments (0)


Add a Comment





Allowed tags: <b><i><br>Add a new comment:


Latest Blog Posts

GStreamer spring in Sweden

18/05/2018

Earlier this month, Olivier Crête, Nicolas Dufresne, George Kiagiadakis & I attended the GStreamer Spring Hackfest in Lund, Sweden. Hosted…

GPU virtualization update

09/05/2018

A few months ago, Robert Foss wrote a blog post about virtualizing GPU Access. Here's a look at some of the major improvements that have…

GStreamer Spring Hackfest

01/05/2018

Generously hosted by Axis in the beautiful Lund, Sweden, the annual spring hackfest is an occasion for the community to get together to…

Upstream Linux support for new NXP i.MX 8

17/04/2018

The i.MX 6 platform has for the past few years enjoyed a large effort to add upstream support to Linux and surrounding projects. The newly…

A new era for Linux's low-level graphics - Part 2

23/03/2018

Following on from part 1 in the series, part 2 covers more developments in low-level graphics, including support for buffer modifiers in…

A new era for Linux's low-level graphics - Part 1

20/03/2018

The latest enhancements to the DRM subsystem have made mainline Linux much more attractive, making drivers easier to write, applications…

Open Since 2005

We use cookies on this website to ensure that you get the best experience. By continuing to use this website you are consenting to the use of these cookies. To find out more please follow this link.

Collabora Ltd © 2005-2018. All rights reserved. Website sitemap.