*

GStreamer gains V4L2 Mem-2-Mem support

Posted on 27/08/2014 by Nicolas Dufresne

Two years since my last post, that seems a long time, but I was busy becoming a GStreamer developer. This story started late in 2009. A team at Samsung (and many core Linux contributors) started adding new type of drivers to the Linux Media Infrastructure API (also known as Video4Linux 2). They introduced video decoding, encoding and video post-processing support through a class of drivers called memory-to-memory.

At the end of 2012, my employer, Collabora was chosen to implement a proof of concept, enabling hardware decoding support to the Cotton Candy, a USB stick size computer based on Samsung Exynos 4412 and built by FXI. The new element has been developed by Sebastien Dröge and was called mfcdec. All this being demonstration code, it never got close to being useful in production..

At the end of 2013, we got contracted again, to bring the demonstration code toward production code. At this point, we took the decision that we where no longer going to build an Exynos specific decoder, but instead re-use the existing GStreamer V4L2 support and do it the “right” way.

It took nearly three months, but with the help of my colleague Julien Isorce, we managed to upstream and ship hardware decoding support for the Cotton Candy. The new element is calledv4l2videoNdec, where videoN is that name of the driver node (to allow having multiple decoder at the same time). The element was well suited for static pipeline and embedded applications, but not as flexible as software decoders for desktop.

At the beginning of 2014, we started a new project with Endless Mobile. This time, the goal was to do hardware accelerated decoding also on an Exynos 4412 platform, but in a desktop environment base on Gnome Shell. Two main issues had to be addressed. The buffer pool in GstV4l2 did not track it’s memory, and the color format produced by this decoder could not be color converter using GLES2 shader (not enough coordinate precision). We had to implement a custom memory allocator and rewrite most of the v4l2 buffer pool code. To handle the color format, we had to implement an element that wraps hardware video converter in order to obtain video frames in a format that can be uploaded to GLES2.

As of today, all this effort has landed into GStreamer and is now part of 1.4 GStreamer release. Some of my colleagues went even further by demonstrating during SIGGRAH the benefits of using V4L2 decoder when combining DMABUF and Wayland. Other team, including Pengutronix on Freescale CUDA and STE have started testing against this new promising decoder which finally brings a standard and low level way of decoding medias on Linux.

Original post

Comments (0)


Add a Comment





Allowed tags: <b><i><br>Add a new comment:


Latest Blog Posts

FOSDEM - Links to recorded presentations (videos)

21/02/2018

From an introduction to Flatpak, to managing build infrastructure of a Debian derivative, to modern tools to debug GStreamer, Collaborans…

SRT, typical examples

20/02/2018

Released earlier this month, the latest version of VLC, the free & open source multimedia player (which also uses the GStreamer framework)…

SRT in GStreamer

16/02/2018

Transmitting low delay, high quality video over the Internet is hard. The trade-off is normally between video quality and transmission delay…

LVEE Winter Edition 2018

13/02/2018

Following a great weekend in Brussels for FOSDEM, Collaborans headed east to Belarus to attend & speak at the winter session of the international…

Virtualizing GPU Access

12/02/2018

For the past few years a clear trend of containerization of applications and services has emerged. Having processes containerized is beneficial…

Kernelci.org automated bisection

16/01/2018

The kernelci.org project aims at continuously testing the mainline Linux kernel, from stable branches to linux-next on a variety of platforms.…

Open Since 2005

We use cookies on this website to ensure that you get the best experience. By continuing to use this website you are consenting to the use of these cookies. To find out more please follow this link.

Collabora Ltd © 2005-2018. All rights reserved. Website sitemap.