GStreamer 1.24 was recently released, and with it came a new plugin to help separate pipelines into different processes.

The new unixfd plugin provides a sink and source elements. They communicate over a unix socket to provide a zero-copy transfer of buffers across pipelines in different processes. Buffers must be backed by file-descriptor memories, such as memfd (see below) or DMABUF (e.g. v4l2src).

One unixfdsink can have multiple unixfdsrc clients, acting like a cross process "tee". However, no caps negotiation happens between pipelines; caps are negotiated by the upstream pipeline alone and then sent from unixfdsink to unixfdsrc. Events and queries are also not currently transmitted, except for caps and EOS.

Example:

$ gst-launch-1.0 v4l2src device=/dev/video0 ! unixfdsink socket-path=/tmp/mysocket
$ gst-launch-1.0 unixfdsrc socket-path=/tmp/mysocket ! queue ! autovideosink
$ gst-launch-1.0 unixfdsrc socket-path=/tmp/mysocket ! queue ! autovideosink

Here we have tree pipelines in different processes transferring webcam frames as DMABUF (depending on webcam capabilities) and displayed twice.

In addition, unixfdsink will propose the new shared memory allocator (GstShmAllocator) in its allocation query. On Linux it uses memfd by default, but it can use POSIX shm_open as fallback. This allows for example videotestsrc to generate frames directly into file-descriptor memories that can then be transferred between processes with no copy:

$ gst-launch-1.0 videotestsrc ! unixfdsink socket-path=/tmp/mysocket
$ gst-launch-1.0 unixfdsrc socket-path=/tmp/mysocket ! queue ! videoconvert ! autovideosink

In order to be able to transmit raw video frames, metadata must also be transferred, such as GstVideoMeta. New functions gst_meta_serialize() and gst_meta_deserialize() have been implemented on most GstMeta to transfer them over the unix socket. That API could also be used in the future to store buffer metadata into a file using e.g. <codegdppay.

Here's a look at this new plugin in action:

We thank our partners at Netflix for sponsoring this work.

Related Posts

GStreamer 1.24: New heights in embedded, network streaming & analytics

A framework to share analytics data in GStreamer

GStreamer 1.22: Bigger and better!

Related Posts

GStreamer 1.24: New heights in embedded, network streaming & analytics

A framework to share analytics data in GStreamer

GStreamer 1.22: Bigger and better!