Leveraging Open Source in Video Streaming Applications

GStreamer integration for Audinate Dante AV-H and Dante Studio

Developed by Audinate, Dante AV is a complete Audio and Video over IP platform offering full control and management. Dante has been adopted by more than 500 manufacturers and is used in more than 4000 networked audio and video products worldwide. Dante AV-H is software targeting AV products implemented on Linux running on x86 and ARM processors that allows device manufacturers to embed Dante AV into their products. The software component, Dante Studio, is a suite of Windows tools designed to enhance workflows for content creators, broadcasters, and post-production professionals by integrating Dante audio with computer-based applications.

As the leading open source multimedia framework, GStreamer makes it easy for developers to create applications for handling audio, video, and other media formats. To further extend Dante's ecosystem by creating a reference implementation of Dante Video based on GStreamer, Audinate tasked Collabora and its team of experts to add GStreamer support to AV-H endpoints and Dante Studio.

Implementing for embedded

When building an embedded product, the video stream is often one of the most complex tasks. To achieve high resolution streaming, these platforms require the use of hardware encoders and decoders, in addition to the camera and display controllers. Making these work together seamlessly can be a challenge, but GStreamer makes this process much easier. Since GStreamer abstracts the various parts of the pipeline, the different elements can be swapped with implementations appropriate for the specific hardware platform. Audinate asked Collabora to develop a reference implementation of a video pipeline for Dante, ensuring it meets all requirements. This implementation is then provided to embedded partners for customization to suit their specific platforms.

Deploying the library further

In the case of Dante Studio, the same approach can be brought to Windows. The requirements of these receivers and transmitters being very specific, the best approach to implement this library was to use the appsrc and appsrc GStreamer elements to feed RTP packets or RAW video frames into a GStreamer pipeline from the application. For example, for the receiver case, the transmitter application would feed RTP packets into the pipeline and process it to receive decoded RAW video frames.

Collabora successfully implemented a higher-level library that wraps GStreamer into it, enabling receiver transmitters to easily decode H.264 and H.265 from RTP packets coming from Dante Studio, and, in the case of sender transmitters, encode video into RTP packets.

Given Dante's wide range of use cases, Collabora designed this library to be platform-independant, ensuring compatibility with Windows and Linux is supported, as well as support for hardware decoding and encoding. The library is also designed to support most types of graphics APIs including NVIDIA, D3D11, Intel, and Rockchip among others.