February 12, 2020
For remote security surveillance, like monitoring an industrial facility where expensive equipment or even human lives might be at stake, maintaining an immediate and high quality video streaming from the areas of interest is a must. With the advent of 5G networks, it's now possible to stream high quality video in real-time with a very low latency that wasn't possible with the past generations of mobile networks. In this domain, the SRT protocol has been picking up speed, and thanks to
srtsink elements available since GStreamer 1.16 (see Olivier Crête's blog post) it's now easier than ever to incorporate low latency streaming into your application.
Here at Collabora we've been lately participating in the design and development of Hwangsaeul—a next generation security video feeds streaming platform with one of our customers, SK Telecom Co..
Hwangsaeul is a cloud relay service that gathers live security video feeds from different locations into a single service to which clients can connect to watch the feeds. Additionally, it also enables continuous recording of each feed. The SRT protocol is utilized by both camera-to-relay and relay-to-client transport in order to minimize latency.
|Figure 1. Hwangsaeul architecture. (Image courtesy of Jeongseok Kim @ SK Telecom Co.)|
The platform consists of a collection of modules communicating through RabbitMQ message broker. For this introduction, three components will be important. The names are in Korean. (Hwangsaeul itself means "stork's nest". It is the street in Seongnam where SK Telecom offices were located.)
Gaeul (개울, brook) is a streaming server that runs on each edge node contributing a video stream to the service. It accesses the video capture device through V4L2 and registers the node with the platform's message broker, allowing video capture and streaming to be remotely controlled from the platform's relay service.
Hwangsae (황새, stork) fulfills two distinct roles: it implements the core SRT relay that provides M:N streaming of the content (capture from one camera may be viewed by many clients at a time) and can record streams into MPEG-TS or MP4 files for later viewing.
Chamge (참게, a small crab) takes care of communication between platform components. On one hand it provides a wrapper around RabbitMQ client library for other services, on the other it implements the Arbiter service, which keeps track of all video capture and relay nodes registered with the platform and facilitates their interaction.
Hwangsaeul is being developed under Apache 2.0 license with the source codes hosted on GitHub. Nightly builds are also available as binary packages for Ubuntu 18.04 and 19.04. To try the platform in its present development shape, add the following PPA:
sudo add-apt-repository ppa:hwangsaeul/nightly sudo apt-get update
In production, the components would be divided across multiple servers with distinct purposes as stream sources, relays, recorders. Here we can, for the sake of simplicity, install everything on one machine:
apt-get install gaeul hwangsae-relay hwangsae-recorder chamge-arbiter
The platform needs a running AMQP broker instance, which can be either installed on-premises, or for our purposes a free account on a cloud service like [cloudamqp.com](https://www.cloudamqp.com/) will be sufficient as well. In any case you'll need your broker's URI to configure Hwangsaeul.
As the next step, edit the parameters in `/etc/hwangsaeul.conf`. The excerpt below shows the minimal configuration. You may change the V4L2 device node to match the camera you want to use for streaming, and don't forget to set the correct URI of your AMQP broker.
[org/hwangsaeul/Gaeul] video-device='/dev/video0' video-source='v4l2src' encoding-method='general' [org/hwangsaeul/Chamge1/Edge/AMQP] amqp-uri='amqp://user:email@example.com/user' [org/hwangsaeul/Chamge1/Arbiter/AMQP] amqp-uri='amqp://user:firstname.lastname@example.org/user' [org/hwangsaeul/Chamge1/Hub/AMQP] amqp-uri='amqp://user:email@example.com/user'
When the configuration is in place, make sure you restart all Hwangsaeul services.
systemctl restart chamge-arbiter systemctl restart gaeul systemctl restart hwangsae-relay systemctl restart hwangsae-recorder
Hwangsaeul is being developed as a platform that integrates into 3rd party applications. To that end, it exposes its functionality as a set of D-Bus interfaces. Here is a couple of examples demonstrating its usage.
In order to initiate a video stream, we first need to know its source. Streaming sources in Hwangsaeul are identified by their unique Edge ID, which can be obtained from its respective
gaeul process over D-Bus using
$ busctl introspect org.hwangsaeul.Gaeul /org/hwangsaeul/Gaeul/Manager org.hwangsaeul.Gaeul.Manager NAME TYPE SIGNATURE RESULT/VALUE FLAGS .GetEdgeId method - s - .State property i 0 emits-change $ busctl call org.hwangsaeul.Gaeul /org/hwangsaeul/Gaeul/Manager org.hwangsaeul.Gaeul.Manager GetEdgeId s "9c51b16460d2186ec3986cd37640999edbd16719825a8b2f3cf85028208a9879"
Streaming itself is controlled through
org.hwangsaeul.Hwangsae1.EdgeInterface of Hwangsae relay agent:
$ busctl introspect org.hwangsaeul.Hwangsae1.RelayAgent /org/hwangsaeul/Hwangsae1/EdgeInterface \ org.hwangsaeul.Hwangsae1.EdgeInterface NAME TYPE SIGNATURE RESULT/VALUE FLAGS .ChangeParameters method siiii - - .Delete method s - - .GetEdgeStatus method s u - .Register method su - - .Start method siiii s - .Stop method s s - .Edges property u 0 emits-change
To start streaming from the edge device whose ID we got in the previous step, one can run:
$ busctl call org.hwangsaeul.Hwangsae1.RelayAgent /org/hwangsaeul/Hwangsae1/EdgeInterface org.hwangsaeul.Hwangsae1.EdgeInterface \ Start 'siiii' '9c51b16460d2186ec3986cd37640999edbd16719825a8b2f3cf85028208a9879' 640 480 30 2048000
The call arguments that follow the Edge ID are the requested width, height, framerate and bitrate in bits per second of the stream. Upon receiving the command, Hwangsaeul will turn on the camera and initiate SRT connection from the edge device to the relay service. If everything goes well, the D-Bus call will return a URL on the relay server from which clients can play the video stream.
gst-play-1.0 tool supports
srt:// URLs, but they should also work in recent versions of common media players like VLC or GNOME Videos (Totem).
|Figure 2. Receiving a SRT stream with gst-play-1.0.|
Stream recording is controlled through
org.hwangsaeul.Hwangsae1.RecorderInterface D-Bus interface:
$ busctl introspect org.hwangsaeul.Hwangsae1.RecorderAgent /org/hwangsaeul/Hwangsae1/RecorderInterface org.hwangsaeul.Hwangsae1.RecorderInterface NAME TYPE SIGNATURE RESULT/VALUE FLAGS .Delete method ss - - .LookupByEdge method sxx a(ssxxx) - .LookupByRecord method sxx sa(sxxx) - .Start method s s - .Stop method s - - .Url method ss s -
To begin recording from an edge device, we only need to know its ID.
$ busctl call org.hwangsaeul.Hwangsae1.RecorderAgent /org/hwangsaeul/Hwangsae1/RecorderInterface org.hwangsaeul.Hwangsae1.RecorderInterface \ Start 's' '9c51b16460d2186ec3986cd37640999edbd16719825a8b2f3cf85028208a9879'
Hwangsaeul will continuously save the stream into a file. In the case when the connection with the stream source gets broken, the recorder will keep making attempts to re-establish it until explicitly told to stop.
Stopping the recording is straightforward too:
$ busctl call org.hwangsaeul.Hwangsae1.RecorderAgent /org/hwangsaeul/Hwangsae1/RecorderInterface org.hwangsaeul.Hwangsae1.RecorderInterface \ Stop 's' '9c51b16460d2186ec3986cd37640999edbd16719825a8b2f3cf85028208a9879'
The file we've just created is kept in the recorder's internal storage. Hwangsaeul provides several methods for searching through the directory of recordings; we will use
<!-- LookupByEdge: @edgeId: a unique ID of an edge device @from: start timestamp of the time range in which to search in microseconds since Jan 01 1970 @to: end timestamp of the time range in which to search in microseconds since Jan 01 1970 @records: an array where each item represents a found recording Looks up recordings from an edge device taken during given time range. The meaning of the values in each array item is: recording ID, fileID, recording start, recording end (both μs since Epoch) and file size in Bytes. --> <method name="LookupByEdge"> <arg name="edgeId" direction="in" type="s"/> <arg name="from" direction="in" type="x"/> <arg name="to" direction="in" type="x"/> <arg name="records" direction="out" type="a(ssxxx)"/> </method>
So, to find our recording run the following request. Because we don't want to filter the results by any time range, passing
to will encompass all possible timestamps.
$ busctl call org.hwangsaeul.Hwangsae1.RecorderAgent /org/hwangsaeul/Hwangsae1/RecorderInterface org.hwangsaeul.Hwangsae1.RecorderInterface \ LookupByEdge 'sxx' '9c51b16460d2186ec3986cd37640999edbd16719825a8b2f3cf85028208a9879' 0 0xffffffffffffff
The call returns one entry. Meaning of the values is described above.
a(ssxxx) 1 "1580479418277126" "1580479418277126-1580479418631803-1580479735978654" 1580479418631803 1580479735978654 140987404
Let's download the file. Normally, the recordings would be served by a full-fledged server like NGINX. In our demonstration we'll take advantage of the small HTTP server included in Hwangsae Recorder. First we need to convert the ID of the requested file to its URL:
<!-- Url: @edgeId: a unique ID of an edge device @fileId: a unique ID of a recording file @url: URL using which the recording can be downloaded from the embedded HTTP server. Gets the URL of a file. --> <method name="Url"> <arg name="edgeId" direction="in" type="s"/> <arg name="fileId" direction="in" type="s"/> <arg name="url" direction="out" type="s"/> </method>
Thus, knowing edge and file IDs:
$ busctl call org.hwangsaeul.Hwangsae1.RecorderAgent /org/hwangsaeul/Hwangsae1/RecorderInterface org.hwangsaeul.Hwangsae1.RecorderInterface \ Url 'ss' '9c51b16460d2186ec3986cd37640999edbd16719825a8b2f3cf85028208a9879' '1580479418277126-1580479418631803-1580479735978654' s "http://192.168.47.155:8090/9c51b16460d2186ec3986cd37640999edbd16719825a8b2f3cf85028208a9879/1580479418277126-1580479418631803-1580479735978654"
Now we can download the recording using your preferred tool.
$ wget --content-disposition http://192.168.47.155:8090/9c51b16460d2186ec3986cd37640999edbd16719825a8b2f3cf85028208a9879/1580479418277126-1580479418631803-1580479735978654 --2020-02-04 16:00:55-- http://192.168.47.155:8090/9c51b16460d2186ec3986cd37640999edbd16719825a8b2f3cf85028208a9879/1580479418277126-1580479418631803-1580479735978654 Connecting to 192.168.47.155:8090... connected. HTTP request sent, awaiting response... 200 OK Length: 140987404 (134M) Saving to: ‘hwangsae-recording-1580479418277126-1580479418631803-1580479735978654.ts’ hwangsae-recording-1580479418277126- 100%[======================================================================>] 134.46M 779MB/s in 0.2s 2020-02-04 16:00:55 (779 MB/s) - ‘hwangsae-recording-1580479418277126-1580479418631803-1580479735978654.ts’ saved [140987404/140987404]
Hwangsaeul streaming service is still in its early development stage, but it already provides the most fundamental APIs for 3rd party consumption. A key missing requirement is stream encryption, ideally through a mechanism that would be compatible also with common web browsers. There are also possibilities to extend SRT support in GStreamer, for example by implementing network adaptive encoding in order to improve the service quality.
Our team at Collabora would love to help you integrate SRT into your platform, using GStreamer, ffmpeg, VLC or your own multimedia framework. Contact us today to see how we can help!
The Panfrost project started as a reverse engineering effort to understand Arm Mali Midgard and Bifrost GPU internals. With the driver getting…
Join us this week at the Spring edition of Linaro Virtual Connect, as we discuss bringing stateless video decoding support to Linux, and…
One year ago, we announced a new partnership with Microsoft to build OpenGL mapping layers to DirectX 12. Today, we're excited to share…