*

Making Viewer UIs for Pitivi

Lubosz Sarnecki avatar

Posted on 13/10/2016 by Lubosz Sarnecki

Share this post:

Being someone who has already experimented with two transformation box approaches for Pitivi in the past, maintainers thought I might be the right person to do a modern one.

Creating a user interface for a video transformation requires three things:

  • The implementation of the transformation
  • A way to draw the widgets over the viewer and
  • Mapping the input to the reverse transformation

The transformation

First of all the implementation of the transformation, which is in our case scaling and translation, is currently done by GES.UriSource, calculated on the CPU. In the first Pitivi transformation box I did in GSoC 2012 this was done by the notorious Frei0r plugins from GStreamer Plugins Bad, which is also a CPU implementation. In the second version this was done on the GPU with the gltransformation element I wrote for GSoC 2014.

A method to draw widgets over the viewer

In Pitivi’s case, the viewer is a GStreamer sink. In all three versions rendering of the overlay widgets was done by Cairo, but it was done differently for all three implementations, since they all used different sinks.

2012

The first one used a hacky solution where the sink and cairo drew in the same Gtk drawing area, acquired for GStreamer with the Gst Overlay API. Many Gtk and GStreamer devs wondered how this worked at all. This and Pitivi switching to more modern sinks was the reason why the first version of the box didn’t stay upstream for long.


lubosz-the-magician

2014

Still using Gst Overlay API, but this time with the glimagesink. The cairo widgets are rendered into OpenGL textures and composed in the glimagesink draw callback in the GStreamer GL context. Worked pretty smooth for me, but didn’t provide a fallback solution for users without GL. Clearly an approach for the future, but how about something solid?

screenshot-from-2016-09-26-16-04-40

2016

Now we have the almighty GtkSink in Pitivi. It is a Gtk widget and overlays can be added via Gtk Overlay. The sink is also exchangeable with GtkGLSink, which uses a GStreamer GL context to display the video texture and also can use GStreamer GL plugins like gltransformation without needing to download the GPU memory with gldownload. Cairo rendering now can be easily added over GStreamer sinks, yay.

screenshot-from-2016-09-26-16-03-27

Linking the UI with the components doing the transformation

The mapping of the input from the UI to the transformation is clearly dependent on the transformation you are using. In the 2012 version I needed to map the input to frei0r^-1. In 2014 I used an OpenGL Model-View-Projection matrix calculated in Graphene, which could also do rotations and 3D transformations (we have a z-axis, yay).

The 2016 implementation uses the inverse transformation for the GES.UriSource transformation, which is done by the GStreamer elements videomixer and videoscale. Of course things like keeping aspect ratio, maintaining limits and transforming Gtk widget coordinates to the transformation’s coordinates are part of this 3rd ingredient.

Extensibility

The new transformation box fits great with Pitivi by making clips selectable from the viewer, so you can manage multiple overlapping clips quite easily. But the best part of this implementation may be its extensibility. I already made two overlay classes, one for the normal clips which uses a GES.UriSource transformation and one for title clips aka GES.TextSource, which is using different coordinates and different GStreamer plugins. In this fashion other plugins can be written for the Pitivi viewer, for example for 3D transformations with gltransformation. Or you could do crazy stuff like a UI for barrel distortion etc.

Clone the code and contribute!

If you have any questions about this ask me. I’m lubosz on Freenode IRC.

Original post

Related Posts

Related Posts

Comments (0)


Add a Comment






Allowed tags: <b><i><br>Add a new comment:


Latest Blog Posts

Quick hack: Speed up your GitLab CI

06/11/2018

Did you know you could register your own PC, or a spare laptop collecting dust in a drawer, to get instant CI going on GitLab? Not only…

Introducing Zink, an OpenGL implementation on top of Vulkan

31/10/2018

For the last month or so, I've been playing with a new project during my work at Collabora, and as I've already briefly talked about at…

On the low adoption of automated testing in FOSS

18/10/2018

For projects of any value and significance, having a comprehensive automated test suite is nowadays considered a standard software engineering…

Recently in Geoclue

12/10/2018

After I started working for Collabora in April, I've finally been able to put some time on maintenance and development of Geoclue again.…

The beauty of Open Source

10/10/2018

Like all software, Open Source software isn't without it's bugs and issues. However, thanks to the nature of Open Source, resolving or mitigating…

MicroDebConf Brasilia

02/10/2018

Last month, the first "MicroDebConf" took place at the Gama campus of the University of Brasilia. Here's a look at how this one day event…

Open Since 2005 logo

We use cookies on this website to ensure that you get the best experience. By continuing to use this website you are consenting to the use of these cookies. To find out more please follow this link.

Collabora Ltd © 2005-2018. All rights reserved. Website sitemap.