Collabora

I’m back from this year’s GStreamer hackfest, which was fantastic as usual — an intersection of great minds, big challenges, flaky Wi-Fi and good food. Christian already did a generic summary, so I’ll be narrating from the GNonLin/GES/PiTiVi perspective. See the end of this blog post for a nice video retrospective.

Edward provided an initial patch to improve the behavior of timestamps and seeking in GNonLin, while Nicolas “Stormer” Dufresne fixed two bugs causing deadlocks. Nicolas spent a lot of time discussing with Wim Taymans, Edward Hervey, Sebastian Dröge and other hackfesters about the architecture of GNonLin in light of GStreamer 1.x. He also fixed looping for the Ogg demuxer in pull mode and, with some help from Mathieu “Forest Ranger” Duponchelle, fleshed out the design for a new tree data structure for GNonLin.

Mathieu the Moustached Avenger worked on implementing keyframes in GES, paving the way for him to create a user interface to animate any effect property in PiTiVi. That user interface will most likely depend on him working on the clutter timeline canvas, so I’m looking forward to improvements in that area.

Thibault “Keyboard Crusher” Saunier finished the implementation of GES Containers and clip groups, then worked on implementing — at long last — audio mixing in GES. This is an essential feature of multitrack audio/video editing, and I’m really happy to see that feature make its comeback for the next release of PiTiVi. This work will also depend on Mathieu’s keyframes UI. A proper reimplementation of video mixing remains to be done, however.

There are lots of outstanding things to solve in GNonLin and GES. Nicolas has a bunch of ideas for things to improve and redesign in GNonLin and I expect much collaboration between Thibault and him to optimize the entire stack for better reliability and performance (for example, adding caps filters to allow realtime downscaling of videos to improve preview performance, configurable downstream buffering for playback to avoid frame drops in CPU-intensive parts of a timeline, etc.). GNonLin and GES have much potential to allow us to be a lot smarter than before.

Personally, I spent most of my time testing, discussing and hacking on some new features for PiTiVi.

• I added a button in the timeline toolbar that toggles the “gapless mode” (automatic ripple edits), which makes your clips behave like magnets and prevents needing to re-arrange them manually all the time. The feature works and will be merged after a customary code review.
• I made some progress on the custom effects UI branch. Once it’s complete, you will be able to easily create custom user interfaces for effects that require it, simply by using a glade/gtkbuilder .ui file (or, if you prefer, a set of widgets from your own python module). Of course, for the majority of effects, our automatically generated user interfaces are still good enough, so we can keep using them and avoid unnecessary work.

(See my previous blog post for a situation report on where we stood with PiTiVi before the hackfest)

I also spent a bit of time setting up my film making gear and shooting various interesting moments of the hackfest. Here’s my montage, which will tell the story much better than a long blog post. Hope you’ll like it:

I would like to thank Collabora for allowing many GStreamer contributors to attend the hackfest, which I consider vital to the health of the GStreamer community. I was happy to meet again with many friends and help push the Free multimedia stack forward. Props to Christian Schaller and Alessandro Decina for organizing the whole thing, too!

Aside from Collabora and Fluendo sponsoring two of our dinners (thanks!), I would also like to thank you, PiTiVi supporters, for making it possible for me to spend some money to thank GStreamer contributors with some food and beer — maximum boost to the GStreamer community! Full disclosure: I used 84 euros worth of PiTiVi donations for that purpose.

Hi, KDE users. I think it is time we had an adult conversation about KMix and usability.

You see, in my last post, there was an awful lot of vitriol spewed about my heavy handed decision to use horizontal sliders. My personal belief is that vertical sliders have a tendency to induce vertigo in older users and ex-paratroopers. There is no way I can live my life with that kind of guilt on my conscious. Paratroopers serve an incredibly useful purpose in our military operations. I simply cannot allow these great ‘murricans to experience pain at my hands.

After much thought and discussion of usability with an awfully friendly bartender (and much tequila), I’ve decided to implement a compromise in KMix2:

Down and to the left means louder!

I hope you’re happy, because I certainly am.

I realize that this is a very controversial change, so I’ve created a kde pastebin entry to maintain this branch. We all know that KDE git is totally unreliable and the sysadmins are convinced that backups are not backups, so no sense in trying to stay there. The geniuses at Hacker News (who are never wrong or sensationalist) tried unsuccessfully to convince the sysadmins otherwise, so I must conclude that I have no reason to keep faith in them.

Time for a little report on recent improvements in Pitivi. Nothing earth-shattering to make you drool with envy; just a lot of fixes, cleanup and improvements to small details. Next week, we will be in Milan for the GStreamer hackfest, so I’ll make sure to give you a nice report on what we managed to accomplish there.

Here’s a list of fixes in Pitivi since my last blog post:

• Fixes for our automated UI tests, including interaction with the filechooser dialogs, the spinbutton widgets (see this bug report), the typing speed, etc.
• Fixes and cleanup for backend tests
• Enforce unicode in preset names, preventing a bug with non-ASCII chars are used in the name of a preset
• Allow presets with “/” in their name
• Fix drag and drop from the media library’s listview mode
• Prevent playing back clip previews in double (that was a subtle one, since windows were exactly on top of each other)
• Make the media library clip previewer work even when the application is in fullscreen mode
• Make effect properties work again
• Take into account project settings in the main window when loading a project
• Conform to the new version of the Freedesktop specification for thumbnail directories
• Make special characters show up correctly in the media library’s iconview mode, remove the ancient filename shortening code and rely on Pango instead.
• Properly handle clip URIs encoding, ensuring that we can find the thumbnails for all files. Also do last-minute checks for that encoding before computing thumbnail hashes.
• Avoid excessive work when searching clips in the media library, improving performance
• Handle special characters in the media library’s search entry
• Force calculating the toolbar height at the last minute to properly handle all screen setups in fullscreen
• Scale down effects thumbnails to fit better in the new listview arrangement
• Fix a race between clicks on the preview widget’s slider and position updates. When using it in the media library/file chooser, the slider would often “jump” back to its previous position instead of seeking. The new behavior is now smooth and reliable.
• Ensure the welcome dialog is initially centered on its parent
• Bring back the checkbox to enable/disable effects
• Initialize GES to prevent a segfault from occurring on some machines at startup
• Make rectangle selection work again in the media library’s iconview mode
• Make the “Restore from backup” dialog modal to the main window

New features:

• Automatically adjust the zoom when inserting to the end of the timeline
• When the viewer is undocked, provide a button to toggle fullscreen mode
• New icons for split, group/ungroup and align
• Specify the duration of missing/moved files when prompting the user about their new location
• Update effect categories, merge “Noise” and “Blur”, add a “Compositing” category, categorize new effects
• Automatically save the last used render directory
• Stop rendering when the user presses Escape
• Use symbolic icons everywhere where it makes sense (in the media library toolbar, property reset buttons, lists, etc.)
• Use the system’s default image viewer to preview images from the media library
• Update the preview widget slider on a more frequent basis, giving it a snappier feeling
• Automatically save and restore the main window’s position. This is especially useful when using detached utility windows.
• Hide the effects toolbar when nothing is selected
• Add a contextual help button in the render dialog to explain container formats
• Allow entering a frame number into the time widget

Architectural changes and cleanups:

• Clean up the build, prune useless files, simplify and centralize the list of dependencies
• Refactor dependency checks to be more reliable and provide a faster application startup
• GES Assets (implemented in GES and integrated in Pitivi), including the notion of GES Project among other things. This paves the way to many other improvements.
• A revised and simplified API has been implemented in GES and integrated in Pitivi. You can see this new API online in the GES API hierarchy documentation.

So yeah, that’s the summarized version… with refactoring and cleanup all over the place, there’s more than a hundred commits (excluding translations).

Stuff that still needs to be done (as always, we need your help here):

• Need to refactor the timeline UI code and the clip transformation UI
• Undo/redo
• Improving/expanding the UI tests, see the test suite wishlist
• Figure out how to better integrate the “welcome dialog”
• Figure out how to shave off the menu bar and how GTK AppMenu will work with accessibility technologies (and Dogtail)
• Improve the title editor UI
• Improve the timeline thumbnailer
• Reimplement the notion of grouping in GES
• Reimplement video compositing and audio mixing. Properly. In GES.
• Make a new keyframe UI that doesn’t suck. Integrate it for audio volume curves and to provide animatable effect properties.
• Fix… all the bugs! _o/
• Insert your own itch to scratch here

Ongoing work in my own/personal repository:

• Custom effect UIs
• New high-resolution icon (pictured above), new mimetypes
• Automatic rippling
• Resurrecting the codecs installer

Update: This isn't actually that much better than letting them access the private key, since nothing is stopping the user from running their own SSH agent, which can be run under strace. A better solution is in the works. Thanks Timo Juhani Lindfors and Bob Proulx for both pointing this out.

At work, we have a shared SSH key between the different people manning the support queue. So far, this has just been a file in a directory where everybody could read it and people would sudo to the support user and then run SSH.

This has bugged me a fair bit, since there was nothing stopping a person from making a copy of the key onto their laptop, except policy.

Thanks to a tip, I got around to implementing this and figured writing up how to do it would be useful.

First, you need a directory readable by root only, I use /var/local/support-ssh here. The other bits you need are a small sudo snippet and a profile.d script.

My sudo snippet looks like:

Defaults!/usr/bin/ssh-add env_keep += "SSH_AUTH_SOCK"
%support ALL=(root)  NOPASSWD: /usr/bin/ssh-add /var/local/support-ssh/id_rsa

Everybody in group support can run ssh-add as root.

The profile.d goes in /etc/profile.d/support.sh and looks like:

if [ -n "$(groups | grep -E "(^| )support( |$)")" ]; then
    export SSH_AUTH_ENV="$HOME/.ssh/agent-env"
    if [ -f "$SSH_AUTH_ENV" ]; then
        . "$SSH_AUTH_ENV"
    fi
    ssh-add -l >/dev/null 2>&1
    if [ $? = 2 ]; then
        mkdir -p "$HOME/.ssh"
        rm -f "$SSH_AUTH_ENV"
        ssh-agent > "$SSH_AUTH_ENV"
        . "$SSH_AUTH_ENV"
    fi
    sudo ssh-add /var/local/support-ssh/id_rsa
fi

The key is unavailable for the user in question because ssh-add is sgid and so runs with group ssh and the process is only debuggable for root. The only thing missing is there's no way to have the agent prompt to use a key and I would like it to die or at least unload keys when the last session for a user is closed, but that doesn't seem trivial to do.

KMix is KDE’s forgotten redheaded stepchild.

Humble beginnings

It is old, has been maintained my Christian Esken since early 2001, and has grown organically. Through no fault of Christian’s or anyone else’s, it is buggy, messy, and nobody else wants to help fix it, or at least has the constitution to do so.

Us redheads need to stick together.

At last year’s Randa, I started working on refactoring and rebuilding KMix from the ground up. Why should adjusting the volume in your desktop be so frustrating? It shouldn’t, thats why.

KMix is back with a new edition~

That is the prototype for the new KMix. The UI design has been shamelessly stolen from pavucontrol:

pavucontrol

As you can see, there is much work left to do. However, I feel that it is ready for others to hack on and test. This restructuring of KMix contains a *lot* of the original KMix code that has been contributed to over the years by over 142(!) people. The biggest committers have been Christian Esken (601), Colin Guthrie (130), and Laurent Montel (123). Mad props to those people and everyone else involved for their work and dedication.

When I started out on this endeavor, I aimed to fix a number of issues I had encountered before:

• Sometimes at startup, your volume would be all kinds of broken since the KDED module, Tray application, and a session startup script would all sometimes try to restore the volumes and sometimes it just wouldn’t.
• The GUI was completely rebuilt with every device change. Back in the early days of KMix, hot-pluggable audio devices wasn’t a major use case so it wasn’t included in the design.
• Since there is no one process that controls all the volume everywhere, that means lots and lots of backend libraries and their respective data structures loaded into many processes, such as plasma, the tray application, the KDED module, the tiny script at startup that restores your volumes.
• All these different points of control had to make sure they didn’t step on each others toes all the time.
• To control the volume over dbus, the tray application had to be running. There was no dbus autolaunch, and why would you want a GUI application to automatically launch and do weird things when you just want a small script to turn down your volume between the hours of 3 am and 6 am?
• pavucontrol had neat VU meters
• I’m just kinda okay at UI and UX design for end-users.

In response, I’ve come up with the following design:

• There exists a unique daemon on the session bus at org.kde.kmixd, which is managed via dbus autolaunch
• This unique daemon supports multiple backends being loaded at once, such as ALSA and PulseAudio (note: currently requires commenting out an if statement)
• Clients don’t need to worry about learning the oddities of ALSA or PulseAudio control since all interaction happens over the bus in a backend-agnostic manner. You are given a set of controls (such as your internal audio output, or the output of chrome’s flash plugin), a set of control groups (Output Devices, Recording Streams, etc), and a single interface to adjusting the master volume (which is whatever the user configures).
• Thanks to dbus autolaunch, you don’t have a daemon running in the background until the very instant during your session that volume control is needed. This helps to decrease startup time and overall resource usage.
• The UI is decoupled from the volume control itself, so others can easily create much better looking UIs in QML, Plasma, QtWidgets, or whatever other language that supports DBus.

All of this is in my kmix-improvements branch on KDE git’s kmix repository. I’ll reiterate: this is mostly just a pre-alpha. Don’t expect everything to work, though I have been using it on my desktops without significant issues. Feedback and contributions are welcome.