October 26, 2021
Recently, the GStreamer project merged all its git repositories into a single, unified repository, often called
monorepo. You can read more about this change here.
One benefit is it greatly simplifies maintaining custom, project specific, GStreamer patches. Previously, projects that needed to develop, or backport, some patches had to go through multiple steps to complete the task. Projects had to fork various git repositories (e.g. gst-plugins-good, gst-plugins-bad, etc), each repository would then have a new branch with the extra commits, and often, gst-build was used to pull all of these repositories together (and gst-build itself had to be patched beforehand to download forked repositories). Thankfully, all that will be a thing of the past.
When maintaining a non-trivial set of patches, it's always a good idea to also run a continuous integration (CI) pipeline. Prior to moving to a single repository, GStreamer's public CI had a complicated manifest mechanism to be able to pull branches from different repositories, and test them together. Replicating that system with a private GitLab instance was close to impossible.
With GStreamer monorepo, all those patches can now be maintained in a single git repository, in a single branch. For example, if you are forking GStreamer's
main branch into a private GitLab instance, the CI will now work almost out of the box! Your team can create Merge Requests in your private GitLab instance, and develop features while relying on your own CI. Of course, once the feature is ready, it often makes sense to upstream. A single Merge Request in the public repository and you're good to go.
1. Clone the GStreamer repository:
git clone https://gitlab.freedesktop.org/gstreamer/gstreamer.git
2. Push the main branch to your private GitLab instance:
git remote add private https://gitlab.example.com/my-project/gstreamer.git git push private main
3. Setup your new GStreamer repository:
Visibility, project features, permissionsmake sure to enable
4. Register your GitLab runners:
Runnersyou'll find the
registration tokento register your gitlab-runner.
That's it! You can now run the same CI pipeline as on the public GitLab instance. Every member of your team can create their fork of that repositry into their own account and create internal Merge Requests.
Also, you'll notice that your CI will rebuild the needed Linux and Windows docker images thanks to Freedesktop's ci-template.
Not everything is that easy however...
GitLab will try to trigger Cerbero's CI into the public repository, which will fail because your access token from your private GitLab instance won't be accepted by another GitLab instance.
You can either edit
.gitlab-ci.yml and delete the
cerbero trigger job, or you can also fork cerbero into our private GitLab instance. Note that if you don't fork cerbero under
gstreamer/cerbero, you will have to edit
ci/gitlab/trigger_cerbero_pipeline.py and change to
CERBERO_PROJECT = 'my-project/cerbero'.
You can delete from
.gitlab-ci.yml jobs that require Windows, or you can setup your own runner.
This unification of Git repositories will greatly simply the workflow for all contributors and distributors. We are really looking forward to seeing it in action in the next major GStreamer release. With the feature freeze having begun on October 18th, we should hopefully see a final release of GStreamer 1.20 before the end of the year. As usual, if you need any assistance with GStreamer, please don't hesitate to contact us, we're here to help!
A step-by-step guide on how to enable 3D acceleration of Vulkan applications in QEMU through the new Venus experimental Vulkan driver for…
Maintaining a non-trivial set of GStreamer patches can be tricky. Thanks to the recent move to a single, unified git repo, you can now easily…
Earlier this year, I joined Collabora as an intern to work on improving testing in libcamera and automating it through KernelCI. Having…
With the LLVM toolchain seeing increasing development and adoption alongside the older, more established GNU toolchain, projects needing…
This summer, Christoph Haag and I had the pleasure of taking part in Google Summer of Code as mentors for xrdesktop, the Open Source project…
Earlier this year, from January to April 2021, I worked on adding support for stateless decoders for GStreamer as part of a multimedia internship…