We're hiring!
*

An xrdesktop summer of code

Lubosz Sarnecki avatar

Lubosz Sarnecki
September 10, 2021

Share this post:

This summer, Christoph Haag and I had the pleasure of taking part in Google Summer of Code (GSoC) as mentors for xrdesktop, the Open Source project bringing the Linux desktop to VR on SteamVR & Monado. Our students, Remco and Manas, were both able to finish their projects and submit merge requests. Kudos!

As our tools of choice, C/GObject and Vulkan were used in both projects, maintaining our goal of keeping xrdesktop low level and thus providing a performant XR experience.

Contributions like these support our vision with projects like xrdesktop and Monado to provide a fully open source XR stack that enables complete control and independence for end users and product builders alike.

Virtual Reality Keyboard

In XR, users want to input text with methods like controllers or hand tracking without having to walk to their desk and use a physical keyboard. The OpenXR API currently does not expose any API for text input, and it is in the applications' responsibility to implement a text input method.

Remco implemented such a virtual keyboard for xrdesktop with support for emojis and 56 languages. Supporting non-english languages and emojis required adding Unicode support to our desktop input synthesis library libinputsynth. The advanced glyph support was achieved by using the Pango font rendering library.


He specified a JSON format for describing keyboard layouts and an importer script that uses the Unicode CLDR Project definitions to achieve internalization. Our current design focus was a simple keyboard with mode switches as seen in the GNOME Shell onscreen keyboard or on mobile phones.

With our internal json format arbitary keyboards can be specified, so one could also replicate a 100% desktop keyboard with numpad or even keyboards that are not feasible in a physical form, like this Japanese keyboard with multiple hundred characters:

Source: https://japan.googleblog.com/2010/04/google.html.


With Remco's work xrdesktop will be able to provide a virtual keyboard internally to xrdesktop, independent of the XR runtime it runs on. We will also be able to expose this keyboard as a stand-alone OpenXR application so it will be usable outside of xrdesktop.

Future work on this project includes implementing input methods for Chinese and other asian languages as well as prediction with suggestions as known from mobile phones and other on-screen keyboards.

More details can be found on Remco's blog post.

GLTF Loading

Manas implemented loading and rendering of GLTF models, which included improving our renderer and adding rendering techniques like normal mapping. This work enables xrdesktop to use assets like VR controller models and 3D environments.

The GLTF loader is based on gthree, the GObject port of three.js.

We are able to parse and render GLTF 2.0 .gtlf and .glb files. With which we will be able to use the controller assets from the W3C Immersive Web WebXR Asset repository in an upcoming release.

Blender's Suzanne model loaded from GLTF.


Future work will make the loader and renderer compatible with the full glTF Sample Model repository from Khronos and provide an asset pipeline from Blender to xrdesktop.

If you are interested in detail's about Manas's project, you can find a summary in his blog post.

Upcoming 0.16 release

These features will get into our next xrdesktop release v0.16, which will be tagged soon. If you are brave enough you can also try our experimental next branches on our freedesktop.org GitLab, following the installation from source instructions from our Wiki.

Don't hesitate to submit issues on our bug tracker or join the discussion on our Discord. Follow @xrdesktop on Twitter.

Comments (5)

    1. Lubosz:
      Sep 14, 2021 at 03:28 PM

      Layouts like this can be defined without problem in our JSON format:
      https://gitlab.freedesktop.org/xrdesktop/xrdesktop/-/blob/next/res/keyboard-layouts/small.json

      Currently we are only providing the common layouts from the Unicode CLDR Project, but it should be easy to add new ones.

      Reply to this comment

      Reply to this comment

    1. Lubosz:
      Sep 14, 2021 at 03:28 PM

      Thanks for finding a better source! We'll update the post.

      Reply to this comment

      Reply to this comment


Add a Comment






Allowed tags: <b><i><br>Add a new comment:


Search the newsroom

Latest Blog Posts

Venus on QEMU: Enabling the new virtual Vulkan driver

26/11/2021

A step-by-step guide on how to enable 3D acceleration of Vulkan applications in QEMU through the new Venus experimental Vulkan driver for…

Run your own CI pipeline with GStreamer's new monorepo

26/10/2021

Maintaining a non-trivial set of GStreamer patches can be tricky. Thanks to the recent move to a single, unified git repo, you can now easily…

Improving test coverage for cameras in KernelCI

08/10/2021

Earlier this year, I joined Collabora as an intern to work on improving testing in libcamera and automating it through KernelCI. Having…

A tale of two toolchains and glibc

30/09/2021

With the LLVM toolchain seeing increasing development and adoption alongside the older, more established GNU toolchain, projects needing…

An xrdesktop summer of code

10/09/2021

This summer, Christoph Haag and I had the pleasure of taking part in Google Summer of Code as mentors for xrdesktop, the Open Source project…

Adding VP9 and MPEG2 stateless support in v4l2codecs for GStreamer

23/06/2021

Earlier this year, from January to April 2021, I worked on adding support for stateless decoders for GStreamer as part of a multimedia internship…

Open Since 2005 logo

We use cookies on this website to ensure that you get the best experience. By continuing to use this website you are consenting to the use of these cookies. To find out more please follow this link.

Collabora Ltd © 2005-2021. All rights reserved. Privacy Notice. Sitemap.

Collabora Limited is registered in England and Wales. Company Registration number: 5513718. Registered office: The Platinum Building, St John's Innovation Park, Cambridge, CB4 0DS, United Kingdom. VAT number: 874 1630 19.