We're hiring!

An xrdesktop summer of code

Lubosz Sarnecki avatar

Lubosz Sarnecki
September 10, 2021

Share this post:

This summer, Christoph Haag and I had the pleasure of taking part in Google Summer of Code (GSoC) as mentors for xrdesktop, the Open Source project bringing the Linux desktop to VR on SteamVR & Monado. Our students, Remco and Manas, were both able to finish their projects and submit merge requests. Kudos!

As our tools of choice, C/GObject and Vulkan were used in both projects, maintaining our goal of keeping xrdesktop low level and thus providing a performant XR experience.

Contributions like these support our vision with projects like xrdesktop and Monado to provide a fully open source XR stack that enables complete control and independence for end users and product builders alike.

Virtual Reality Keyboard

In XR, users want to input text with methods like controllers or hand tracking without having to walk to their desk and use a physical keyboard. The OpenXR API currently does not expose any API for text input, and it is in the applications' responsibility to implement a text input method.

Remco implemented such a virtual keyboard for xrdesktop with support for emojis and 56 languages. Supporting non-english languages and emojis required adding Unicode support to our desktop input synthesis library libinputsynth. The advanced glyph support was achieved by using the Pango font rendering library.

He specified a JSON format for describing keyboard layouts and an importer script that uses the Unicode CLDR Project definitions to achieve internalization. Our current design focus was a simple keyboard with mode switches as seen in the GNOME Shell onscreen keyboard or on mobile phones.

With our internal json format arbitary keyboards can be specified, so one could also replicate a 100% desktop keyboard with numpad or even keyboards that are not feasible in a physical form, like this Japanese keyboard with multiple hundred characters:

Source: https://japan.googleblog.com/2010/04/google.html.

With Remco's work xrdesktop will be able to provide a virtual keyboard internally to xrdesktop, independent of the XR runtime it runs on. We will also be able to expose this keyboard as a stand-alone OpenXR application so it will be usable outside of xrdesktop.

Future work on this project includes implementing input methods for Chinese and other asian languages as well as prediction with suggestions as known from mobile phones and other on-screen keyboards.

More details can be found on Remco's blog post.

GLTF Loading

Manas implemented loading and rendering of GLTF models, which included improving our renderer and adding rendering techniques like normal mapping. This work enables xrdesktop to use assets like VR controller models and 3D environments.

The GLTF loader is based on gthree, the GObject port of three.js.

We are able to parse and render GLTF 2.0 .gtlf and .glb files. With which we will be able to use the controller assets from the W3C Immersive Web WebXR Asset repository in an upcoming release.

Blender's Suzanne model loaded from GLTF.

Future work will make the loader and renderer compatible with the full glTF Sample Model repository from Khronos and provide an asset pipeline from Blender to xrdesktop.

If you are interested in detail's about Manas's project, you can find a summary in his blog post.

Upcoming 0.16 release

These features will get into our next xrdesktop release v0.16, which will be tagged soon. If you are brave enough you can also try our experimental next branches on our freedesktop.org GitLab, following the installation from source instructions from our Wiki.

Don't hesitate to submit issues on our bug tracker or join the discussion on our Discord. Follow @xrdesktop on Twitter.

Comments (5)

    1. Lubosz:
      Sep 14, 2021 at 03:28 PM

      Layouts like this can be defined without problem in our JSON format:

      Currently we are only providing the common layouts from the Unicode CLDR Project, but it should be easy to add new ones.

      Reply to this comment

      Reply to this comment

    1. Lubosz:
      Sep 14, 2021 at 03:28 PM

      Thanks for finding a better source! We'll update the post.

      Reply to this comment

      Reply to this comment

Add a Comment

Allowed tags: <b><i><br>Add a new comment:

Search the newsroom

Latest Blog Posts

Visual-inertial tracking for Monado


Monado now has initial support for 6DoF ("inside-out") tracking for devices with cameras and an IMU! Three free and open source SLAM/VIO…

Spotlight on Meson's full-featured developer environment


When developing an application or a library, it is very common to want to run it without installing it, or to install it into a custom prefix…

How to write a Vulkan driver in 2022


An incredible amount has changed in Mesa and in the Vulkan ecosystems since we wrote the first Vulkan driver in Mesa for Intel hardware…

Improving the reliability of file system monitoring tools


Every file system used in production has tools to try to recover from system crashes. To provide a better infrastructure for those tools,…

PipeWire: A year in review & a look ahead


The PipeWire project made major strides over the past few years, bringing shiny new features, and paving the way for new possibilities in…

Landing a new syscall, part 1: What is futex?


Over the past 18 months, we have been on a roller-coaster ride developing futex2, a new set of system calls. As part of this effort, the…

Open Since 2005 logo

We use cookies on this website to ensure that you get the best experience. By continuing to use this website you are consenting to the use of these cookies. To find out more please follow this link.

Collabora Ltd © 2005-2022. All rights reserved. Privacy Notice. Sitemap.