We're hiring!

An xrdesktop summer of code

Lubosz Sarnecki avatar

Lubosz Sarnecki
September 10, 2021

Share this post:

Reading time:

This summer, Christoph Haag and I had the pleasure of taking part in Google Summer of Code (GSoC) as mentors for xrdesktop, the Open Source project bringing the Linux desktop to VR on SteamVR & Monado. Our students, Remco and Manas, were both able to finish their projects and submit merge requests. Kudos!

As our tools of choice, C/GObject and Vulkan were used in both projects, maintaining our goal of keeping xrdesktop low level and thus providing a performant XR experience.

Contributions like these support our vision with projects like xrdesktop and Monado to provide a fully open source XR stack that enables complete control and independence for end users and product builders alike.

Virtual Reality Keyboard

In XR, users want to input text with methods like controllers or hand tracking without having to walk to their desk and use a physical keyboard. The OpenXR API currently does not expose any API for text input, and it is in the applications' responsibility to implement a text input method.

Remco implemented such a virtual keyboard for xrdesktop with support for emojis and 56 languages. Supporting non-english languages and emojis required adding Unicode support to our desktop input synthesis library libinputsynth. The advanced glyph support was achieved by using the Pango font rendering library.

He specified a JSON format for describing keyboard layouts and an importer script that uses the Unicode CLDR Project definitions to achieve internalization. Our current design focus was a simple keyboard with mode switches as seen in the GNOME Shell onscreen keyboard or on mobile phones.

With our internal json format arbitary keyboards can be specified, so one could also replicate a 100% desktop keyboard with numpad or even keyboards that are not feasible in a physical form, like this Japanese keyboard with multiple hundred characters:

Source: https://japan.googleblog.com/2010/04/google.html.

With Remco's work xrdesktop will be able to provide a virtual keyboard internally to xrdesktop, independent of the XR runtime it runs on. We will also be able to expose this keyboard as a stand-alone OpenXR application so it will be usable outside of xrdesktop.

Future work on this project includes implementing input methods for Chinese and other asian languages as well as prediction with suggestions as known from mobile phones and other on-screen keyboards.

More details can be found on Remco's blog post.

GLTF Loading

Manas implemented loading and rendering of GLTF models, which included improving our renderer and adding rendering techniques like normal mapping. This work enables xrdesktop to use assets like VR controller models and 3D environments.

The GLTF loader is based on gthree, the GObject port of three.js.

We are able to parse and render GLTF 2.0 .gtlf and .glb files. With which we will be able to use the controller assets from the W3C Immersive Web WebXR Asset repository in an upcoming release.

Blender's Suzanne model loaded from GLTF.

Future work will make the loader and renderer compatible with the full glTF Sample Model repository from Khronos and provide an asset pipeline from Blender to xrdesktop.

If you are interested in detail's about Manas's project, you can find a summary in his blog post.

Upcoming 0.16 release

These features will get into our next xrdesktop release v0.16, which will be tagged soon. If you are brave enough you can also try our experimental next branches on our freedesktop.org GitLab, following the installation from source instructions from our Wiki.

Don't hesitate to submit issues on our bug tracker or join the discussion on our Discord. Follow @xrdesktop on Twitter.

Comments (5)

    1. Lubosz:
      Sep 14, 2021 at 03:28 PM

      Layouts like this can be defined without problem in our JSON format:

      Currently we are only providing the common layouts from the Unicode CLDR Project, but it should be easy to add new ones.

      Reply to this comment

      Reply to this comment

    1. Lubosz:
      Sep 14, 2021 at 03:28 PM

      Thanks for finding a better source! We'll update the post.

      Reply to this comment

      Reply to this comment

Add a Comment

Allowed tags: <b><i><br>Add a new comment:

Search the newsroom

Latest Blog Posts

MLfix to quickly fix datasets


Contrary to traditional software development, data is more important than code in machine learning. Building a high-performing model requires…

USB 3.0 preliminary support in U-boot for Radxa Rock-5B


A while back I presented USB 2.0 host support that was added to U-boot for the Radxa Rock-5B RK3588 Rockchip board. This time, USB 3.0 was…

Radxa Rock-5B PCIe and RTL8125B networking in U-boot


Work continues on the Radxa ROCK5B RK388, as PCIe and RTL8125B networking support in U-boot have now been added. Publishing code as Open…

Introducing Multiview for NVK


NVK, an open-source Vulkan driver for NVIDIA hardware that is part of Mesa, now supports the Vulkan extension VK_KHR_multiview.

Adding bootloader support for USB 2.0 Host for Radxa ROCK 5B RK3588


The beauty of Open Source is that we can reuse code written by many other people, keep their authorship, and credit them for their work,…

Meson & VSCode: Develop your project in a modern IDE


Want to develop your Meson project in a modern IDE? Make sure to install Meson VSCode extension which is now fully functional with the recent…

Open Since 2005 logo

We use cookies on this website to ensure that you get the best experience. By continuing to use this website you are consenting to the use of these cookies. To find out more please follow this link.

Collabora Ltd © 2005-2023. All rights reserved. Privacy Notice. Sitemap.