Moses Turner
February 24, 2023
Share this post:
Reading time:
In our previous blog post, I teased our new optical hand tracking pipeline, codenamed Mercury. Work on this tracking method began around January 2022.
Now, after a little over a year of development, it's finally ready for the public to use!
Mercury currently works on Valve Index, Windows Mixed Reality headsets (Reverb G2 v2 shown below), Oculus Rift S (early support), and Luxonis cameras (video shows a OAK-D Pro W Dev mounted on a North Star). It estimates hand pose accurately, supports limited hand-over-hand interactions, tracks fast hand movements, and is useable for drawing, typing and UI interaction.
Usage documentation is here if you want to run it in Monado. Expect in the coming few weeks some improvements to hand tracked-state stability, reductions to background jitter, a SteamVR driver so you can use it on Windows, and a much more detailed blog post showcasing the work we did to get here!
14/03/2023
We're proud to announce that Monado, the free and open source XR platform, has been accepted as a mentoring organization for XROS, the XR…
13/03/2023
As a recent new member of STMicroelectronics' Partner Program, we're excited to be showcasing the STM32MP1 at Embedded World this week,…
09/03/2023
Nestled in the historic city of Nuremberg, the annual Embedded World conference will be taking place from March 14 to 16. Collabora will…
Whether writing a line of code or shaping a longer-term strategic software development plan, we'll help you navigate the ever-evolving world of Open Source.
한국어 버전의 Collabora.com 보기
Acesse Collabora.com em Português
Comments (4)
Eli:
Mar 01, 2023 at 12:56 AM
How do I get it on windows 10?
Reply to this comment
Reply to this comment
Roman:
Mar 01, 2023 at 05:33 AM
This is very exciting! I hope SteamVR driver is coming that would let it be used in the similar way as UltraLeap does it, only it has so much better potential, as no extra hardware is needed, and HMDs cameras are wide-angle and perfectly aligned. Whe SteamVR driver is ready, would it work with existing WMR OpenXR runtime, or would it replace it? Or would it only work with StemVR OpenXR runtime (which is inferior for Reverb G2 and other native WMR HMDs)?
Reply to this comment
Reply to this comment
RalfMarco:
Mar 01, 2023 at 10:17 AM
This is fantastic!
A openXR toolkit driver would be amazing. For MSFS Community and VR Pilots. This Handtracking in an Airplane - Game changer! Many fly with the reverb G2.
Reply to this comment
Reply to this comment
hyperdrive:
Mar 04, 2023 at 11:03 PM
will it work while you are holding ANY type of controllers, so you can have access to thumb sticks and buttons while still having full finger tracking? otherwise, it's kind of useless because I shouldn't have to give up certain controller necessities for a small but wanted/fun gimic that most games don't even support. I say this because the quest 2 has this issue, and time and time again I see people on the quest 2 try to do simple things like walking and jumping while using the full finger tracking and I see that as a big problem and should have never happened in the first place.
Reply to this comment
Reply to this comment
Add a Comment