The Mixed Reality OpenXR Plugin package is an extension of Unity's OpenXR Plugin and supports a suite of features for HoloLens 2 and Windows Mixed Reality headsets. Before continuing, make sure your Unity project is configured for OpenXR.
OpenXR is an open, royalty-free standard developed by Khronos that aims to simplify AR/VR development by allowing developers to target a wide range of AR/VR devices. Use this plug-in to enable OpenXR in XR Plug-in Management. Version information Verified for Unity. Package version 1.0.3 is verified for Unity Editor version 2020.3. Unity OpenXR provides support for the Hololens 2 Hand interaction profile. This layout inherits from so bindings that use XR Controller and are. OpenVR Unity XR Plugin The purpose of this package is to provide OpenVR rendering to Unity XR. This package provides the necessary sdk libraries for users to build Applications that work with the OpenVR runtime. The OpenVR XR Plugin gives you access to rendering on all. The Unity OpenXR package provides a set of controller layouts for various devices that you can bind to your actions when using the Unity Input System. For more information, see the Interaction profile features section. To use OpenXR Input, you must select the correct interaction profiles.
What's supported
The following features are currently supported:
- Supports UWP applications for HoloLens 2, and optimize for HoloLens 2 application model.
- Supports Win32 VR applications for Windows Mixed Reality headset with latest controller profiles and holographic app remoting.
- World scale tracking using Anchors and Unbounded space.
- Anchor storage API to persist anchors to HoloLens 2 local storage.
- Motion controller and hand interactions, including the new HP Reverb G2 controller.
- Articulated hand tracking using 26 joints and joint radius inputs.
- Eye gaze interaction on HoloLens 2.
- Locating photo/video (PV) camera on HoloLens 2.
- Mixed Reality Capture using 3rd eye rendering through PV camera.
- Supports 'Play' to HoloLens 2 with the Holographic Remoting app, allowing developers to debug scripts without building and deploying to the device.
- Compatible with MRTK Unity 2.5.3 and newer through MRTK OpenXR provider support.
- Compatible with Unity ARFoundation 4.0 or later.
- (Added in 0.1.3) Supports desktop app Holographic Remoting from a built and deployed Windows Standalone app.
- (Added in 0.1.4) Supports QR code tracking on HoloLens2 through SpatialGraphNode
- (Added in 0.2.0) Supports Anchor in Holographic Remoting
- (Added in 0.2.0) Supports both hand joints and hand mesh tracking
- (Added in 0.2.0) Supports ARPlaneSubsystems for plane detection and place hologram using ARRaycastManager.
- (0.9.0) Supports XRMeshSubsystem and ARMeshManager for spatial mapping.
- (Added in 0.9.0) Supports the Azure Spatial Anchors SDK for Windows plugin. For more information, see the Mixed Reality + OpenXR Azure Spatial Anchors sample project on GitHub.
- (Added in 0.9.1) Supports desktop app Holographic Remoting from a built and deployed Windows UWP app.
Holographic Remoting setup
First, you need to install the Holographic Remoting Player app from the Microsoft Store on your HoloLens 2
Run the Holographic Remoting Player app on HoloLens 2 and you'll see the version number and IP address to connect to
- You'll need v2.4 or later to work with the OpenXR plugin
Holographic Remoting in Unity Editor play mode
Building a UWP Unity project in Visual Studio project and then packaging and deploying it to a HoloLens 2 device can take some time. One solution is to enable the Holographic Editor Remoting feature, which lets you debug your C# script using “Play” mode directly to a HoloLens 2 device over your network. This scenario avoids the overhead of building and deploying a UWP package to remote device.
Follow the steps in Holographic Remoting setup
Open Edit -> Project Settings, navigate to XR plug-in Management, and check the Windows Mixed Reality feature set box:
Expand the Features section under OpenXR and select Show All
Check the Holographic Editor Remoting checkbox and input the IP address you get from the Holographic Remoting app:
Now you can click the “Play” button to play your Unity app into the Holographic Remoting app on your HoloLens. You can also attach Visual Studio to Unity to debug C# scripts in the play mode.
Note
As of version 0.1.0, the Holographic Remoting runtime doesn’t support Anchors, and ARAnchorManager functionalities will not work through remoting. This feature is coming in future releases.
Motion controller and hand interactions
To learn the basics about mixed reality interactions in Unity, visit the Unity Manual for Unity XR Input. This Unity documentation covers the mappings from controller-specific inputs to more generalizable InputFeatureUsages, how available XR inputs can be identified and categorized, how to read data from these inputs, and more.
The Mixed Reality OpenXR Plugin provides additional input interaction profiles, mapped to standard InputFeatureUsages as detailed below:
| InputFeatureUsage | HP Reverb G2 Controller (OpenXR) | HoloLens Hand (OpenXR) |
|---|---|---|
| primary2DAxis | Joystick | |
| primary2DAxisClick | Joystick - Click | |
| trigger | Trigger | |
| grip | Grip | Air tap or squeeze |
| primaryButton | [X/A] - Press | Air tap |
| secondaryButton | [Y/B] - Press | |
| gripButton | Grip - Press | |
| triggerButton | Trigger - Press | |
| menuButton | Menu |
Aim and Grip Poses
You have access to two sets of poses through OpenXR input interactions:
- The grip poses for rendering objects in the hand
- The aim poses for pointing into the world.
More information on this design and the differences between the two poses can be found in the OpenXR Specification - Input Subpaths.
Poses supplied by the InputFeatureUsages DevicePosition, DeviceRotation, DeviceVelocity, and DeviceAngularVelocity all represent the OpenXR grip pose. InputFeatureUsages related to grip poses are defined in Unity’s CommonUsages.
Poses supplied by the InputFeatureUsages PointerPosition, PointerRotation, PointerVelocity, and PointerAngularVelocity all represent the OpenXR aim pose. These InputFeatureUsages aren't defined in any included C# files, so you'll need to define your own InputFeatureUsages as follows:
Haptics
For information on using haptics in Unity’s XR Input system, documentation can be found at the Unity Manual for Unity XR Input - Haptics.
QR codes
HoloLens 2 can detect QR codes in the environment around the headset, establishing a coordinate system at each code's real-world location. You can find more details in our QR code tracking documentation. When using the OpenXR plugin, grab the SpatialGraphNodeId from the QR API and use the Microsoft.MixedReality.OpenXR.SpatialGraphNode API to locate the QR code.
For reference, we have a QR tracking sample project on GitHub with more a detailed usage explanation for the SpatialGraphNode API.
What's coming soon
The following issues and missing features are known with Mixed Reality OpenXR plugin version 0.9.2. We're working on these and will release fixes and new features in upcoming releases.
- ARM64 is the only supported platform for HoloLens 2 apps. The ARM platform is coming in a future release.
Troubleshooting
When you suspend and resume a Unity app on HoloLens 2, the app can't correctly resume, which leads to 4 spinning dots in the HoloLens view.
- Set Depth submission Mode to None in the OpenXR project settings as a workaround
Join GamesBeat Summit 2021 this April 28-29. Register for a free or VIP pass today.
When the Khronos Group released OpenXR — a royalty-free standard designed to make cross-platform VR and AR app development easier, supporting multiple platforms — it was backed by many of the mixed reality industry’s biggest names, including Epic Games, Microsoft, Oculus, and Valve, to name just a few. Today, Unity confirmed that its eponymous 3D engine will start supporting some OpenXR platforms by the end of 2020, with a focus on “providing the best developer experience on Unity supported platforms.”
Though Unity publicly backed OpenXR in March 2019 and has actively contributed to the standard, it hadn’t committed to a timeline for actually bringing OpenXR support to the Unity engine, which is used by untold numbers of 3D apps and games, as well as automotive, film, and engineering firms. Mixed reality applications such as Childish Gambino’s interactive music experience Pharos AR have been built using Unity, but until now, might not have easily reached every device a developer would hope to target. Leading PC VR headsets, HoloLens 2, and both Oculus platforms all support OpenXR, which is expected to be the backbone for most future AR and VR devices as well.
From a big-picture perspective, Unity’s promise of OpenXR support means that a significant number of developers will be able to bring previously created mixed reality content to whatever platforms prove popular, and create new AR/VR apps with Unity that work across virtually any XR device. That said, implementation won’t necessarily be exactly the same from platform to platform, and Unity is warning that due to the “unbounded combinations” of possible hardware and software, it can’t test or guarantee optimal performance for every platform out there.
Unity is currently working to support partners’ OpenXR runtimes and expects to offer initial previews of the Unity engine’s OpenXR support on “some” partner platforms before year’s end. The next stage will be early in 2021, when Unity will offer preliminary OpenXR 1.0 specification-compliant support for non-partner OpenXR runtimes and devices, with plans to improve them based on reported issues, sharing test results and spec changes with the Khronos Group.
GamesBeat
GamesBeat's creed when covering the game industry is 'where passion meets business.' What does this mean? We want to tell you how the news matters to you -- not just as a decision-maker at a game studio, but also as a fan of games. Whether you read our articles, listen to our podcasts, or watch our videos, GamesBeat will help you learn about the industry and enjoy engaging with it.Unity Openxr Document
How will you do that? Membership includes access to:Unity Openxr Oculus
- Newsletters, such as DeanBeat
- The wonderful, educational, and fun speakers at our events
- Networking opportunities
- Special members-only interviews, chats, and 'open office' events with GamesBeat staff
- Chatting with community members, GamesBeat staff, and other guests in our Discord
- And maybe even a fun prize or two
- Introductions to like-minded parties