Mixed Reality with Wave Engine

In this tutorial, we will cover the creation of your first Mixed reality app using Wave EngineWe will find out why using this engine is a good choice for creating your holographic experience.

In this app we will load a plane in front of the viewer.

Prerequisites

For simulating your app you will need a Windows 10 PC configured with the following tools:

Project assets

  • Download the files required by the project here.

1 – Create a New Project

To create your Mixed reality app with Wave Engine you have to create a new project.

  1. Start Wave Engine Visual Editor
  2. Select File > New Project… (A dialog will appear)
  3. Enter a project name (e.g. “MixedRealityGame”)
  4. Enter the location to save your project.
  5. Select OK.

A new project will be created.

2 – Setup your app for Mixed reality

Now we will setup the project for Mixed reality creating a UWP profile project.

  1. Select Edit > Project Properties…
  2. Click on the ‘+’ button. A profile dialog will appear
  3. Select UWP in Platform.
  4. Enter a profile name (e.g. “MixedRealityProfile“)
  5. Select MixedReality in LauncherType.

This will create a new Wave Engine profile and a new Visual Studio solution.

3 – Setup the MixedReality Camera

To create the camera for Mixed Reality we just have to create an empty entity with the VRCameraRig and the MixedRealityProvider components.

First of all, we will clear the scene removing the default entities of Wave Engine template.

  1. In the Entities Hierarchy panel, select all the existing entities.
  2. Remove them pressing Ctrl+Delete.
  3. Select Create > Empty Entity 3D 
  4. In Entity Hierarchy select ’empty’ entity.
  5. In Entity Details set the name to camera.

Now we will add the VRCameraRig component. This will create a hierarchy of entities under the root entity.

  1. In the top-left area of Entity Details, select the ‘+’ button. The Add Component dialog will appear.
  2. Enter VRCameraRig in the filter textbox.
  3. Select VRCameraRig in the component list.

Now we will add the MixedRealityProvider component exactly in the same way.

  1. In the top-left area of Entity Details, select the ‘+’ button. The Add Component dialog will appear.
  2. Enter MixedRealityProvider in the filter textbox and select it from the the component list.

4 – Create a plane Hologram

Holograms in WaveEngine are just normal entities, so creating them is really easy: All you have to do is place your entities in your scene. One meter in the real world is approximately one unit in WaveEngine world.

In this scene, we will create a plane in front of the user. First, we will import the assets.

  1. In the Asset Details panel, select Asset folder.
  2. Open the folder of the project asset previously downloaded.
  3. Drag the files (a380.fbx, AirbusLightingMap.png and ColorPalette.png) to the Asset Details.
  4. Wait until the assets appear in the Asset Details.

Now we will create a new Entity.

  1. Drag the a380 model asset to the Scene area.
  2. Select the ‘model’ entity in the Entity Hierarchy panel.
  3. In the Entity Details, change its name (e.g. ‘plane’).
  4. Find the Transform3D component, and change LocalPosition to (X: 0, Y: 0, Z: -2). This will place the new entity 2 meters in front of the user’s starting position.

To apply a new material to the new plane entity:

  1. Select Assets > Create Material. A new Material Editor dialog will appear.
  2. Select that DualMaterial in the Shader combo.
  3. Select Diffuse1Path. A Floating panel will appear with the textures of your project.
  4. Select ‘ColorPalette.png’ texture.
  5. Select Diffuse2Path. 
  6. Select  AirbusLightingMap.png‘ texture.
  7. Uncheck LightingEnabled checkbox.
  8. Select Create. The Material Editor dialog will close and a new ‘MyMaterial’ material asset will appear in AssetDetails.
  9. Rename ‘MyMaterial’ (e.g. ‘PlaneMat’).
  10. Drag ‘PlaneMat’ to the ‘mainPlane’ entity in the scene.

Now you can see the final plane hologram fully textured.

5 – Add placing behavior

In Wave Engine we can create our own entity components to extend and customize its appearance or behavior. For this app, we will create a behavior that will place the entity in front of the camera when the user is making the select gesture with his hand.

We will add these behaviors in the Visual Studio solution. First of all we have to add the WaveEngine.MixedReality reference to the Windows Solution.

  1. Select File > Open C# Solution. Visual Studio will be launched with the default Windows Solution.
  2. Select MixedRealityGame project.
  3. In Visual Studio, select Project > Manage Nuget Packages…
  4. Select Browse.
  5. Search for ‘WaveEngine.MixedReality’
  6. Select WaveEngine.MixedReality package.
  7. Select Install.
  8. Select OK.

Now we creates the new behavior

  1. In the Solution Explorer, select MixedRealityGameSource shared project.
  2. Select Project > New Item…
  3. Select Visual Studio C# in the left panel and Class in the item list.
  4. Name it as PlaceBehavior.cs.
  5. Select Add.

We need to edit the GestureManager.cs file to perform these steps.

  • Access the SpatialInputManager and MixedRealityService services.
  • Detect when the user is selecting every frame.
  • In that case, place the entity in front of the user at a certain distance.

You are welcome to write your entity behavior following these guidelines (part1 and part2) , or you can replace the file contents with the following code block:

Now let’s assign the new behavior to the mainPlane entity:

  1. Go back to the Wave Visual Editor.
  2. A prompt will appear asking permission to reload the project. Click Yes.
  3. Wait until the reload is completed.
  4. On Entities Hierarchy, select ‘mainPlane’ entity.
  5. On the top left area of Entity Details, select the ‘+’ button. The Add Component Dialog will appear.
  6. Write PlaceBehavior in the Filter textbox.
  7. Select PlaceBehavior in the component list.
  8. Select Ok.
  9. Find the PlaceBehavior component, and set the PlaceDistance property to 2.

6 – Build and Deploy

The WaveEngine project should have created two Visual Studio solutions for both of our app profiles: Windows (default) and Mixed Reality.

  1. In the project root folder, select ‘MixedRealityGame_MixedReality.sln’ (The solution name will vary depending on the project and profile names). Visual Studio will be launched with our Mixed Reality solution.

The instructions differ for deploying our game in a physical HoloLens versus the emulator. Follow the instructions that match your setup.

HoloLens over Wi-Fi

  1. Click on the Arrow next to the LocalMachine button, and change the deployment target to Remote Machine.
  2. Enter the IP address of your HoloLens device and change Authentication Mode to Universal (Unencrypted Protocol).
  3. Select Debug > Start. If this is the first time deploying to your device, you will need to pair it with Visual Studio.

HoloLens over USB

  1. Click on the Arrow next to the LocalMachine button, and change the deployment target to Device.
  2. Select Debug > Start.

Emulator

  1. Click on the Arrow next to the LocalMachine button, and change the deployment target to HoloLens Emulator.
  2. Select Debug > Start.

Debug and Release

If you want to deploy the app in Release,  you need to adjust the target:

  1. Using the toolbar in Visual Studio, change the target from Debug to Release.
  2. Use Debug > Start without debugging for deploying, instead of Debug > Start.

Try your app

Now that your app is deployed, try moving all around your room, and observe the plane and how it moves when you make the select gesture with your hand.

Leave a Reply

Your email address will not be published. Required fields are marked *