The gaze is one of the different forms of input, but also one of the most important ones, used with a HoloLens. The gaze is used for targeting holographic objects and your environment. The gaze helps you as developer to determine where the user is looking at and it allows you to predict what his/her attentions are. The position and orientation of the head is used to determine where the gaze is pointing at. The gaze is presented as a holographic object itself. You can use any type of visual gaze as you wish allowing you to create some nifty gaze cursors.

The purpose of this post is to explain how you implement a gaze which interacts with the physical environment around you. It expects that you have a project setup with Unity containing at least one scene containing a Main Camera and Directional Light. I will be using the default project setup I described in an earlier post.

Creating a gaze cursor object

As already mentioned you can use anything for a gaze. For now, we will keep things simple and use the 3D cube object you can create with Unity.

We will create a prefab for the gaze containing a 3D object, material to style the 3D object and code to handle the gaze. The prefab is actually a package containing everything needed to place a gaze cursor in your scene. Changes made in the prefab will reflect into the objects based on that prefab in your scene. Kind of inheritance with C#

  • Go to the Hierarchy view
  • Select the scene in the hierarchy. In our case the scene is called SceneStart
  • Go to menu GameObjectà3D ObjectàCube
  • Go to the Project view
  • Create a new folder named Gaze under the Assets category
  • Go into the new folder named Gaze
  • Drag the Cube from the Hierarchy view to the Gaze folder
  • Remove the Cube from the Hierarchy view
  • Right-mouse click àCreateàPrefab and name it GazeCursor
  • Right-mouse click àCreateàMaterial and name it GazeMaterial
  • Configure the material by changing the color in Albedo
  • Configure the Cube to disable the Box collider. If the collider of the cube is active, you will notice that the cube is coming towards you due to hitting itself continuously.
  • Drag the Cube from the Gaze folder into the Cube prefab
  • Drag the GazeMaterial from the Gaze folder into the Cube prefab

At this moment, we have a GazeCursor containing no code.

Gaze cursor script

The gaze cursor script handles the routine to check if the gaze cursor hits the physical environment defined by the HoloLens. We will be adding a C# script to the GazeCursor prefab.

  • Go to the Project view
  • Go into the folder Gaze Cursor
  • Right-mouse Click àCreateàC# Script
  • Rename it to GazeCursor.cs
using UnityEngine;

public class GazeCursor : MonoBehaviour
{
    private MeshRenderer meshRenderer;

    // Use this for initialization
    void Start()
    {
        // Grab the mesh renderer that's on the same object as this script.
        meshRenderer = this.gameObject.GetComponentInChildren();
    }

    // Update is called once per frame
    void Update()
    {
        // Do a raycast into the world based on the user's
        // head position and orientation.
        var headPosition = Camera.main.transform.position;
        var gazeDirection = Camera.main.transform.forward;

        RaycastHit hitInfo;
        if (Physics.Raycast(headPosition, gazeDirection, out hitInfo))
        {
            // If the raycast hit a hologram...

            // Display the cursor mesh.
            meshRenderer.enabled = true;
            // Move the cursor to the point where the raycast hit.
            this.transform.position = hitInfo.point;
            // Rotate the cursor to hug the surface of the hologram.
            this.transform.rotation =
                Quaternion.FromToRotation(Vector3.up, hitInfo.normal);
        }
        else
        {
            // If the raycast did not hit a hologram, hide the cursor mesh.
            meshRenderer.enabled = false;
        }
    }
}

  • Drag the GazeCursor.cs file into the Inspector view of the Gaze Cursor prefab.

The script is similar as the code from the Microsoft examples of HoloLens. The Physics.Raycast will return a hit a soon as the Gaze Cursor hits a holographic object in your environment.

Extending the scene

We have created a Gaze Cursor for our scene. To use the objects, we need to add them both to your active scene. Drag both prefabs into the Hierarchy view as a child into that scene.

The result

And that’s all the work we had to do. Let’s see how it runs in the HoloLens. Build your application and let Unity create the Visual Studio solution. Open the solution and run the application on your HoloLens device. The result should be like the movie below.

Previous articleSetting up an application environment for HoloLens
Next articleCreate a gaze which interacts with the environment for HoloLens
A professional which inspires, motivates and educates businesses on how to leverage emerging technologies using Virtual, Augmented and Mixed Reality in their journey to digital innovation. He has a strong background on SharePoint, Office 365 and Microsoft Azure and is working with machine learning, artificial intelligence and cognitive services. Primarily focused on manufacturing, construction, industry, logistics and maritime/offshore, his goal is helping businesses to achieve more on creating, improving, smartening and shortening of business processes by using services, apps and devices. Furthermore, he engages in speaking, blogging and is organizer of events like the Mixed Reality User Group, Mixed Reality Talk @ VR, SP&C and the Global AI/MR Bootcamp. With more than 20 years of IT Business experience, acting in different roles like solution architect, he believes the future is just starting! Highly experienced in all realities, HoloLens and other devices, User experiences on devices and connecting the world to the cloud will help him to shape that future!

LEAVE A REPLY

Please enter your comment!
Please enter your name here