Connecting your HoloLens app with Microsoft Graph – Create a scene in Unity and connect your library – 4 of 4

Creating HoloLens applications is just cool! But connecting them to services in the Cloud makes it even cooler. Think of provisioning information on real-life objects or straw through Cloud data like persons, related contacts, documents and other stuff. This allows you to build rich applications containing information you normally process in a 2D world  like your browsers. By extending it to a 3D world, you are able to process the data in a completely different way. Think of creating teams of people within your organization and group them based on specialties, getting a more clear inside view of your site structure in SharePoint or have a 3D model of the Microsoft Graph entities related objects.

This article is split into several posts which together describes how to connect your HoloLens App with the Microsoft Graph.

1. Introduction 
2. Register, Connect and get data
3. Reference your class library in Unity
4. Create a scene in Unity and connect your library (this post)

Create a scene

For this example i’m using an available asset from the asset store to build curved user interfaces. It is supposed to be used for virtual reality glasses but with some adjustments you can use it also for HoloLens.

Part of the hierarchy of the canvas being used is the part of the friends panel. It contains a title and one or more (in our case 5) Friend GameObjects. In the image below you can see the structure of a single Friend GameObject which contains the layout, text and image parts.

The result at the end will look something like below. Each Friend GameObject contains the persons picture, name and some other values.

Getting the data in there is not that hard. We need several classes to get it to work. First of all a  graph source class handling the calls to the OfficeGraphLibrary class library. We attach a person controller to each Friend GameObject as a component. Finally we need a main controller to handle the data from the graph source to the attached controllers.

GraphDataSource class

This class handles all the calls to the OfficeGraphLibrary class library. It contains a method called GetAllPersons to get all the users from the Microsoft Graph. A return method is given as property which is called when the underlaying functionality has finished the call.

public class GraphDataSource : IDataSource
{
    private List persons = new List();
    private bool personsLoaded = false;

    public List Persons()
    {
        return persons;
    }

    public void GetAllPersons()
    {
#if !UNITY_EDITOR
        DataManager.Instance.GetAllPersons(OnGetAllPersonsCompleted);
#endif
    }

    public bool PersonsLoadCompleted()
    {
        if (personsLoaded)
        {
            personsLoaded = false;

            return true;
        }

        return false;
    }

#if !UNITY_EDITOR
    private void OnGetAllPersonsCompleted(List items)
    {
        persons.Clear();

        foreach (ItemData item in items)
        {
            persons.Add(item as PersonData);
        }

        personsLoaded = true;
    }
#endif
}

The method OnGetAllPersonsCompleted is called when the request to the Microsoft Graph returns a result. The result is a list of ItemData derived objects. In our case the PersonData objects containing user data. We load the returned data into a list as part of the class and set the property personsLoaded to indicate that the users are loaded.

Because this method is called from inside an asynchronous call in a separate thread, you can’t access any scripts or assets within your scene. That is the reason we need to use the personsLoaded flag to indicate it is ready.

You also see that we use a precompiler directive. Use the !UNITY_EDITOR directive to exclude code for the Unity Editor itself. This makes sure that we can still build a Visual Studio solution from Unity without using a replacement class library.

MainController class

This class is used to reference and attach the user data to the Friend GameObjects in Unity. First of all it calls the GetAllControllers method. This method searches for all GameObjects containing the component PersonController. Secondly it calls the GetAllPersons method on the data source class to retrieve asynchronously the users from the Microsoft Graph.

public class MainController : MonoBehaviour
{
    private List personControllers = new List();

    private GraphDataSource dataSource = new GraphDataSource();

    void Start()
    {
        GetAllControllers();

        dataSource.GetAllPersons();
    }

    void Update()
    {
        if (dataSource.PersonsLoadCompleted())
        {
            UpdatePersonControllers(dataSource.Persons());
        }
    }

    private void GetAllControllers()
    {
        PersonController[] controllers = gameObject.transform.Find("Panel/PersonsPanel").gameObject.GetComponentsInChildren();

        foreach (PersonControllercontroller in controllers)
        {
            personControllers.Add(controller);
        }
    }

    private void UpdatePeopleControllers(List persons)
    {
       for (int index = 0; index < personControllers.Count; index++)
       {
           if (index < persons.Count)
           {
               personControllers[index].Person = persons[index];
           }
           else
           {
              personControllers[index].Person = null;
           }
       }    
    }
}

Within the Update() method we continuously check if the personsLoaded flag is set or not. As soon as it is set we update the person controllers by attaching the PersonData to the controller. This example just fills the controllers as far as possible. Normally you would dynamically create your Friend GameObjects with the attached PersonController and implement paging. But for now it is just an example.

PersonController class

This last class is used to get the values from the PersonData object into the different components of a Friend GameObject and its underlaying structure. This script is attached to each Friend GameObject.

First of all we get all the necessary GameObjects and Components from the underlaying structure in  the Awake() method. This allows us to set the values later in the UpdateObject() method which is called when a new PersonData object is attached to the controller.

public class PersonController : MonoBehaviour
{
    private Image photo = null;
    private Text fullName = null;

    private GameObject ImageObject = null;
    private GameObject TextContainerObject = null;

    private PersonData person = null;

    private Sprite photoSprite = null;

    private void Awake()
    {
        ImageObject = gameObject.transform.Find("Image").gameObject;
        TextContainerObject = gameObject.transform.Find("TextContainer").gameObject;

        photo = gameObject.transform.Find("Image/Image").gameObject.GetComponentInChildren();
        fullName = gameObject.transform.Find("TextContainer/Topline/Text").gameObject.GetComponent();

        UpdateObject();    
    }

    private void UpdateObject()
    {
        ImageObject.SetActive(person != null);
        TextContainerObject.SetActive(person != null);

        if (person != null)
        {
            if (person.PhotoDetail.Photo != null)
            {
                // create texture with image byte array
                Texture2D photoTexture = new Texture2D(person.PhotoDetail.Width, person.PhotoDetail.Height);
                photoTexture.LoadImage(person.PhotoDetail.Photo);

                // create new sprite
                photoSprite = Sprite.Create(photoTexture, new Rect(0, 0, person.PhotoDetail.Width - 1, person.PhotoDetail.Height - 1), new Vector2(0.5f, 0.5f));

                // assign sprite
                photo.sprite = photoSprite;
            }
            else
            {
                photo.sprite = null;
            }

            fullName.text = person.FullName;
        }
    }

    public People Person
    {
        get
        {
            return person;
        }
        set
        {
            person = value;
            UpdateObject();
        }
    }

The UpdateObject() method does the actual setting of all the values. First we check if there is actually a photo present in the PersonData object. If that is the case we use a Texture to load the image from the byte[] array. Then a Sprite is created based on the Texture and the center point of the Sprite is set to the middle of the image. Finally we assign the Sprite and set the name of the user.

Conclusion

And that’s all! Of course this is just an example of how you can attach the data to assets in your scene. You need to decide for yourself how your scene will look like. Also don’t forget to implement a more dynamically number of Friend GameObjects with paging.

Keep in mind that when you implement functionality to access data from the cloud, in this case the Microsoft Graph, that it is wise to move that implementation to a separate class library. It also makes testing easier. You can create a separate UWP test application which references the same class library to test the majority of calls which you make to the Microsoft Graph without continuously deploying pieces of code to the HoloLens.