It is great to work for a company, ETTU, that does a lot with new technology and embraces being innovative. This week we got the developer edition of the HoloLens at our company. And I had the privilege to try it out.
As soon as I heard that we ordered a HoloLens I started to investigate what you could do with the device. I knew the possibilities and did a lot of reading about the device itself. The HoloLens is an augment reality device. For me this means that it mixes the digital world with the “real” world we are living in. What does that mean? It means that it allows you to interact with objects you create yourself, which suddenly live in your world through the HoloLens. Examples are placing an Internet Explorer view at one of your walls in the office, playing a game that uses your environment to interact with, modelling a design that can be viewed from all sides by just walking around it and many more.
I already started to look into what is needed to create your own augment reality applications. There is a lot of content to be found on the internet. It is possible to start developing applications or try out stuff before you have the HoloLens. Microsoft has released a HoloLens emulator. It does not replace the actual HoloLens but allows you to get started with some of the development.
Using the HoloLens emulator is not the same as wearing the actual HoloLens device. The movements you normally would do with your head to focus on an object and moving around in a room is now translated into keys on your keyboard. Also it is not possible, I didn’t found any solution for that yet, to view the actual room in the HoloLens emulator. Yes, there is the possibility to load a room. But still you only can view that room when the “spatial mapping view” is enabled which results in a wireframe of the room instead of an actual room. There is another possibility by adding a 3D generated view of the room through the device portal page of the HoloLens, import it via Unity and map it exactly at the right position inside your scene with your own virtual created elements. But that still doesn’t give you an actual room as you would expect.
Start developing applications
To start developing your own applications for HoloLens, you will need some good hardware. You will need at least 12GB of ram and if possible an i7 Core processor. You could do with less but building and deploying the application will take longer. Used for developing augment reality applications is a combination of Visual Studio 2015 SP3 and Unity. Unity is only needed when you want to model 3D objects for your application. To be honest in most cases you will use Unity as a starting point. The process is rather straight forward. Unity is a 3D modeling studio application which is used for creating virtual objects and for example games. Unity has the ability to create 3D scenes in combination with code functions to interact with the objects in that scene or the scene itself. It allows you than to generate a Visual Studio project based on your scene and all those interactions. That Visual Studio project can be enhanced and expanded with coding by using Visual Studio. When you finished you have the ability to deploy and run the application in the HoloLens emulator or in the actual HoloLens device. To deploy applications on the device you will need to enable “Developer mode” in the device. Since the device runs Windows 10 it makes that rather easy through the well-known settings page. You can connect to the device via the USB cable or through WIFI. In both cases, the first time you will need to validate the connection between your Visual Studio and the device by a PIN code. And there you are! Your application has deployed and runs on the device. I will go deeper into developing applications for HoloLens in a separate blog post.
There are some great tutorials available online to understand the different features which are provided by the HoloLens. You will need to learn to work with those features to interact with the Windows Holographic Platform. Some of these core features are gaze, gestures, voice input, spatial sound and spatial mapping. Gaze tells you where the user is looking at. Instead of the eyes the position and orientation of the user’s head is used. That gaze also determines the view area where holograms are projected. The view area is the area in which the holograms are shown, when they exist in that area, in combination with the environment.
With gestures and voice input you can interact with holograms. You can use your hand or an optional clicker. In my case I only used my hand. Both hands are tracked by the HoloLens. There are three core interactions, press, release and bloom with HoloLens. Press and release is noticed when the user does an air-tap hand movement. Bloom is a special gesture which allows you to go back to the start menu. Voice commands can be easily placed on your holograms in your application. For example, when the user points with the gaze at a certain hologram object, it can be dropped by using a voice command like “drop object”. The voice commands are also used when interacting with Cortana, selecting holograms, Dictation and something called “See it, Say it!”. HoloLens will select a certain button if you say the text of the label. Spatial sound allows you to place sounds in a three-dimensional space around the user. This is used to make the holographic experience more believable. Last but not least is spatial mapping. Spatial mapping creates a detailed representation of the environment around the HoloLens. This will help you to create more convincing mixed reality experience. It allows you to merge both virtual and real world together.
The HoloLens is a great holographic device to create augment reality applications. It takes some time to get used to the device and how to interact with your surroundings. But the possibilities are endless. Nevertheless I think the device with the price of 3000 dollars is more at his place when you do modeling of buildings or devices then playing games with it. Developing applications seems not too difficult. While I’m not a designer which creates 3D objects in Unity there is a lot of packages containing 3D objects created by others which you can get for free or for a small amount. The SDK, which you will need to program against the HoloLens, is just incredible! Even while this device is still in its first version, you don’t need much code to get things up and running.
For me there were two downsides to the device. First the weight. It feels heavy and in some cases it actually felt discomfortable. Secondly the view area in which the virtual objects are projected is to my opinion too small. It is a square area in front of the direction you are looking in. That means that holograms which are not within that area are not shown. You can compare it when you talk with someone you still notice and see things around you without moving your head. In case of the HoloLens those areas are not filled with holograms which gives a narrow effect.
Still the device is great! I will continue to work with it and see what we can achieve with the device for our customers or just for fun!