How to integrate the Vuforia platform with your HoloLens projects

Vuforia is an augmented reality platform which allows us to bind holograms to images, object and environments in the real world. It allows us to position holograms at an exact location and by that delivers a unique experience for business solutions. Think of additional information next to an image in a book, a 3D model of a machine as a layer over the real part in that machine, tracking of images with digital content and more.

The case

I will explain how easy it is to use Vuforia with a real example. The example is about showing a 3D model of a car as soon as the user with a HoloLens looks at the logo of the car brand. Because I deeply love the brand Lamborghini, I will be using their logo and as a 3D model the Lamborghini Aventador. The 3D model will be shown above the logo. It will follow the logo in any direction. We will be using a target database on the HoloLens device to achieve maximum performance of tracking.

At the end of the post you will find a youtube movie showing the result of our HoloLens application.

There are some steps involved which are described below.

Create a Unity solution for the HoloLens app

Create a Unity solution called “ImageTracking” and make sure you have 3D selected. Unity will now create a project for you.

When the project is created you will need to do several steps to make sure that the Unity solution is working for HoloLens. First save the scene as main.unity. Make sure you have set the properties of the camera object. That means that Clear Flags should be Solid Color and background is set to black.

Go to Edit à Project Settings à Quality and set the default quality to Fastest.

Go to Edit à Project Settings à Player. Under Other settings select the option Virtual Reality Supported and make sure that Windows HoloGraphic is choosen. Under Publishing settings you need to select the following capabilities

• InternetClient
• WebCam
• SpatialPerception (only if you intend to use the Surface Observer API)

Other capabilities can be chosen according to your wishes for your app.

Finally try to build the application using File à Build & Settings. Select Windows Store platform and as target device the HoloLens. Check Unity c# projects and Development build. Build the solution in a separate folder called App.

Vuforia integration

We need a unity package to get Vuforia integrated in our Unity project. The package you need is not easily found.

Go to https://developer.vuforia.com/downloads/samples and look for the “Download for Unity” link under “Digital eyewear. You need to be logged in as a registered user. This registration is free. The zip file, currently version 6.2.10, contains the unity package we need. Get the Hololens-6-2-10.unitpackage from the zip file.

Go to Assets à Import package à Custom package and select this file. The package will now be imported into your project.

Select the Import button to import all the parts. We need to add the prefab “ARCamera” to our scene. This prefab is found in the project window under Assets à Vuforia à Prefabs. You can drag the prefab in the hierarchy of the scene.

We are almost there. Select your initial camera (which is the HoloLens camera) as the Central Anchor Point. This is done in the Vuforia Behaviour Script component of the ARCamera.

License key and target database

Before we can start using the Vuforia engine we need to have a license key and a target database. You will need to register at the developer portal of Vuforia. This registration is free. When you are registered and logged in we can create a license key.

Go to https://developer.vuforia.com/license-manager and click on the “Add License Key” button.

Select “Development” as project type. This is free of charge. Then name your App and finish the screen.

Open the license by clicking on the name of the license.

You will see the License key. Copy the key and open the Unity project. Select the ARCamera and click on the “Open Vuforia configuration” in the component “Vuforia Behaviour script”.

Go back to gameobject ARCamera and click on Open Vuforia configuration.

Copy the license key in the App License Key field. Make sure that Eyewear type is set to Optical See-Through and See Through Config is set to HoloLens.

Now get back to the developer portal of Vuforia and open the Target Manager. Click on the “Add Database” button.

Enter the name “ImageTrackingDatabase” and select the type “Device”. This means that we are going to have the database on the HoloLens device. Press the “Create” button to generate the database.

Open the database by clicking on the database name. In the following screen, we have the option to add targets. Targets are images and objects we use to target on through Vuforia.

We add the Lamborghini logo as jpg file. Keep in mind that the only formats which are allowed are 8-bit gray scale or 24-bit RGB of file types JPG and PNG. I noticed that it is easier to get it to work with JPG than with PNG.

Secondly, we need to specify the size of the target in scene units. Since Unity scene units are in meters we specify that our logo is around 10 cm large. Set the width to 0.1.

Enter a name and press the “Add” button to add the target.

Your new target will be shown in the target list. The target gets a rating. The rating determines the quality of the image or object. The quality determines how good Vuforia will recognize the target. More information about this can be found in optimizing target detection and tracking stability.

Because we have decided to have the database on the HoloLens device we need to download the database. Click on the “Download database (all)” button.

Select Unity Editor as the desired development platform and select the “Download” button.

You will get a ImageTrackingDatabase.unitypackage file which you need to import into you Unity project via Assets à Import package à Custom Package.

Click on the “Import” button to import the database and its assets into your project.

Go back to gameobject ARCamera and click on Open Vuforia configuration.

Select your dataset and make sure you select the Activate. Deselect the demo StonesAndChips database. We won’t need that one.

I noticed that the image we have supplied in the database seems not be imported correctly. It is called Lamborghini_scaled.jpg. We need to fix that one.

Go to the scaled image in Assets/Editor/QCAR/ImageTargetTextures/ImagetrackingDatabase

Set the Texture Shape of the image to “” and click on the “Apply” button.

Adding tracking to the Unity scene.

To have tracking of the Lamborghini logo we need to create an additional empty GameObject in the scene. We call that one “Tracking”. We need to add some components to that GameObject.

Add a component called “Image Target Behaviour”. This component needs to be configured with the right database and image target.

Make sure you select “Enable Extended Tracking”.

Add a component called “Turn Off Behaviour”. The previous component added the logo to the scene. You could remove it if you want. But it could be handy in the Unity Editor to correctly position the 3D model we want to show. This script disables and hides the logo when you run the application.

Finally, we need to add the component “Default Trackable Event Handler”. This script shows or hides all child GameObjects depending on tracking found or lost.

In our Unity project I have imported a 3D model of a Lamborghini Aventador found online. I have added the model as a child of the GameObject “Tracking”.

I resized the 3D model to a tenth to make it smaller than the logo. You can use any size, position or rotation as you please.

The result

Build the Unity project and create a Visual Studio solution. Deploy the solution to your HoloLens device. Print the Logo on a piece of paper. It does not have to be on paper. It even works from your screen.