Microsoft’s HoloLens Has the Potential to be Transformative

[[Coi] Mr.CLuBBeR]

We tried Microsoft's augmented reality demo and couldn’t stop smiling

Many suspected that Microsoft would toss its hat into the virtual reality headset game. After all, Oculus VR was successful enough with its Kickstarter campaign that Facebook ended up purchasing it for two billion dollars and longtime console rival Sony jumped into the fray not long ago with its Project Morpheus. While Microsoft did reveal its own head-mounted display, HoloLens isn’t competing in the VR space, but is instead paving new way for augmented realities. We got a chance to try it ourselves and you’re probably wondering, “Is it any good?” Simply put, if it's executed right, it has the potential to be transformative.

A prototype sample of Microsoft Hololens headset

How does an AR headset differ from a VR headset? Whereas VR headsets try to completely take you into a virtual world, Microsoft’s augmented reality HoloLens is happy to keep you in reality. It instead opts to inject virtual holograms into your own real-life world (think the Princess Leah hologram and you pretty much get the idea). “This is your world with holograms,” Microsoft said of the device at its Windows 10 keynote. Unlike the fictitious technology in Star Wars, however, here you have to wear a headset. There is a see-through lens in front of your eyes as you wear the HoloLens and augmented reality objects are beamed into a rectangle in front of you. The rectangle isn’t all encompassing, however. You’ll still be able to see around the rectangle. The visual box is akin to you sitting in the middle of a movie theater, in that you can see more than just the lit screen. You can also technically see through this rectangle as well, considering it’s a see-through glass-like material, but we must say that Microsoft has done an incredible job making the background behind the rectangle disappear. We had to stick up a hand in front of our face to make sure that we could still see through it, and even then it was hard not to focus on the augmented reality visuals right in front of us.
While our developer kit unit was wired and featured a chest mount tethered to a jenky-looking headset (so jenky that Microsoft wouldn’t let us take any pictures of it), Microsoft says that the consumer version of the HoloLens will be completely wireless and will not require a separate device, like a smartphone or computer. The Hololens won’t require markers or even have an external camera either, even though it supports positional head-tracking. Instead, it will run Windows 10 itself and has its own dedicated CPU, GPU, and a new holographic processing unit (HPC) that processes all of these sensors together in real-time. Another little quirk that the dev kit has that the consumer version won’t is that Microsoft had to measure the distance between our eyes to properly configure the headset to our needs. Microsoft says this will be handled automatically with the consumer release.
The headset also has integrated speakers that provide spatial sound and a built-in mic that will allow you to issue voice commands. In addition, Hololens has an integrated depth sensor, supports stereoscopic 3D, and can track your finger gestures (provided they are in your own line of sight). Activating a command is as easy as holding your right fist one foot away from your chest (with your knuckles facing you), lifting your index finger up to the ceiling, and flicking up and down with said finger. What took a little time getting used to was the fact that you don’t control the headset’s cursor with your finger, instead the cursor is always fixated in the middle of your vision, so you essentially use your eyes to point at objects. We did get used to this after about five minutes, however.

The Minecraft-inspired demo looked nearly as good as this picture depicts here.

At Microsoft's Windows 10 event, the company showed off four HoloLens demos. The first one we tried was called HoloBuilder. It’s essentially a Minecraft-inspired augmented reality demo. We were situated in a living room with the headset on and when we looked around, the various desks and coffee tables had virtual 3D buildings and structures situated on top of them. It’s extremely impressive how seamless the real and the virtual blend together. It never felt like the augmented reality objects were inappropriately floating in space or didn’t have a sense of presence to them. It felt like all the Minecraft castles and farmlands were actually there (albeit in virtual miniature LEGO form). We couldn’t help but shout out expletives at how unbelievable it felt at times. Considering this was our first demo, we expected a rough experience but were blown away by how good it all was. Even in its early state, this demo left us in awe. One of the picture frames in the room featured a cavern and it really felt like we could stick our arm into the cave. The 3D depth here is amazing. Another experience had us looking at a short table on the floor. On top of the table were blocks of TNT that we could look at and explode with our activate finger gesture. When we blew up the box of dynamite, we saw the virtual earth open up to reveal magma underneath the floor. This might sound like hyperbole, but it looked so incredibly convincing that we had to step on it ourselves to make sure it wasn’t actually there.
The next demo that we went to had Microsoft representatives showing off the HoloLens’ HoloStudio tool. Microsoft believes that this program will bring about a “new medium for artistic expression and creation.” In short, HoloStudio is the company’s tool that will allow you to use Hololens to easily create 3D augmented reality objects using simple hand gestures and voice commands. From here, you’ll be able to get these objects 3D printed. Microsoft says HoloStudio represents a “perfect print preview for 3D printing.” This tool is quite interesting because it essentially blends the physical and digital worlds together. We saw a live demo of a Microsoft employee building a virtual toy koala in under two minutes. The toy looked quite impressive, but perhaps more impressive is that, according to Microsoft, the employee didn’t have any 3D modeling experience prior to prepping for the demo. He also showed us a 3D model of an X-Wing that looked quite accurate to the Star Wars incarnation. According to the rep, it took him only about an hour and a half. The idea here is to make building 3D objects easy for beginners and that you won’t have to be a professional 3D artist to construct interesting designs. In the live Holostudio live demo, the Microsoft rep was able to pull objects out from a virtual toolbox and copy/reorient them with simple tap/voice commands. The software looks promising, but unfortunately Microsoft couldn’t tell us if it would be bundling the software for free with purchases of HoloLens.

Here is a 3D printed figure of what the virtual koala looked like.

The next demo we did took us to space, specifically Mars. Microsoft has been working with NASA’s Jet Propulsion Labs to recreate a 3D rendering of Mars using Curiosity's images. While our physical body was located in some office in Bellevue, Washington, it felt like we were walking along the red planet. When we asked if any of the landscape were computer rendered, the Microsoft rep told us that all the imagery was actually from Mars, and that it was the closest thing man has ever come to walking on the planet. Suffice it to say our jaws hit the floor. It just looked so realistic. Never did it feel flat or like images were being stitched together. We actually felt like we were on Mars (at least as best as a see-through augmented related lens can deliver). Microsoft also sent up a tour guide in another physical room to show us around the alien planet. Our tour guide was completely gold (picture the Silver Surface except gold). This is but one humanoid prototype Microsoft said it was working on. This tour guide has Superman-like laser beams pointing out of his eyes (minus the laser sound effects) that would point us in the direction of interesting objects spread throughout the desert terrain. By focusing in on objects, we could use the flick command to zoom in on surfaces. Again, the headset features positional head tracking so we were able to get on our knees and closely observe the rocks underneath our feet. It was a surreal experience and we could definitely see NASA using this headset for more research.

Who knew Skype could be so informative?

The last demo we saw had us interfacing with a real human being over Skype. While that sounds rather boring, the premise was that this person would help us repair a broken electrical socket on a wall. What makes the demo interesting is that the person on the other end of the Skype call could see what we were seeing and she was able to walk us through all of the steps to fix the wall socket, connecting wires and all. One particularly interesting thing about this demo is that once she asked us to take a seat by the wall socket, she asked us to look down at the tools. So we did what we were asked and looked at the virtual tools below her video feed. She asked us to look down at the tools again and we realized she meant the real physical hammers and such on the desk right next to us. It’s sort of crazy how the physical and digital are already becoming hard to distinguish in AR. Once we picked up the real physical tools, we were able to pin her video feed to the side of the wall socket (so that it wouldn’t float in the middle of our vision and obstruct our view of the wall socket). From here, she was able to give us very clear and precise instructions on how to fix the issue and it worked about as well as someone giving you instructions over the shoulder in real life.
It’s extremely exciting to see what AR will allow people to do. Obviously helping someone fix an electrical socket over Skype is one of them, but you’ll also be able to get cooking lessons from your mom or learn how to fix a car from your dad and so on. We see a lot of potential here in the professional world as well. It could potentially aid surgeons in the ER or help soldiers avoid potential land mines in the desert. Then, yes, of course there are game opportunities out there. You could potentially do some unique eye-spy or hide-and-seek type games around your house or blast aliens as they start coming in from your kitchen. Microsoft says gaming will be a big component of HoloLens, but it will be up to the developers to push the boundaries of what’s possible with AR games. In addition, as we also saw with the Mars demo, virtual tourism could be a big thing.

If pulled off right, Hololens could be transformative.

Of course, virtual tourism may be better suited for VR experiences where you are completely visually isolated from the outside world, and some people were bothered by the fact that you could see through and around the lens. It didn’t bother us, however. It was actually hard to stop smiling at points.
Microsoft is being quite coy when it comes to the technical aspects of the device. When we asked the company what resolution the HoloLens ran at, it merely said “HD.” In theory this means 720P and up. Regardless, from our experiential test, we didn’t have any major issues with the resolution and thought it looked quite sharp for a developer kit. Of course, we’ll take higher resolution any day of the week, but the resolution that Microsoft is currently running is ready for consumer release.

HoloLens could open up 3D modeling for beginners.

When will it be releasing? Microsoft says during the Windows 10 launch timeframe. Considering that Microsoft aims to release Windows 10 sometime this year, hopefully you won’t have to wait too long to try it yourself.


More...