Under the Hood: The State of VR

Anyone who’s paid any attention to the exciting and expanding virtual reality industry knows that there are two competitors leading the market – Valve and Facebook’s Oculus. Valve’s Vive headset, manufactured by HTC, and the Oculus Rift and now the newly released Oculus Go, manufactured in-house, are essentially the only real options for VR gaming at the moment. Even Lenovo has now managed to throw its hat into the wireless ring with the Mirage Solo (built on Google’s Daydream VR platform).

The real story of VR, however, is playing out under the headset. See, the real profitability in VR gaming has nothing to do with the hardware, a one-time purchase; the main goal is the sale of software. Producing a headset merely allows a company to control the distribution of its software, meaning that everyone who buys the headset is essentially locked into purchasing their games from that company. Valve has Steam, and Facebook has Oculus Home as distribution platforms; however, the story of VR software – and therefore the story of VR itself – runs deeper than this.

There are a lot of different kinds of software involved with running VR applications. Perhaps the most important – depending on who you ask – is the application itself. The creation of 3D virtual worlds isn’t really new – video games have consisted of maneuvering around 3D worlds since the N64 days. The only real difference with VR is the number of displays – one viewpoint for each eye, which is a trivial change – and the controls. Instead of a joystick telling the application to move the viewpoint around, automatic sensors in a headset accomplish the same thing. So really, the creation of virtual spaces is hardly different from the creation of modern video games – and because of this, the same tools are used. These tools are called game engines, and they provide a software framework for a lot of things necessary in the creation of a game; without them, developers would have to waste time on things like how physics should work in their game, when that problem has already been pretty well solved. The current leading game engine is one called Unity – and it’s currently leading the VR market as well, being used by 59% of VR developers. However, while VR and virtual world creation have their roots in gaming, that’s certainly not the extent of their future; this is something recognized by Unity’s top competitor, Unreal Engine.

So Unity and Unreal are game engines used to create VR applications. Okay. But that’s not the whole story. As mentioned previously, there’s a lot of software that goes into running a VR application. For one thing, there’s the underlying operating system – something that’s often forgotten. For all their wonderful modern capacities, computers can still only perform one task at a time (per processor) – it’s the operating system’s job to simulate parallelization of tasks to keep all your programs running at the same time. Right now, VR gaming – like too much of gaming in general – seems to be Windows-specific. Then there are the device-specific driver programs and runtime environments – programs managed by the OS. In the case of the Rift, the runtime environment is installed automatically by Oculus Home; for the Vive, the runtime is SteamVR, installed by Steam.

Here, things get a little more confusing. Let’s go back to the creation of VR applications. To make a game work with a specific VR headset, the developer must follow a certain protocol – basically, it needs to be speaking the same language as the headset. Valve and Oculus have made this easy by releasing SDKs (software development kits) that allow any developer to make their game compatible with either headset (or both – more on this in a bit). Oculus has their own Software Developer Kit (SDK), while Valve has released something called OpenVR. OpenVR is a little tricky – it’s not actually just for making games compatible with the Vive; it’s more ambitious than that. OpenVR provides two protocols: one for software developers, which allows applications to talk to a runtime environment; and one for device manufacturers, which allows the runtime environment to talk to the drivers of any device implementing OpenVR. In this, Valve attempts to create an industry standard – and very early on in the industry. Widespread adoption would be great for Valve, as OpenVR currently necessitates the presence of Steam on the computer in question. Valve, using OpenVR’s device driver interface, has even managed to get the Rift to play its Steam games. Instead of talking directly to the Rift’s drivers, SteamVR talks to the Oculus runtime, which in turn passes on the message to the actual headset. The most interesting thing about this is that it requires both Valve and Oculus to allow, explicitly or implicitly, the mixing of headsets and storefronts.

At the moment, both Unity and Unreal Engine provide native support for both OpenVR and the Oculus DK2 – or both. The ease that comes with this is part of what’s keeping both engines at the forefront of VR development.

Let’s take a look now at some of the VR players out there other than Valve and Facebook. The biggest question with any corner of the tech industry tends to be, “What has Google been doing?” and in this case, the answer is “a lot”. Google has for quite a while now had its own top-tier operating system on the market – just not for desktop computers. Now, they’re building on top of their Android mobile OS a full VR user experience calledDaydream. It comes with many of the user interface and windowing features typical of an operating system, and it helps Android deal with switching between all those VR tasks. Daydream comes with a relatively inexpensive headset that holds an Android phone, which is the real computer powering the experience. Daydream provides an interesting look into the future, into how our experiences with virtual reality might be managed. A similar project was launched last year by a startup known as Marvel, which raised over $100,000 on Indiegogo, more than 500% of its $20,000 goal.

Additionally, Microsoft itself has gotten into the VR market with its Windows Holographic (now Windows Mixed Reality) project. The chief Mixed Reality device is the Microsoft HoloLens, an augmented reality (AR) device kind of like the Google Glass, designed to display “holograms” (that is, 3D virtual models) in the real world. Again, these AR experiences tend to be created in Unity – Microsoft even recommends the use of Unity in its Mixed Reality “Academy”. Like with Daydream, Windows Mixed Reality will have a special sort of connection with the underlying operating system, giving perhaps a leg up in the VR competition against third parties like Valve and Facebook, already reliant upon Microsoft Windows.

That’s most of the big companies, but let’s tackle the other two giants: Apple and Amazon. Apple, though not yet too much in the public eye, has actually been quite active in VR and especially in AR. In a 2016 interview with ABC News, Apple CEO Tim Cook said that Apple is “doing a lot of things” with AR. He talks about how he believes AR will have a much larger market, moving forward, than pure VR. Apple has already filed several VR and AR related patents, including one for a transparent AR device and one for a VR headset to be used with the iPhone, much like Google’s Daydream headset and Samsung’s GearVR.

As for Amazon, they seem to be staying out of the hardware market for now – though they’ve already begun work on software for other companies’ VR platforms. A job posting from the company called for someone to head a new VR division, tasked with “building the Virtual Reality experience within Amazon Video”. This makes sense – for a while now, Amazon has quite successfully competed with services like Netflix and Hulu with its Prime Video. And, as stated, before, the big money in VR is in software, not hardware. It remains to be seen whether or not Amazon will at some point begin work on its own VR platform and/or headset.

The last facet of the story of VR is the applications themselves – and at this point, they’re mostly video games. While major studios like EA and Ubisoft have begun to dabble in VR, most of the interesting content is coming from startups. This kind of makes sense – large companies have institutional inertia preventing major shifts from happening instantly, but brand new companies can tackle whatever they want immediately. One of the most popular VR games – one played by just about anyone who owns a VR headset, as it was one of the first (and still one of the best) VR first-person shooters – is one called Raw Data, developed by VR startup studio Survios. Survios has raised many millions of dollars from both venture capital firms and investors like MGM – VR development is a lucrative industry. Another important game so far in the world of VR is AltspaceVR (also the name of the startup), an almost MMO-style game that allows VR users around the world to meet up, engage in activities like disc golf and “Holograms Against Humanity”, and even attend live events hosted by celebrities like Reggie Watts.

On the subject of VR MMOs – an incredibly successful Kickstarter campaign for a game called OrbusVR last year raised $34,524 – more than 340% of its goal. The hype is out there for VR development, to be sure. The game is still in development, with a projected release date of December 2017. It’s described as “a room-scale fantasy VR MMO featuring combat, exploration, and crafting”. Its described mechanics definitely sound similar to World of Warcraft and other popular MMOs on the market. The idea of hanging out with friends regardless of physical location and teaming up to fight dragons (or whatever) is exciting for anybody, not just gamers; and while, like VR in general, this mechanic may start with video games, it’s sure to expand into all kinds of social virtual worlds.

Eugene Chung, Founder and CEO of Penrose Studios said, “Rarely do we see radically new art forms emerge, but we recognize them when we see them. To those who have experienced VR, it is that we are quite literally looking at the future. Storytelling in virtual reality is still in its infancy, but it represents an exciting new medium of artistic expression. It is a revolutionary departure from the moving pictures of today.”

From a slightly different vantage point, Josh Sackman, President of AppliedVR added, “There is a lot of really engaging, entertaining VR content out there. But, there are very few content developers who are performing clinical validation and extensive user research to understand the effectiveness and usability of their content.”

We’re at a very interesting point in the history of technology. This is the dawn of virtual reality, a technology that has the power to entirely transform our day-to-day lives. It’s a relatively small market at the moment, but it has incredibly exciting future potential. It’s likely that in addition to hardware, the underlying software of VR will continue to diversify in coming years. Over-reliance on a single platform isn’t sustainable – competitors will emerge, and competition will drive innovation. It will be exciting to see what the future holds.