I knew the Vive headset I was wearing was too tight. This was the HTC Vive Pro Eye, a new model introduced at CES this month in Las Vegas, that adds full-on eye-tracking. It’s otherwise very similar to the existing high-end Vive Pro headset that HTC introduced in 2018. I’ve gone through enough eye-tracking setups to know that it would work better on my glasses if loosened just a bit. Looking up, down, following a blue dot… and the setup worked. From that point on, it was hard to tell that eye tracking was even happening. But it was.
Eye tracking has been a promised feature in VR and AR for years, and has been available previously in modified VR headsets. Tobii, a prominent eye-tracking technology company, built its tech into the HTC Vive Pro Eye. I’ve used Tobii eye tracking on Vive headsets before this, and it’s been available as an upgrade option for those who wanted it. But the Vive Pro Eye is the first VR mass-market hardware with eye tracking available standard. It won’t be the last.
The Vive Pro Eye is aiming for enterprise markets (which makes sense, since the Vive Pro Eye is expected to cost more than $799 for the headset alone), and Tobii’s already built analytics tools using eye tracking to help examine how people interact with advertisements or store layouts. It sounds like eye tracking will largely be a tool used to study engagement, improve graphics and also help improve controls (if I can look at something and then move my hand to control that one thing, it will massively improve aiming).
Qualcomm’s next-generation mobile VR headsets are expected to have eye tracking. Mixed-reality headsets like the Magic Leap One and possibly the next-gen Microsoft HoloLens will include it. And in the next few years, every VR and AR headset in existence will probably have it, too.
Some potential killer apps
Ovation’s public speaking training software was one of a half-dozen or so eye-enabled experiences at HTC’s CES preview of its enterprise-focused VR hardware, coming out by midyear. I stood up in front of an audience in a simulated CES press conference, delivering a speech off a teleprompter. I gestured with my hands, tried to keep my delivery interesting. The avatar crowd followed me with their eyes, looking right at me. And thanks to embedded eye tracking, I was looking right at them.
I aced the VR public speaking @htcvive eye tracking training program by Ovation. Like, demolished it. #ces2019 (high score for the day!) pic.twitter.com/d2DJsjmp9g
— Scott Stein (@jetscott)
A heat map after my speech showed exactly where my eyes were wandering, and a test score rewarded me for dividing my gaze between both sides of the room, and between teleprompters and audience. It turns out I aced my public speaking exam. And eye tracking was able to deliver analytics that regular VR headsets can’t.
I’ve been doing VR from 9:30am to 3:23pm. Stopping now. Here is me, again unflattering, in a Lockheed flight simulator w @htcvive pro eye tracking. #CES2019 pic.twitter.com/kV5AyCWS33
— Scott Stein (@jetscott)
Lockheed Martin’s Prepare3D flight simulation software had me sitting in a chair with flight simulator controls and with VR it felt like I was in a cockpit studded with readouts. While following directions, the training software could also follow my eye movements and help me know if I was looking in the right places, or if I was becoming distracted.
An MLB Home Run Derby VR experience that will be installed around MLB ballparks had me batting against virtual pitches with an actual connected bat, but also using my eye movement to engage with pitches and control my swings.
Day of VR, ongoing: unflattering photo of me getting in a car w eye tracking @htcvive Pro. #CES2019 pic.twitter.com/YoQvVc44UT
— Scott Stein (@jetscott)
VR’s next frontier: Foveated rendering
Eye tracking can help improve graphics on lower-powered devices by using foveated rendering, showing in highest detail only what the fovea or center of eye takes in. Graphics on high-end PCs can be pushed even further, too, by using the same technique.
I saw this in action in ZeroLight’s demonstration of a virtual BMW car showroom, where I walked around and checked out details of a BMW M in VR and looked at dashboard details in high resolution, changing interior details and colors for comparison. Once the demo was paused, I saw how the Vive Pro Eye rendered everything in my vision at higher resolution, and fuzzier at the edges.
But eye tracking has a weirder, darker side: Who will know where I’m looking? And will companies like Facebook, Google, and other VR and AR manufacturers make sure that data is handled responsibly? I’m reminded of a conversation I had last year with a mixed-reality executive who reminded me that eye tracking could eventually be a form of mind reading.. .perhaps even a flag for future behavioral tendencies. Imagine a future in which a ‘wandering eye’ becomes a literal Minority Report-style ‘precrime.’ Eye-tracking technology is a great and mysterious power, and headsets and phones are going to gain it really soon.
The Vive Pro Eye is just the start.