June 21st was like Christmas in June for VR enthusiasts like myself. Meta shared their progress on their ongoing projects. Even though I'm not sure that this was a statement for us enthusiasts or a publicly-traded company explaining their current financial results to its stakeholders, I was thrilled to know about their approach to tackling those hard problems. And as I read through this amazing dev-logish writing I've felt impatient for what the future holds for us.
TL;DR Creating a visually appealing, aesthetic virtual reality headset with an accessible price point is hard and it will take some time but fortunately, there are lab full of people working on it.
They reinterpreted the meaning of the “time machine”. In their words, a time machine is far from consumer-ready, built for the purpose of probing what might be possible with years of research and development. That definition inspired me since it glimpses the future for us and with the current prototypes, things seem achievable.
They showed us four headset concepts as solutions for four different crucial problems: Half Dome Series, Butterscotch, Starburst, and Holocake. The fifth one, Mirror Lake is only a concept with the combinations of all the best features of the previous four. It is a concept that took Meta 7 years to develop.
Mark Zuckerberg and Head of Display System Research Douglas Lanman sets the bar to passing the visual Turing test. Regular Turing test was designed to test the intelligence of a computer. Test considered successful if a person cannot distinguish the machine from another human being. Visual Turing is for testing our human eyes whether they are seeing a real or virtual world. It is a relatively subjective test and no VR technology can pass it today.
Meta Research focuses on four features to be able to pass the virtual Turing test.
- Varifocal technology that adjusts the focus of the display based on what we’re looking
- Resolution that approaches and ultimately exceeds our human vision
- Distortion correction to help address optical aberrations
- And high dynamic range (HDR) technology that expands the range of color, brightness, and contrast you can experience in VR
It took me a few minutes until I understand they are not working on a new generation Quest. They were all prototypes for a good reason. Lanman notes the complexity of the task: “Designing and building headsets that incorporate that collection of technologies is difficult and time-consuming work because, with headset displays, all technical systems are interconnected. Everything competes for that same size, weight, power, and cost budget, while also needing to fit into a compact, wearable form factor.” And it’s not just a matter of squeezing all the technology into a tight budget — each element of the stack must also be compatible with all the others.
Half Dome Series
Vergence Accommodation Conflict (VAC) is a well-known phenomenon in VR. VAC occurs when the binocular disparity, which causes the eyes to the verge, disagrees with the distance that the eyes are focusing (accommodation)^(). In simpler terms, it is what makes you feel nauseous when you are wearing a VR headset. To solve this issue, RL works on the varifocal technology. Varifocal tech has the power to adjust the focus of the display based on what you’re looking at. As we hear from the first prototypes weight and aesthetics weren’t the only issue. Headset prototype was very loud due to moving mechanical parts. With the HD3, they ditched the motors and replaced them with multiple liquid crystal lenses. Why isn’t this already hitting our shelves? Because CTO of Meta, John Carmack, points out two points: It is costly and imperfect varifocal may not be that much better than what we have now. Why making this real is important? Lanman has a great answer for this, which he talked about in this detailed presentation in EI 2020 Plenary: Quality Screen Time: Leveraging Computational Displays for Spatial Computing, because “VR is not a television it is a great personal action space.”
Four iterations of the Half Dome prototype. Credit: Meta/ Reality Labs
Image clarity is also crucial for passing the virtual Turing test. Our human eyes’ retinal resolution is around 60 pixels per degree (ppd) and Butterscotch aims for that and beyond. My understanding from their paper is, that it is even possible to go beyond retinal resolution. And some existing screens can already achieve that. When we compare those visions side by side, we can clearly see the difference. I remember feeling quite impressed when I moved on to the previous generations of VR headsets like HTC Vive or Rift to Quest 2 and cannot wait to see the virtual world from Butterscotch lenses.