We’re not expecting to see Apple Glasses before 2022 at the earliest, but we’ve now seen a prototype of one rival effort: Facebook AR glasses.
Interestingly, this prototype has no display and is entirely focused on an auditory experience. It’s part super-sophisticated hearing aid, part virtual reality device …
The Facebook Reality Labs Research team revealed the prototype, which is likely just one element of an eventual commercial product.
Engadget explains.
Facebook explains that the same approach can be taken to accurately simulate real-world sounds in things like virtual reality experiences.
Members of the Facebook Reality Labs (FRL) Research audio team detail their work on two technologies in particular. One of the, which the team is calling “Perceptual Superpowers,” will allow you to enhance certain sounds and dim background noise in the real world. It will, for instance, give you a way clearly hear someone sitting across you in a noisy restaurant.
The technology can do that by using the glasses’ multiple microphones to capture sounds around you. It will then take note of your head and eye movements to figure out the sounds you actually want to hear and the ones you want filter out. While the researchers aren’t specifically developing the technology for the hard of hearing, the glasses could help those who’d rather not use hearing aids in social situations.
Thomas Lunner, one of the researchers who also worked on the world’s first digital hearing aid, said: “By putting hearing impaired people on par with people with normal hearing, we could help them become more socially engaged. This resonates very well with Facebook’s mission in the sense that hearing loss often keeps people away from social situations.”
The FRL Research audio team is also developing “Audio Presence,” which will recreate how sounds travel and bounce around in the real world for AR and VR. It would make audio from the glasses sound realistic, like you’re actually listening to what’s happening around you instead of something that’s coming from a device.
(Incidentally, Apple also has a patent for a method of identifying where sound comes from, which could also be used in real or virtual worlds.)
When someone speaks to you in a room, one of your ears hears the sound before the other. The volume is also different in each ear. On top of that, the shape of the ear changes how each of us hears sound ever so slightly. All of these signals tell your brain where a sound comes from. And sound interacts with your environment, bouncing off the walls before making it to your ear. These are the core components that, if reproduced accurately, let virtual sounds replicate real ones.
In 2017, the audio research team helped ship spatialized audio — virtual sounds that mimic the directions sounds come from in real life. It also invented high-quality acoustic simulation technologies that make virtual environments even more believable. These technologies pushed forward the state of the art of spatial audio and powers many of today’s experiences on Oculus Quest and the Rift Platform, including First Steps and Oculus First Contact. The next frontier is personalizing spatial audio and modeling how sound interacts with real environments.
Facebook says the same tech can also effectively give human beings the same ability as HomePods: hearing a quietly spoken sentence while loud music is playing. HomePods use a combination of microphones and speakers to allow Siri to filter out the music that is playing in order to hear your voice. Facebook has demonstrated how its prototype device can allow us to do the same thing.
It’s certainly interesting stuff, and the fact that Facebook has chosen to reveal this work suggests it does intend these capabilities to make it to market at some point. The whole field of AR and VR glasses and headsets is certainly going to be a fascinating one over the next few years.
A few physical speakers in the room play music at full volume. I’m in another room where Tourbabin calls me. I pick up, and he begins to read an online article in a normal speaking voice, which is easily drowned out by the noise in his cacophonous room.
Tourbabin then flips a switch — and I can suddenly hear his voice coming through, clear as day. It’s as if someone’s turned down the volume of the background noise so I can easily focus in on what he’s trying to say. It’s like getting a call from a friend at a rock concert or in a subway station, yet somehow I can hear his voice clearly and understandably.
