Quest 3 Firmware Clip Shows Depth Sensor 3D Room Meshing

A clip found in Meta's firmware visualizes Quest 3 using its depth sensor to generate a detailed 3D mesh of its environment.

Meta has hinted that Quest 3 is capable of "intelligently understanding and responding to objects in your physical space", but hasn't yet gone into detail about what that actually means.

The short clip, found by firmware sleuth Samulia, visualizes depth mapping coming from the headset - likely representing the depth sensor - as it pans around the room. A detailed environment mesh is visible on furniture and objects.

The clip also appears to show semantic understanding, with the couch being labelled with a couch icon.

It ends by showing a mixed reality character standing on the real couch and then climbing over it, an example of occlusion, all while casting a realistic shadow across the scene. Without occlusion the character would be rendered in front and would always break the illusion. Occlusion is arguably as important to mixed reality as positional tracking is to VR.

Current Quest headsets can achieve basic occlusion if you manually mark out rectangular cuboids to represent your walls and furniture. This arduous process means almost no Quest developer today actually leverages the scene graph features though. Quest Pro was originally supposed to have a depth sensor, but it was dropped just a few months before release. Had it not been, it's possible Quest Pro could have had these same room scanning features Quest 3 now seems set to introduce.

Other clips found in the firmware by Samulia back in June suggest the room mesh won't only be used for mixed reality though.

To use VR on existing Quest headsets you manually draw Guardian bounds on your floor via a passthrough view of your room. An eight foot boundary wall will then show when you get near those bounds. But this limits you to using VR only within completely empty spaces with no furniture of any height, even if you could reach over it.

A Meta product roadmap leaked to The Verge in March mentioned Quest 3 having a feature called “Smart Guardian”. One of the previously leaked clips appeared to show this - the user in VR can see an outline of nearby furniture and is made aware when approaching real walls. Compared to traditional Guardian you could leverage more of your room for your hands in VR, including the space above beds and tables.

Meta hasn't yet revealed all of Quest 3's specs and features. What it has officially stated is the visor is 40% thinner thanks to pancake lenses, the GPU is more than twice as powerful, and it has dual color cameras and a depth sensor for mixed reality with scene understanding. The base model has 128GB storage and will be priced at $500.

Meta hasn't yet revealed the exact launch date, but the company opened preorders for Quest 2 and Quest Pro at its yearly Connect conference and this year it's scheduled for September 27.

For a summary of everything announced and discovered about Quest 3 so far, read our full specs & features roundup here.