Through the collaboration, the companies hope to enable researchers to advance Habitat - Facebook AI's simulation platform for training embodied agents (virtual robots) - to help robots better understand and interact with the physical world. The resulting Habitat-Matterport 3D Research Dataset (HM3D) is a collection of 1,000 of Matterport's high-resolution digital twins made up of residential, commercial, and civic spaces generated precisely from real-world environments.
The companies say they expect HM3D to play a significant role in advancing embodied AI research, which seeks to teach robots and virtual AI assistants to understand and interact with the complexities of the physical world.
Dhruv Batra, Research Scientist at Facebook AI Research says, "Until now, this rich spatial data has been glaringly absent in the field, so HM3D has the potential to change the landscape of embodied AI and 3D computer vision. Our hope is that the 3D dataset brings researchers closer to building intelligent machines - to do for embodied AI what pioneers before us did for 2D computer vision and other areas of AI."
HM3D is free and available now for academic, non-commercial research. Researchers can use it with FAIR's Habitat simulator to train embodied agents, such as home robots and AI assistants, at scale. HM3D, say the companies, is a foundational step towards helping these agents navigate through real-world environments and better understand the variations of spaces such as bedrooms, bathrooms, kitchen and hallways, as well as the different configurations of those rooms within every structure.
It can also assist robots in recognizing how objects within rooms are typically arranged so that instructions are correctly understood. This research could one day be used in production applications like robots that can retrieve medicine from a bedroom nightstand or AR glasses that can help people remember where they left their keys.
Conway Chen, Vice President of Business Development and Alliances at Matterport says, "We are excited to collaborate with Facebook as we