Gibson Env: Real-World Perception for Embodied Agents

Gibson Env: Восприятие реального мира для воплощённых агентов
Jitendra Malik, Fei Xia, Amir Zamir, Zhiyang He, Alexander F. Sax, Silvio Savarese, Alexander Sax (Sasha)
2018-06-01

Gibson Environmentembodied agentslearning-in-simulationreal-world perceptionsim-to-real transfer
Developing visual perception models for active agents and sensorimotor control in the physical world are cumbersome as existing algorithms are too slow to efficiently learn in real-time and robots are fragile and costly. This has given rise to learning-in-simulation which consequently casts a question on whether the results transfer to real-world. In this paper, we investigate developing real-world perception for active agents, propose Gibson Environment for this purpose, and showcase a set of perceptual tasks learned therein. Gibson is based upon virtualizing real spaces, rather than artificially designed ones, and currently includes over 1400 floor spaces from 572 full buildings. The main characteristics of Gibson are: I. being from the real-world and reflecting its semantic complexity, II. having an internal synthesis mechanism "Goggles" enabling deploying the trained models in real-world without needing domain adaptation, III. embodiment of agents and making them subject to constraints of physics and space.
1
Gibson embodies agents under physical and spatial constraints and demonstrates a set of perceptual tasks learned within the environment.
2
Gibson virtualizes real spaces rather than artificially designed environments, comprising over 1,400 floor spaces across 572 full buildings.
3
Its internal synthesis mechanism, “Goggles,” is designed to support deploying trained models in the real world without domain adaptation.
4
The environment reflects real-world semantic complexity, enabling perception learning in settings more representative of physical deployment.
5
The paper introduces Gibson Environment, a simulation platform for developing real-world visual perception and sensorimotor models for embodied agents.

Gibson Environment: a virtualized collection of real-world indoor spaces for embodied agents

Real-world visual perception for active embodied agents, including perceptual task learning and transfer to physical deployment under physics and spatial constraints

Publication Details
Publication Date
2018-06-01
Journal
Publisher
ISSN
Cited by
771
Access Type
Author Information
Authors
Jitendra Malik
Fei Xia
Amir Zamir
Zhiyang He
Alexander F. Sax
Silvio Savarese
Alexander Sax (Sasha)
Explore further
Open the scid.ai AI chat with a ready-made request: it will find papers on a similar topic and help build a literature review.
Find similar papers in the chat →
Make a presentation
100%