Enveloping users and computers in a collaborative 3D augmented reality

We present EMMIE (Environment Management for Multiuser Information Environments), a prototype experimental user interface to a collaborative augmented environment. Users share a 3D virtual space and manipulate virtual objects that represent information to be discussed. We refer to EMMIE as a hybrid user interface because it combines a variety of different technologies and techniques, including virtual elements such as 3D widgets, and physical objects such as tracked displays and input devices. See-through head-worn displays overlay the virtual environment on the physical environment, visualizing the pervasive "virtual ether" within which all interaction occurs. Our prototype includes additional 2D and 3D displays, ranging from palm-sized to wall-sized, allowing the most appropriate one to be used for any task. Objects can be moved among displays (including across dimensionalities) through drag-and-drop. In analogy to 2D window managers, we describe a prototype implementation of a shared 3D environment manager that is distributed across displays, machines, and operating systems. We also discuss two methods we are exploring for handling information privacy in such an environment.