The Mixed Reality Simulator Project

Cha Lee, Tobias Höllerer, and Doug A. Bowman

Controllability and reproducibility are two of the main principles of the scientific method and yet traditional user experiments in the domain of Augmented Reality (AR) display systems are very difficult to control and repeat. Due to the many choices in hardware and software, it is generally the case that each display system is unique. This makes it difficult for researchers to repeat experiments performed by others. By using the traditional approach, a researcher would need the exact display system (or something very close to it) to repeat an experiment. This can be costly and impractical. Mixed Reality (MR) Simulation is a practical approach to conducting controlled and repeatable experiments. MR Simulation is the concept of using a high-fidelity Virtual Reality display system to simulate a range of lower-fidelity display systems. We can simulate a variety of display systems used by other researchers by simulating the level of fidelity of said system, making it practical for reproducing previous experiments. This approach also offers more controllability when we conduct user experiments in a simulated display system. By simulating the hardware, the software, and even the environment, we can control all factors in an Augmented Reality experiment which is often impossible with real-world setups. An added benefit of using MR Simulation is the ability of evaluating non-existing or hard to obtain technologies by simply simulating the technology. This project investigates the validity of this approach. Through a series of experiments we will empirically show that for a representative set of tasks and conditions, it is both practical and valid to run AR user experiments using MR Simulation.

Published Work:

Examining the Equivalence of Simulated and Real AR on a Visual Following and Identification Task.

Poster at IEEE VR 2012.


Evaluating the Impact of Recovery Density on Augmented Reality Tracking.





The Role of Latency in the Validity of AR Simulation.

IEEE VR 2010.


A Replication Study Testing the Validity of AR Simulation in VR for Controlled Experiments.

Poster at IEEE/ACM ISMAR 2009.



An Immaterial Depth-Fused 3D Display.

ACM VRST 2007. Best Student Paper.