Shadow Simulation in Mixed Reality

Young Wn Song
Tobias Höllerer

Abstract

In this work we introduce a testbed for MR shadow simulation. We filmed a simple physical scene, in which an occluder object is moved relative to a light source, casting a shadow onto a desktop environment. The moving object is tracked using ARToolkit [1] markers. We filmed several versions of the video, with and without the physical occluder, considering different light source strengths, positions, and receiver geometry. Real shadows from moving objects are compared with virtual shadows from synthesized objects in the same scene so that we can assess the realism of the virtual shadows. We are making the videos and tracking information available to the research community to serve as a benchmark test for new shadow generation algorithms. Results of new algorithms (and their parameter settings) can quickly be explored and compared with the physical shadows.

Simulation Overview

There are two cameras. The first camera is located on the ceiling to track the coordinates of the table and the moving object. The second camera is located in the foreground in order to gain a close-up view of the shadows created. Two light sources are used at two different positions. One position is used to cast a shadow onto the table, and the other position is used to cast a shadow onto the wall. The box is located on the table, adding an extra degree of geometrical complication. The cylinder is suspended in the air and operated by strings so that we can cast moving shadows. Our testbed consists of three sets of data:

  1. physical videos with a moving object, which is used for tracking the coordinates of the moving object
  2. background videos without the moving object, which is used for reproducing the virtual objects and the virtual shadows
  3. Tracking data, which represents the coordinate systems of the table and moving cylinder.

We provide different simulation parameters:

  1. Two light sources: one is a halogen linear light with 300W and the other is a spherical light source with 150W.
  2. Two different light positions: one is at the center of the table to cast shadows only onto the table and the other is position is intended for casting shadows onto the wall.
  3. With or without an extra geometrical object: we use a box to represent an extra geometrical object.
  4. An object moves along a single axis in one direction or two different directions.

Object Tracking

Figure 1 Figure 2 Figure 3
We use ARToolkit [1] to track the coordinate systems of the table and the moving object. Figure 1 shows the origin of the table and Figure 2 shows the origin of the cylinder. We calculate a relative position of the cyliner with respect to the table at each frame. Figure 3 shows the virtual object and the virtual shadow reproduced into the background image.

Synthesize Virtual Shadows into Physical Scenes

First we model real shadow receivers by virtual objects that are invisible for the users as in [2]. In our simulations, the invisible objects are the table, the box, and the wall. All these geometric data including light positions can be found here. A triangle mesh file for the cylinder used in our simulation can be found here. To reproduce the virtual object and the virtual shadow for a given viewpoint (overhead view or front view), we need a coordinate system (Mt) for the fixed table and a relative position with respect to the table (Mr). For each frame we render the virtual object into Mt*Mr and cast the shadow. The transformations of the moving object are mostly translations with respect to the table. There are some rotations but we saved only translations for better tracking results. The data files for the table contain 3X4 modelview matrices in row major order. The data files for the moving object contains translations with respect to the table.

Code Example

Available Soon.

Simulation Data Using a Spherical Light Source


Simulation Data Using a Halogen Linear Light Source

Publications

Available Soon.

References

[1] ARToolkit, http://www.hitl.washington.edu/artoolkit/
[2] Michael Haller , Stephan Drab , Werner Hartmann, A real-time shadow approach for an augmented reality application using shadow volumes, Proceedings of the ACM symposium on Virtual reality software and technology, October 01-03, 2003, Osaka, Japan