Four Eyes Lab

Computational photography is an emerging research area that aims to extend or enhance the capabilities of digital photography by adding computational elements to the imaging process. Mobile phones have recently been gaining attention as a computing platform, due to advances in processing power and interactive displays, as well as enhanced programmability. Modern mobile devices are also equipped with good digital cameras. Their optical quality, resolution and photographic features have been quickly improving in recent years, making mobile phones a promising platform for computational photography.

The mobile scenario provides a unique and interesting platform for research and development in computational photography. Elements such as wireless network connectivity, high-resolution touchscreen, dedicated graphics processors, and presence of sensors such as accelerometers and magnetometers enable exciting applications. On the other hand, photography in this scenario poses additional challenges due to limitations of the photographic hardware when compared to high-end cameras, as well as the fact that photos are usually taken from a hand-held device, without resorting to the use of a tripod. Furthermore, traditional computational photography techniques can be computationally expensive, but in the mobile scenario it is often desired to immediately preview the capture result instead of saving the images for later off-line processing in a powerful computer. This motivates research on algorithms that make efficient use of the constrained computing resources to quickly preview the results, enabling users to recompose and recapture their photographs if desired while they are still present at the scene.

We are investigating novel techniques to address the challenges and to explore the possibilities of the mobile photography scenario. In particular, we are interested in:

• Mobile Epsilon Photography: Epsilon Photography broadly refers to techniques that capture multiple images of the scene while varying capture parameters by small ("epsilon") amounts, and then combine the images to generate a photograph that could not have been obtained through a single capture. Examples include panoramic stitching (varying point of view), high dynamic range (HDR) imaging (varying exposure time), and all-in-focus fusion (varying focusing distance).

• Photography in Motion Scenarios: motion is often present in mobile photography situations. The quality of the captured images is dramatically influenced by movement, which can be of two types: camera motion, and motion of objects in the scene.

• Novel Interactive Interfaces for Photography: the presence of a programmable touchscreen enables the creation of novel interfaces for photography that aid the user in the composition, capture and generation of their photographs.

Recent papers

Daniel Vaquero, Natasha Gelfand, Marius Tico, Kari Pulli, Matthew Turk.
Generalized Autofocus
IEEE WACV 2011

Andrew Adams, Eino-Ville Talvala, Sung Hee Park, David E. Jacobs, Boris Ajdin, Natasha Gelfand, Jennifer Dolson, Daniel Vaquero, Jongmin Baek, Marius Tico, Henrik P.A. Lensch, Wojciech Matusik, Kari Pulli, Mark Horowitz, Marc Levoy.
The Frankencamera: An Experimental Platform for Computational Photography
ACM SIGGRAPH 2010

Related projects

Camera 2.0 at Stanford University