Home

TranslatAR: A Mobile Augmented Reality Translator

TranslatAR: A Mobile Augmented Reality Translator

Victor Fragoso, Steffen Gauglitz, Shane Zamora, Jim Kleban, Matthew Turk
Marc Petter, Charles Baur (EPFL)

IEEE Workshop on Applications of Computer Vision (WACV'11)
Kona, Hawaii, 2011

TranslatAR_1

We present a mobile augmented reality (AR) translation system, using a smartphone’s camera and touchscreen, that requires the user to simply tap on the word of interest once in order to produce a translation, presented as an AR overlay. The translation seamlessly replaces the original text in the live camera stream, matching background and foreground colors estimated from the source images. For this purpose, we developed an efficient algorithm for accurately  detecting the location and orientation of the text in a live camera stream that is robust to perspective distortion, and we combine it with OCR and a text-to-text translation engine.

Related Publications

TranslatAR: A Mobile Augmented Reality Translator. Victor Fragoso, Steffen Gauglitz, Shane Zamora, Jim Kleban, Matthew Turk. IEEE Workshop on Applications of Computer Vision (WACV 2011). [pdf]

Citation

@InProceedings{TranslatAR_WACV2011,
author = {Victor Fragoso and Steffen Gauglitz and Shane Zamora and Jim Kleban and Matthew Turk},
title = {TranslatAR: A Mobile Augmented Reality Translator},
booktitle = {IEEE Workshop on Applications of Computer Vision (WACV'11)},
address = {Kona, Hawaii},
month = {January},
year = {2011}
}

Automatic text detection for mobile augmented reality translation. Marc Petter, Victor Fragoso, Matthew Turk, Charles Baur, IEEE International Workshop on Mobile Vision (IWMV 2011).

Citation

@InProceedings{Petter_IWMV2011,
author = {Marc Petter and Victor Fragoso and Matthew Turk and Charles Baur},
title = {Automatic text detection for mobile augmented reality translation},
booktitle = {IEEE International Workshop on Mobile Vision (IWMV'11)},
address = {Barcelona, Spain},
month = {November},
year = {2011}
}

Video: [Demo]
Related Links

[Project Page]