Panoramic view reconstruction for stereoscopic teleoperation of a humanoid robot

Konstantinos Theofilis, Jason Orlosky, Yukie Nagai, Kiyoshi Kiyokawa

Research output: Chapter in Book/Report/Conference proceedingConference contribution

12 Scopus citations


In recent years, robots have become more commonplace as a method for humans to remotely conduct observations or interactions, with commercial applications ranging from customer service to assembly line work. Virtual and augmented reality have shown much promise as methods to visualize and assist with teleoperation of these robots. However, a number of problems still exist with such bidirectional teleoperator relationships, including the intersensory conflict induced by delay of images received on the teleoperator side of a remote session. This can result in nausea, simulation sickness, and unnatural interaction, especially when stereoscopic cameras are present. As a step towards addressing this problem, we introduce the use of stereo panoramic reconstruction in conjunction with an augmented reality interface to improve both the interaction and the sense of embodiment of a teleoperator working through a robot's perspective. Unlike current methods that send latent streams of the robot's eye-cameras to the user, we view reconstruction that also incorporates direct control of the robot's head. This algorithm uses queued time and angle stamped images to reconstruct the robot's stereoscopic view for the teleoperator, which allows for low-latency viewing and field of view expansion. We then conduct experiments comparing our method against a direct camera feed from the robot's eyes. In addition to measuring latency at every point of the data pipeline, we show that this panoramic representation reduces perceived visual delay and elicits positive user feedback.3

Original languageEnglish (US)
Title of host publicationHumanoids 2016 - IEEE-RAS International Conference on Humanoid Robots
PublisherIEEE Computer Society
Number of pages7
ISBN (Electronic)9781509047185
StatePublished - Dec 30 2016
Externally publishedYes
Event16th IEEE-RAS International Conference on Humanoid Robots, Humanoids 2016 - Cancun, Mexico
Duration: Nov 15 2016Nov 17 2016

Publication series

NameIEEE-RAS International Conference on Humanoid Robots
ISSN (Print)2164-0572
ISSN (Electronic)2164-0580


Conference16th IEEE-RAS International Conference on Humanoid Robots, Humanoids 2016

ASJC Scopus subject areas

  • Artificial Intelligence
  • Computer Vision and Pattern Recognition
  • Hardware and Architecture
  • Human-Computer Interaction
  • Electrical and Electronic Engineering


Dive into the research topics of 'Panoramic view reconstruction for stereoscopic teleoperation of a humanoid robot'. Together they form a unique fingerprint.

Cite this