TY - GEN
T1 - DotWarp
T2 - 16th Adjunct IEEE International Symposium on Mixed and Augmented Reality, ISMAR-Adjunct 2017
AU - Kim, Peter
AU - Orlosky, Jason
AU - Kiyokawa, Kiyoshi
AU - Ratsamee, Photchara
AU - Mashita, Tomohiro
N1 - Publisher Copyright:
© 2017 IEEE.
PY - 2017/10/27
Y1 - 2017/10/27
N2 - A significant issue associated with the use of video see-through head-mounted displays (VST-HMD) for augmented reality is the presence of latency between real-world images and the images displayed to the HMD. For a static scene, this latency provides no real problem, however for dynamic scenes, which arise when the HMD user moves their head, when real-world objects move, or a combination of the two, the accompanying delay may result in significant registration error. To address this issue, we present DotWarp, a novel latency reduction technique for VST-HMDs that does not rely on head motion and compensates for the delay arising from real-world object motion. The algorithm requires a two-camera setup and matches dynamic objects in both images by tracking on the faster image and warping the pixels of the slower image, with the fast and slow components being RGB and IR components, respectively, for our system. First, moving objects are extracted from the faster camera scene using a motioncompensating background subtraction algorithm and tracked using a robust correlation tracker. Then, temporal correspondence between the two camera images is computed using sensor update information and the objects' positions in the slower image are shifted to match the corresponding positions in the faster image. Finally, the gaps in the slower image left behind by the shifted objects are filled in with background pixel data from previous frames using homography from the background subtraction model. In this manner, the augmented image is more closely matched with the real-world image and the perceived registration of the camera is significantly improved, with initial results of an 81.64% reduction in registration error.
AB - A significant issue associated with the use of video see-through head-mounted displays (VST-HMD) for augmented reality is the presence of latency between real-world images and the images displayed to the HMD. For a static scene, this latency provides no real problem, however for dynamic scenes, which arise when the HMD user moves their head, when real-world objects move, or a combination of the two, the accompanying delay may result in significant registration error. To address this issue, we present DotWarp, a novel latency reduction technique for VST-HMDs that does not rely on head motion and compensates for the delay arising from real-world object motion. The algorithm requires a two-camera setup and matches dynamic objects in both images by tracking on the faster image and warping the pixels of the slower image, with the fast and slow components being RGB and IR components, respectively, for our system. First, moving objects are extracted from the faster camera scene using a motioncompensating background subtraction algorithm and tracked using a robust correlation tracker. Then, temporal correspondence between the two camera images is computed using sensor update information and the objects' positions in the slower image are shifted to match the corresponding positions in the faster image. Finally, the gaps in the slower image left behind by the shifted objects are filled in with background pixel data from previous frames using homography from the background subtraction model. In this manner, the augmented image is more closely matched with the real-world image and the perceived registration of the camera is significantly improved, with initial results of an 81.64% reduction in registration error.
KW - Augmented reality
KW - Delay compensation
KW - Timewarping
UR - http://www.scopus.com/inward/record.url?scp=85040222160&partnerID=8YFLogxK
UR - http://www.scopus.com/inward/citedby.url?scp=85040222160&partnerID=8YFLogxK
U2 - 10.1109/ISMAR-Adjunct.2017.61
DO - 10.1109/ISMAR-Adjunct.2017.61
M3 - Conference contribution
AN - SCOPUS:85040222160
T3 - Adjunct Proceedings of the 2017 IEEE International Symposium on Mixed and Augmented Reality, ISMAR-Adjunct 2017
SP - 184
EP - 185
BT - Adjunct Proceedings of the 2017 IEEE International Symposium on Mixed and Augmented Reality, ISMAR-Adjunct 2017
A2 - Broll, Wolfgang
A2 - Regenbrecht, Holger
A2 - Bruder, Gerd
A2 - Servieres, Myriam
A2 - Sugimoto, Maki
A2 - Swan, J Edward
PB - Institute of Electrical and Electronics Engineers Inc.
Y2 - 9 October 2017 through 13 October 2017
ER -