Rainbow Learner: Lighting Environment Estimation from a Structural-color based AR Marker

Yuji Tsukagoshi, Yuki Uranishi, Jason Orlosky, Kiyomi Ito, Haruo Takemura

Research output: Chapter in Book/Report/Conference proceedingConference contribution

Abstract

This paper proposes a method for estimating lighting environments from an AR marker coupled with the structural color patterns inherent to a compact disc (CD) form-factor. To achieve photometric consistency, these patterns are used as input to a Conditional Generative Adversarial Network (CGAN), which allows us to efficiently and quickly generate estimations of an environment map. We construct a dataset from pairs of images of the structural color pattern and environment map captured in multiple scenes, and the CGAN is then trained with this dataset. Experiments show that we can generate visually accurate reconstructions with this method for certain scenes, and that the environment map can be estimated in real time.

Original languageEnglish (US)
Title of host publicationProceedings - 2020 IEEE International Conference on Artificial Intelligence and Virtual Reality, AIVR 2020
PublisherInstitute of Electrical and Electronics Engineers Inc.
Pages362-365
Number of pages4
ISBN (Electronic)9781728174631
DOIs
StatePublished - Dec 2020
Externally publishedYes
Event3rd IEEE International Conference on Artificial Intelligence and Virtual Reality, AIVR 2020 - Virtual, Utrecht, Netherlands
Duration: Dec 14 2020Dec 18 2020

Publication series

NameProceedings - 2020 IEEE International Conference on Artificial Intelligence and Virtual Reality, AIVR 2020

Conference

Conference3rd IEEE International Conference on Artificial Intelligence and Virtual Reality, AIVR 2020
Country/TerritoryNetherlands
CityVirtual, Utrecht
Period12/14/2012/18/20

Keywords

  • augmented reality
  • lighting estimation
  • neural network

ASJC Scopus subject areas

  • Artificial Intelligence
  • Computer Science Applications
  • Media Technology

Fingerprint

Dive into the research topics of 'Rainbow Learner: Lighting Environment Estimation from a Structural-color based AR Marker'. Together they form a unique fingerprint.

Cite this