Simultaneous Localization and Appearance Estimation with a
Consumer RGB-D Camera
Hongzhi Wu Zhaotian Wang Kun Zhou
IEEE Transactions. on Visualization and Computer Graphics, 22, 8(Aug. 2016), 2012-2023.
Hongzhi Wu Zhaotian Wang Kun Zhou
IEEE Transactions. on Visualization and Computer Graphics, 22, 8(Aug. 2016), 2012-2023.
Abstract
Acquiring general material appearance with hand-held consumer RGB-D cameras is difficult for casual users, due to the inaccuracy in reconstructed camera poses and geometry, as well as the unknown lighting that is coupled with materials in measured color images. To tackle these challenges, we present a novel technique, called Simultaneous Localization and Appearance Estimation (SLAE), for estimating the spatially varying isotropic surface reflectance, solely from color and depth images captured with an RGB-D camera under unknown environment illumination. The core of our approach is a joint optimization, which alternates among solving for plausible camera poses, materials, the environment lighting and normals. To refine camera poses, we exploit the rich spatial and view-dependent variations of materials, treating the object as a localization-self-calibrating model. To recover the unknown lighting, measured color images along with the current estimate of materials are used in a global optimization, efficiently solved by exploiting the sparsity in the wavelet domain. We demonstrate the substantially improved quality of estimated appearance on a variety of daily objects.
Downloads
Paper(.PDF) [Hi-res, 22.2MB] [Low-res, 1.5MB] [IEEE Xplore (OpenAccess)]
Bibtex [.TXT]
Video [Vimeo HD] [Youtube] [.MP4, 29.4MB]
Presentation Slides [.PPSX, 85.2MB]
Executables and Data
Here we release the binary executables and related data to compute the xmas-ball example in the paper, to help reproduce our result as well as stimulate further research along the direction of casual material appearance acquisition. Please email the first author if there is any problem.
Xmas-ball Dataset [.7z, 1.47GB]
Due to the large size of the file, it is recommended to use a download manager such as FlashGet to get the file.