Image Completion with Intrinsic Reflectance Guidance

by Soomin Kim , Taeyoung Kim, Min H. Kim, and Sung-Eui Yoon

(To appear) British Machine Vision Conference (BMVC 2017)

This figure shows the basic intrinsic image property we utilize for image completion. Image (a) shows an input image, where the blanket is removed (masked as red). Image (b) presents correspondences of source patches using color and Laplacian properties. Image (c) visualizes our correspondence search using intrinsic reflectance. Note that our method brings source patches from virtually similar materials.


Our method inpaints the input image by separating intrinsic reflectance and shading so that our method can complete the image with intrinsic scene materials, rather than shadowed color values. Other state-of-the-art methods inpaint the blank region using color values directly, resulting in unrealistic artifacts.


Abstract

Patch-based image completion methods often fail in searching patch correspondences of similar materials due to shading caused by scene illumination, resulting in inappropriate image completion with dissimilar materials. We therefore present a novel image completion method that additionally accounts for intrinsic reflectance of scene objects, when searching patch correspondences. Our method examines both intrinsic reflectances and color image structures to avoid false correspondences of different materials so that our method can search and vote illumination-invariant patches robustly, allowing for image completion mainly with homogeneous materials. Our results validate that our reflectance-guided inpainting can produce more natural and consistent images than state-of-the-art inpainting methods even under various illumination conditions.

Contents

Paper (author preprint): PDF
Supplementary material

The code will be available soon!!

School of Computing
KAIST
291 Daehak-ro, Yuseong-gu, Daejeon, 34341
South Korea
sglabkaist dot gmail dot com