Please wait a minute...
Journal of Zhejiang University-SCIENCE A (Applied Physics & Engineering)  2009, Vol. 10 Issue (12): 1738-1749    DOI: 10.1631/jzus.A0820806
Computer Science and Technology     
Depth-aided inpainting for disocclusion restoration of multi-view images using depth-image-based rendering
Kai LUO, Dong-xiao LI, Ya-mei FENG, Ming ZHANG
Department of Information Science and Electronic Engineering, Zhejiang University, Hangzhou 310027, China
Download:     PDF (0 KB)     
Export: BibTeX | EndNote (RIS)      

Abstract  A new algorithm is proposed for restoring disocclusion regions in depth-image-based rendering (DIBR) warped images. Current solutions include layered depth image (LDI), pre-filtering methods, and post-processing methods. The LDI is complicated, and pre-filtering of depth images causes noticeable geometrical distortions in cases of large baseline warping. This paper presents a depth-aided inpainting method which inherits merits from Criminisi’s inpainting algorithm. The proposed method features incorporation of a depth cue into texture estimation. The algorithm efficiently handles depth ambiguity by penalizing larger Lagrange multipliers of filling points closer to the warping position compared with the surrounding existing points. We perform morphological operations on depth images to accelerate the algorithm convergence, and adopt a luma-first strategy to adapt to various color sampling formats. Experiments on test multi-view sequence showed that our method has superiority in depth differentiation and geometrical loyalty in the restoration of warped images. Also, peak signal-to-noise ratio (PSNR) statistics on non-hole regions and whole image comparisons both compare favorably to those obtained by state of the art techniques.

Key wordsDepth-aided inpainting      Disocclusion restoration      Depth-image-based rendering (DIBR)      Image warping      Stereoscopic image      Multi-view image      3D-TV     
Received: 20 November 2008     
CLC:  TP391.41  
Cite this article:

Kai LUO, Dong-xiao LI, Ya-mei FENG, Ming ZHANG. Depth-aided inpainting for disocclusion restoration of multi-view images using depth-image-based rendering. Journal of Zhejiang University-SCIENCE A (Applied Physics & Engineering), 2009, 10(12): 1738-1749.

URL:

http://www.zjujournals.com/xueshu/zjus-a/10.1631/jzus.A0820806     OR     http://www.zjujournals.com/xueshu/zjus-a/Y2009/V10/I12/1738

[1] Sheng-yang YU, Fang-lin WANG, Yun-feng XUE, Jie YANG. Bayesian moving object detection in dynamic scenes using an adaptive foreground model[J]. Journal of Zhejiang University-SCIENCE A (Applied Physics & Engineering), 2009, 10(12): 1750-1758.
[2] Edgar SCAVINO, Dzuraidah Abdul WAHAB, Aini HUSSAIN, Hassan BASRI, Mohd Marzuki MUSTAFA. Application of automated image analysis to the identification and extraction of recyclable plastic bottles[J]. Journal of Zhejiang University-SCIENCE A (Applied Physics & Engineering), 2009, 10(6): 794-799.
[3] Yue-ting ZHUANG, Yu-shun WANG, Timothy K. SHIH, Nick C. TANG. Patch-guided facial image inpainting by shape propagation[J]. Journal of Zhejiang University-SCIENCE A (Applied Physics & Engineering), 2009, 10(2): 232-238.
[4] Miguel Ángel SOTELO, José BARRIGA. Blind spot detection using vision for automotive applications[J]. Journal of Zhejiang University-SCIENCE A (Applied Physics & Engineering), 2008, 9(10): 1369-1372.
[5] Xiang PAN, Yi-jun WU. GSM-MRF based classification approach for real-time moving object detection[J]. Journal of Zhejiang University-SCIENCE A (Applied Physics & Engineering), 2008, 9(2): 250-255.
[6] ZHU Le-qing, ZHANG San-yuan, YE Xiu-zi. Implementing VLPR systems based on TMS320DM642[J]. Journal of Zhejiang University-SCIENCE A (Applied Physics & Engineering), 2007, 8(12): 2005-2016.
[7] GUI Jiang-sheng, RAO Xiu-qin, YING Yi-bin. Fruit shape detection by level set[J]. Journal of Zhejiang University-SCIENCE A (Applied Physics & Engineering), 2007, 8(8): 1232-1236.
[8] HU Cheng-cheng, YE Xiu-zi, ZHANG Yin, YU Rong-dong, YANG Jian, ZHU Jun. 3D graphical visualization of the genetic architectures underlying complex traits in multiple environments[J]. Journal of Zhejiang University-SCIENCE A (Applied Physics & Engineering), 2007, 8(4): 563-567.
[9] JIANG Ren-jie, QI Fei-hu, XU Li, WU Guo-rong, ZHU Kai-hua. A learning-based method to detect and segment text from scene images[J]. Journal of Zhejiang University-SCIENCE A (Applied Physics & Engineering), 2007, 8(4): 568-574.
[10] ZHU Kai-hua, QI Fei-hu, JIANG Ren-jie, XU Li. Automatic character detection and segmentation in natural scene images[J]. Journal of Zhejiang University-SCIENCE A (Applied Physics & Engineering), 2007, 8(1 ): 10-.
[11] QIN Ai-hong, XIONG Hua, PENG Hao-yu, LIU Zhen, SHI Jiao-ying. Cluster parallel rendering based on encoded mesh[J]. Journal of Zhejiang University-SCIENCE A (Applied Physics & Engineering), 2006, 7(7): 1124-1133.
[12] ZHU En, ZHANG Jian-ming, YIN Jian-ping, ZHANG Guo-min, HU Chun-feng. Removing the remaining ridges in fingerprint segmentation[J]. Journal of Zhejiang University-SCIENCE A (Applied Physics & Engineering), 2006, 7(6 ): 7-.
[13] GUAN Ye-peng, GU Wei-kang. A matching algorithm based on hybrid matrices consisting of reference differences and disparities[J]. Journal of Zhejiang University-SCIENCE A (Applied Physics & Engineering), 2004, 5(7): 796-802.
[14] WANG Tao, BU Jia-Jun, CHEN Chun. A color based face detection system using multiple templates[J]. Journal of Zhejiang University-SCIENCE A (Applied Physics & Engineering), 2003, 4(2): 162-165.
[15] WU Dong-hui, YE Xiu-qing, GU Wei-kang. A closed-loop algorithm to detect human face using color and reinforcement learning[J]. Journal of Zhejiang University-SCIENCE A (Applied Physics & Engineering), 2002, 3(1): 72-76.