An image inpainting method based on exemplarbased image restoration technology was proposed to improve the quality of hole filling in view synthesis by using depth information. This method fills the holes and provides a synthesized view through depth map preprocessing, image warping, depth inpainting and color image inpainting. The final synthesized view quality is improved by the depth edge’s unidirectional dilation, the strategy that firstly inpaintes synthesized depth map and then the synthesized image, and the preediting of matched patch. The experimental results show that this method is superior to the existing methods in terms of subjective visual perception and objective index, such as peak signal to noise ratio (PSNR) and time consuming. The proposed method also has outstanding effects on maintaining the foreground edges and eliminating the infiltration from foreground to background．
［1］ SHUM H Y, KANG S B. A review of imagebased rendering techniques ［C］ ∥ Proceedings of SPIE 4067, Visual Communications and Image Processing 2000. Perth:SPIE, 2000: 2-13．
［2］ FEHN C. Depthimagebased rendering (DIBR), compression, and transmission for a new approach on 3DTV ［C］ ∥ Proceedings of SPIE 5291,Stereoscopic Displays and Virtual Reality Systems XI. San Jose: SPIE, 2004: 93-104．
［3］ SHADE J, GORTLER S, HE L, et al. Layered depth images ［C］ ∥ Proceedings of the 25th Annual Conference on Computer Graphics and Interactive Techniques. Orlando: ACM, 1998: 231-242．
［4］ 陈坤斌,刘海旭,李学明. 构造全局背景的虚拟视点合成算法［J］. 信号处理,2013, 29(10): 1307-1314．
CHEN Kunbin, LIU Haixu, LI Xueming. Virtual view synthesis using generated global background ［J］. Journal of Signal Processing, 2013, 29(10): 1307-1314．
［5］ YAO C, TILLO T, ZHAO Y, et al. Depth map driven hole filling algorithm exploiting temporal correlation information ［J］. IEEE Transactions on Broadcasting, 2014, 60(2): 394-404．
［6］ 王奎,安平,张兆杨, 等. Kinect深度图像快速修复算法［J］.上海大学学报：自然科学版,2012,18(5): 454-458．
WANG Kui, AN Ping, ZHANG Zhaoyang, et al. Fast inpainting algorithm for kinect depth map ［J］. Journal of Shanghai University：Natural Science Edition,2012,18(5):454-458．
［7］ ZHANG L, TAM W J. Stereoscopic image generation based on depth images for 3DTV ［J］. IEEE Transactions on Broadcasting, 2005, 51(2): 191-199．
［8］ LEE P J, EFFENDI. Nongeometric distortion smoothing approach for depth map preprocessing ［J］. IEEE Transactions on Multimedia, 2011, 13(2): 246-254．
［9］ 高凯,陈贺新,赵岩,等. 面向虚拟视点绘制的深度图滤波及上采样方法［J］. 中国图象图形学报,2013,18(9):1085-1092．
GAO Kai, CHEN Hexin, ZHAO Yan, et al. Depth map filtering and upsampling method for virtual view rendering ［J］. Journal of Image and Graphics, 2013,18(9): 1085-1092．
［10］ 李应彬, 冯杰, 张华熊,等. 基于改进双边滤波的Kinect深度图像空洞修复算法研究［J］. 工业控制计算机, 2013, 26(11):105-106,109．
LI Yingbing, FENG Jie, ZHANG Huaxiong, et al. New algorithm of depth hole filling based on intensive bilateral filter ［J］. Industrial Control Computer, 2013, 26(11): 105-106,109．
［11］ DARIBO I, SAITO H. Depthaided image inpainting for novel view synthesis ［C］ ∥ Proceedings of 2010 IEEE International Workshop on Multimedia Signal Processing (MMSP). (S.l.): IEEE, 2010: 167-170．
［12］ GAUTIER J, LE MEUR O, GUILLEMOT C. Depth based image completion for view synthesis ［C］ ∥ 3DTV Conference: the True VisionCapture,Transmission and Display of 3D Video (3DTVCON).Antalya:IEEE,2011:14．
［13］ MA L N, DO L, WITH D, et al. Depthguided inpainting algorithm for freeviewpoint video ［C］ ∥ 2012 19th IEEE International Conferenceon on Image Processing (ICIP). Orlando： IEEE, 2012:1721-1724．
［14］ WU H Y, FENG J, ZHANG H X, et al. A virtual view synthesis algorithm based on image inpainting ［C］ ∥ Third International Conference on Networking and Distributed Computing. Hangzhou: IEEE, 2012:153-156．
［15］ XU X Y, PO L M, Cheung C H, et al. Depthaided exemplarbased hole filling for DIBR view synthesis ［C］∥ 2013 IEEE International Symposium on Circuits and Systems (ISCAS). Beijing: IEEE, 2013:2840-2843．
［16］ ZITNICK C L, SING B K, MATTHEW U, et al. High quality video view interpolation using a layered representation ［J］. ACM Transactions on Graphics. 2004, 23(3): 600608．
［17］ SCHARSTEIN D. Stereo vision for view synthesis ［C］ ∥ IEEE Computer Society Conference on Computer Vision and Pattern Recognition(CVPR’96). San Francisco: IEEE,1996: 852-858.
［18］ TELEA A. An image inpainting technique based on the fast marching method ［J］. Journal of Graphics Tools, 2004, 9(1): 2334．
［19］ CRIMINISI A, PEREZ P, TOYAMA K, Region filling and object removal by exemplarbased image inpainting ［J］. IEEE Transactions on Image Processing, 2004, 13(9): 1200-1212．
［20］ PEREZ P, GANGNET M, AND BLAKE A. Poisson image editing ［J］. Proceedings of the SIGGRAPH, 2003, 22(3): 313-318．
［21］ SCHARSTEIN D, PAL C. Learning conditional random fields for stereo ［C］ ∥ IEEE Computer Society Conference on Computer Vision and Pattern Recognition (CVPR 2007). Minneapolis: IEEE, 2007: 18．
［22］ XU X Y, PO L M, NG K H, et al. Depth map misalignment correction and dilation for DIBR view synthesis ［J］. Signal Processing: Image Communication. 2013, 28(9): 1023-1045．
［23］ TIAN D, LAI P L, PATRICK L. View synthesis techniques for 3D video ［C］ ∥ SPIE Optical Engineering and Applications. San Diego:SPIE, 2009: 74430T-74430T-11．
YANG Bing, WANG Xiao-hua, YANG Xin, HUANG Xiao-xi. Face recognition method based on HOG pyramid[J]. JOURNAL OF ZHEJIANG UNIVERSITY (ENGINEERING SCIENCE), 2014, 48(9): 1564-1569.
