Please wait a minute...
浙江大学学报(工学版)
计算机技术、信息电子     
视点合成中基于深度的空洞修复
杜歆,邹泷
浙江大学 信息与电子工程学系, 浙江 杭州 310027
Depth based hole filling in view synthesis
DU Xin, ZOU Shuang
Department of Information Science and Electronic Engineering, Zhejiang University, Hangzhou 310027, China
 全文: PDF(2731 KB)   HTML
摘要:

为提高视点合成中空洞的修复质量,基于样本图像修复技术,提出利用深度信息进行修复的方法.通过深度图预处理、图像映射变换、合成深度图修复以及合成视图修复等步骤,实现对空洞区域的填充,得到虚拟视点视图.所提出的深度边缘单向膨胀预处理方法、先修复合成深度图再修复合成视图的策略以及匹配块的预编辑方法,可以提高最终合成视图的质量.实验结果表明:所提出方法在主观视觉对比、峰值信噪比(PSNR)以及运行耗时上优于现有方法,在图像边缘连续性的保持、前景和背景像素渗透现象的消除等方面具有较好的效果.

Abstract:

An image inpainting method based on exemplarbased image restoration technology was proposed to improve the quality of hole filling in view synthesis by using depth information. This method fills the holes and provides a synthesized view through depth map preprocessing, image warping, depth inpainting and color image inpainting. The  final synthesized view quality is improved by the  depth edge’s unidirectional dilation, the strategy that firstly inpaintes synthesized depth map and then the synthesized image, and the preediting of matched patch. The experimental results show that this method is superior to the existing methods in terms of subjective visual perception and objective index, such as peak signal to noise ratio (PSNR) and time consuming. The proposed method also has outstanding effects on maintaining the foreground edges and eliminating the infiltration from foreground to background.

出版日期: 2015-10-15
:  TP 391.4  
基金资助:

国家自然科学基金资助项目(61271339);浙江省自然科学基金资助项目(LY12F01019)

作者简介: 杜歆(1975-),男,副教授,博士,从事计算机视觉、3D视频处理研究.ORCID:0000-0002-6215-9733.E-mail: duxin@zju.edu.cn
服务  
把本文推荐给朋友
加入引用管理器
E-mail Alert
RSS
作者相关文章  

引用本文:

杜歆,邹泷. 视点合成中基于深度的空洞修复[J]. 浙江大学学报(工学版), 10.3785/j.issn.1008973X.2015.09.002.

DU Xin, ZOU Shuang. Depth based hole filling in view synthesis. JOURNAL OF ZHEJIANG UNIVERSITY (ENGINEERING SCIENCE), 10.3785/j.issn.1008973X.2015.09.002.

链接本文:

http://www.zjujournals.com/eng/CN/10.3785/j.issn.1008973X.2015.09.002        http://www.zjujournals.com/eng/CN/Y2015/V49/I9/1616

[1] SHUM H Y, KANG S B. A review of imagebased rendering techniques [C] ∥ Proceedings of SPIE 4067, Visual Communications and Image Processing 2000. Perth:SPIE, 2000: 2-13.
[2] FEHN C. Depthimagebased rendering (DIBR), compression, and transmission for a new approach on 3DTV [C] ∥ Proceedings of  SPIE 5291,Stereoscopic Displays and Virtual Reality Systems XI. San Jose: SPIE, 2004: 93-104.
[3] SHADE J, GORTLER S, HE L, et al. Layered depth images [C] ∥ Proceedings of the 25th Annual Conference on Computer Graphics and Interactive Techniques. Orlando: ACM, 1998: 231-242.
[4] 陈坤斌,刘海旭,李学明. 构造全局背景的虚拟视点合成算法[J]. 信号处理,2013, 29(10): 1307-1314.
CHEN Kunbin, LIU Haixu, LI Xueming. Virtual view synthesis using generated global background [J]. Journal of Signal Processing, 2013, 29(10): 1307-1314.
[5] YAO C, TILLO T, ZHAO Y, et al. Depth map driven hole filling algorithm exploiting temporal correlation information [J]. IEEE Transactions on Broadcasting, 2014, 60(2): 394-404.
[6] 王奎,安平,张兆杨, 等. Kinect深度图像快速修复算法[J].上海大学学报:自然科学版,2012,18(5): 454-458.
WANG Kui, AN Ping, ZHANG Zhaoyang, et al. Fast inpainting algorithm for kinect depth map [J]. Journal of Shanghai University:Natural Science Edition,2012,18(5):454-458.
[7] ZHANG L, TAM W J. Stereoscopic image generation based on depth images for 3DTV [J]. IEEE Transactions on Broadcasting, 2005, 51(2): 191-199.
[8] LEE P J, EFFENDI. Nongeometric distortion smoothing approach for depth map preprocessing [J]. IEEE Transactions on Multimedia, 2011, 13(2): 246-254.
[9] 高凯,陈贺新,赵岩,等. 面向虚拟视点绘制的深度图滤波及上采样方法[J]. 中国图象图形学报,2013,18(9):1085-1092.
GAO Kai, CHEN Hexin, ZHAO Yan, et al. Depth map filtering and upsampling method for virtual view rendering [J]. Journal of Image and Graphics, 2013,18(9): 1085-1092.
[10] 李应彬, 冯杰, 张华熊,等. 基于改进双边滤波的Kinect深度图像空洞修复算法研究[J]. 工业控制计算机, 2013, 26(11):105-106,109.
LI Yingbing, FENG Jie, ZHANG Huaxiong, et al. New algorithm of depth hole filling based on intensive bilateral filter [J]. Industrial Control Computer, 2013, 26(11): 105-106,109.
[11] DARIBO I, SAITO H. Depthaided image inpainting for novel view synthesis [C] ∥ Proceedings of 2010 IEEE International Workshop on Multimedia Signal Processing (MMSP). (S.l.): IEEE, 2010: 167-170.
[12] GAUTIER J, LE MEUR O, GUILLEMOT C. Depth based image completion for view synthesis [C] ∥ 3DTV Conference: the True VisionCapture,Transmission and Display of 3D Video (3DTVCON).Antalya:IEEE,2011:14.
[13] MA L N, DO L, WITH D, et al. Depthguided inpainting algorithm for freeviewpoint video [C] ∥ 2012 19th IEEE International Conferenceon on Image Processing (ICIP). Orlando: IEEE, 2012:1721-1724.
[14] WU H Y, FENG J, ZHANG H X, et al. A virtual view synthesis algorithm based on image inpainting [C] ∥ Third International Conference on Networking and Distributed Computing. Hangzhou: IEEE, 2012:153-156.
[15] XU X Y, PO L M, Cheung C H, et al. Depthaided exemplarbased hole filling for DIBR view synthesis [C]∥ 2013 IEEE International Symposium on Circuits and Systems (ISCAS). Beijing: IEEE, 2013:2840-2843.
[16] ZITNICK C L, SING B K, MATTHEW U, et al. High quality video view interpolation using a layered representation [J]. ACM Transactions on Graphics. 2004, 23(3): 600608.
[17] SCHARSTEIN D. Stereo vision for view synthesis [C] ∥ IEEE Computer Society Conference on Computer Vision and Pattern Recognition(CVPR’96). San Francisco: IEEE,1996: 852-858.
[18] TELEA A. An image inpainting technique based on the fast marching method [J]. Journal of Graphics Tools, 2004, 9(1): 2334.
[19] CRIMINISI A, PEREZ P, TOYAMA K, Region filling and object removal by exemplarbased image inpainting [J]. IEEE Transactions on Image Processing, 2004, 13(9): 1200-1212.
[20] PEREZ P, GANGNET M, AND BLAKE A. Poisson image editing [J]. Proceedings of the SIGGRAPH, 2003, 22(3): 313-318.
[21] SCHARSTEIN D, PAL C. Learning conditional random fields for stereo [C] ∥ IEEE Computer Society Conference on Computer Vision and Pattern Recognition (CVPR 2007). Minneapolis: IEEE, 2007: 18.
[22] XU X Y, PO L M, NG K H, et al. Depth map misalignment correction and dilation for DIBR view synthesis [J]. Signal Processing: Image Communication. 2013, 28(9): 1023-1045.
[23] TIAN D, LAI P L, PATRICK L. View synthesis techniques for 3D video [C] ∥ SPIE Optical Engineering and Applications. San Diego:SPIE, 2009: 74430T-74430T-11.

[1] 尤海辉, 马增益, 唐义军, 王月兰, 郑林, 俞钟, 吉澄军. 循环流化床入炉垃圾热值软测量[J]. 浙江大学学报(工学版), 2017, 51(6): 1163-1172.
[2] 董巍, 沈会良. 双向纹理函数稀疏采集与重建[J]. 浙江大学学报(工学版), 2016, 50(12): 2364-2370.
[3] 郭浩东, 陈岭, 丁永锋, 陈根才. 运动识别中基于主题的特征构建方法[J]. 浙江大学学报(工学版), 2016, 50(6): 1149-1154.
[4] 唐有宝, 卜巍, 邬向前. 多层次MSER自然场景文本检测[J]. 浙江大学学报(工学版), 2016, 50(6): 1134-1140.
[5] 沈延斌, 陈岭, 郭浩东, 陈根才. 基于深度学习的放置方式和位置无关运动识别[J]. 浙江大学学报(工学版), 2016, 50(6): 1141-1148.
[6] 陈实, 郑楷洪, 孙凌云, 李彦. 可穿戴计算设备中振动表情的设计与应用[J]. 浙江大学学报(工学版), 2015, 49(12): 2298-2304.
[7] 吴一全,殷骏,朱丽,袁永明. 基于蜂群优化或分解的二维Arimoto灰度熵阈值分割[J]. 浙江大学学报(工学版), 2015, 49(9): 1625-1633.
[8] 孙凌云, 何博伟, 刘征, 杨智渊. 基于语义细胞的语音情感识别[J]. 浙江大学学报(工学版), 2015, 49(6): 1001-1009.
[9] 王媛媛, 郭延恩, 施国全, 韦俊霞, 夏顺仁. 基于声图像的海底地形边界提取算法[J]. 浙江大学学报(工学版), 2015, 49(2): 376-383.
[10] 程健, 项志宇, 于海滨, 刘济林. 城市复杂环境下基于三维激光雷达实时车辆检测[J]. 浙江大学学报(工学版), 2014, 48(12): 2101-2106.
[11] 杨钟亮, 唐智川, 陈育苗, 高增桂. 面向双侧训练的前臂外骨骼肌肉力-电关系识别模型[J]. 浙江大学学报(工学版), 2014, 48(12): 2152-2161.
[12] 杨冰,王小华,杨鑫,黄孝喜. 基于HOG金字塔人脸识别方法[J]. 浙江大学学报(工学版), 2014, 48(9): 1564-1569.
[13] 杨力,刘济林. 基于非监督特征学习的分叉道路检测算法[J]. 浙江大学学报(工学版), 2014, 48(9): 1558-1563.
[14] 胡祝华, 赵瑶池, 程杰仁, 彭金莲. 基于改进DRLSE的运动目标分割方法[J]. 浙江大学学报(工学版), 2014, 48(8): 1488-1495.
[15] 杨力, 刘俊毅, 王延长, 刘济林. 基于全景相机和全向激光雷达的致密三维重建[J]. 浙江大学学报(工学版), 2014, 48(8): 1481-1487.