Please wait a minute...
浙江大学学报(工学版)  2020, Vol. 54 Issue (2): 320-330    DOI: 10.3785/j.issn.1008-973X.2020.02.013
计算机技术、信息工程     
结合编码曝光和运动先验信息的局部模糊图像复原
叶晓杰1(),崔光茫1,2,3,于快快2,*(),赵巨峰1,3,朱礼尧1
1. 杭州电子科技大学 电子信息学院,浙江 杭州 310018
2. 光电信息控制和安全技术重点实验室,天津 300308
3. 浙江省装备电子研究重点实验室,浙江 杭州 310018
Restoration of local blurred images based on coded exposure and motion prior information
Xiao-jie YE1(),Guang-mang CUI1,2,3,Kuai-kuai YU2,*(),Ju-feng ZHAO1,3,Li-yao ZHU1
1. School of Electronics and Information, Hangzhou Dianzi University, Hangzhou 310018, China
2. Science and Technology on Electro-Optical Information Security Control Laboratory, Tianjin 300308, China
3. Zhejiang Provincial Key Laboratory of Equipment Electronics, Hangzhou 310018, China
 全文: PDF(2616 KB)   HTML
摘要:

针对局部运动模糊图像复原的病态性和背景被破坏的问题,提出基于编码曝光和运动先验信息的局部模糊图像复原方法. 分析编码曝光成像理论模型,建立最优码字选取的适应度函数准则. 通过物像关系,获得运动目标的点扩散函数(PSF)像移尺度初步估计参数,作为运动先验信息. 采用背景差分法进行目标提取,综合编码曝光运动模糊图像的叠加特性,实现对运动模糊目标区域的精确提取. 结合先验信息,引入基于贝叶斯最大后验概率框架的student-t复原算法进行PSF精确估计和复原重建,快速迭代得到复原结果. 搭建实验仿真系统,并开展针对实际运动目标的复原实验. 实验结果表明,该方法能有效改善传统曝光中运动模糊复原的病态性问题,抑制复原过程中目标图像边缘振铃及背景噪声的放大效应,所复原图像具有更好的主客观评价结果.

关键词: 编码曝光运动模糊运动先验信息局部复原点扩散函数    
Abstract:

A local blurred image restoration method based on the coded exposure and motion prior information was proposed, aiming at the problem of ill-posed and background damage along with the local motion blurred image restoration process. The theoretical model of coded exposure imaging was analyzed, and the fitness function criterion for the optimum code sequence selection was established. Through the object-image relationship, the preliminary estimation scale parameters of point spread function (PSF) for moving target were obtained as the prior information of motion. The background difference subtraction method was used to extract the motion blurred target area accurately by synthesizing the overlapping characteristics of the motion blurred image by coded exposure. Combined with the prior information, the student-t restoration algorithm based on the Bayesian maximum posteriori probability framework was introduced to estimate and reconstruct the PSF accurately. And the restoration results were obtained by several iterations fast. The experimental simulation system was built and the reconstruct experiments aming at the real motion targets were carried out. Experimental results show that the method can improve the pathological problem of motion blur restoration in traditional exposure effectively, as well as restrain the magnification effect of edge ringing and background noise in the restoration process. The images restored by the proposed method have better subjective and objective evaluation results than that of other methods.

Key words: coded exposure    motion blur    motion prior information    local restoration    point spread function
收稿日期: 2019-05-06 出版日期: 2020-03-10
CLC:  TN 911  
基金资助: 国家自然科学基金资助项目(61805063);浙江省自然科学基金资助项目(LY18F050007)
通讯作者: 于快快     E-mail: 13588029671@163.com;oespfa2018@126.com
作者简介: 叶晓杰(1996—),男,硕士生,从事图像处理研究. orcid.org/0000-0003-4546-2826. E-mail: 13588029671@163.com
服务  
把本文推荐给朋友
加入引用管理器
E-mail Alert
作者相关文章  
叶晓杰
崔光茫
于快快
赵巨峰
朱礼尧

引用本文:

叶晓杰,崔光茫,于快快,赵巨峰,朱礼尧. 结合编码曝光和运动先验信息的局部模糊图像复原[J]. 浙江大学学报(工学版), 2020, 54(2): 320-330.

Xiao-jie YE,Guang-mang CUI,Kuai-kuai YU,Ju-feng ZHAO,Li-yao ZHU. Restoration of local blurred images based on coded exposure and motion prior information. Journal of ZheJiang University (Engineering Science), 2020, 54(2): 320-330.

链接本文:

http://www.zjujournals.com/eng/CN/10.3785/j.issn.1008-973X.2020.02.013        http://www.zjujournals.com/eng/CN/Y2020/V54/I2/320

图 1  传统曝光模式示意图
图 2  编码曝光模式示意图
图 3  基于编码曝光和运动先验信息的局部模糊图像复原方案实施流程图
图 4  运动模糊目标与背景
图 5  目标模板和运行目标提取图
图 6  基于编码曝光的局部运动模糊图像采集实验平台
图 7  不同方法的目标提取结果
图 8  清晰图像及局部放大图
图 9  编码曝光模糊图和不同方法复原结果(PSF像移尺度约为15个像素)
图 10  编码曝光模糊图和不同方法复原结果(PSF像移尺度约为20个像素)
图 11  编码曝光模糊图和不同方法复原结果(PSF像移尺度约为25个像素)
算法 15像素 20像素 25像素
SSIM PSNR SSIM PSNR SSIM PSNR
RL 0.788 2 34.271 5 0.494 6 30.908 3 0.640 6 32.006 4
Levin 0.840 9 30.839 5 0.696 0 30.651 3 0.750 2 31.301 3
本研究算法 0.876 9 36.513 8 0.697 1 32.145 9 0.770 3 33.852 2
表 1  不同长度不同算法下的复原评价指标
图 12  不同曝光模式复原效果(PSF像移尺度约为15个像素)
图 13  不同曝光模式复原效果(PSF像移尺度约为20个像素)
图 14  不同曝光模式复原效果(PSF像移尺度约为25个像素)
曝光方式 15像素 20像素 25像素
SSIM PSNR SSIM PSNR SSIM PSNR
传统曝光复原 0.601 6 30.966 2 0.513 8 30.373 9 0.535 2 31.154 0
编码曝光复原 0.876 9 36.513 8 0.697 1 32.145 9 0.770 3 33.852 2
表 2  不同曝光方式的复原评价指标
图 15  编码曝光下基于student-t算法的全局和局部复原图像
图 16  编码曝光下不同背景不同目标的复原结果图
复原算法 SSIM PSNR
RL 0.533 7 30.127 3
Levin 0.626 8 30.088 3
本研究算法 0.678 5 34.477 9
表 3  图18的运动复原结果评价指标
图 17  实际运动目标的复原结果
图 18  不同复原方法对实际运动目标的复原结果(局部放大区域)
1 李东升, 陈春晓, 王章立, 等 基于全局方差和噪声估计的维纳滤波图像的复原方法[J]. 生物医学工程研究, 2017, 36 (4): 331- 335
LI Dong-sheng, CHEN Chun-xiao, WANG Zhang-li, et al Restoration method of Wiener filtered image based on global variance and noise estimation[J]. Journal of Biomedical Engineering Research, 2017, 36 (4): 331- 335
2 朱一帆 基于逆滤波、维纳滤波算法的图像复原技术的研究[J]. 中国新通信, 2018, 20 (6): 238
ZHU Yi-fan Research on image restoration technology based on inverse filtering and Wiener filtering algorithm[J]. China New Telecommunications, 2018, 20 (6): 238
doi: 10.3969/j.issn.1673-4866.2018.06.190
3 徐岩, 曾祥波 基于红色暗通道先验和逆滤波的水下图像复原[J]. 激光与光电子学进展, 2018, 55 (2): 221- 228
XU Yan, ZENG Xiang-bo Underwater image restoration based on red dark channel prior and inverse filtering[J]. Laser and Optoelectronics Progress, 2018, 55 (2): 221- 228
4 肖宿, 韩国强 基于变量分离和加权最小二乘法的图像复原[J]. 计算机应用研究, 2012, (4): 1584- 1587
XIAO Su, HAN Guo-qiang Image restoration based on variable separation and weighted least square method[J]. Application Research of Computers, 2012, (4): 1584- 1587
doi: 10.3969/j.issn.1001-3695.2012.04.108
5 叶鹏钊, 冯华君, 徐之海, 等 基于块效应抑制的压缩降质模糊图像盲复原[J]. 浙江大学学报: 工学版, 2018, 52 (2): 406- 412
YE Peng-zhao, FENG Hua-jun, XU Zhi-hai, et al Blind restoration of compressed degraded blurred images based on block effect suppression[J]. Journal of Zhejiang University: Engineering Science, 2018, 52 (2): 406- 412
6 苏慧, 冯华君, 徐之海, 等 基于数值保真项优化的TDI遥感图像复原方法[J]. 浙江大学学报: 工学版, 2018, 52 (4): 674- 679
SU Hui, FENG Hua-jun, XU Zhi-hai, et al TDI remote sensing image restoration method based on numerical fidelity optimization[J]. Journal of Zhejiang University: Engineering Science, 2018, 52 (4): 674- 679
7 张玉叶, 周胜明, 赵育良, 等 高速运动目标的运动模糊图像复原研究[J]. 红外与激光工程, 2017, (4): 257- 262
ZHANG Yu-ye, ZHOU Sheng-ming, ZHAO Yu-liang, et al Research on motion blurred image restoration of high-speed moving targets[J]. Infrared and Laser Engineering, 2017, (4): 257- 262
8 RASKAR R, AGRAWAL A, TUMBLIN J Coded exposure photography: motion deblurring using fluttered shutter[J]. ACM Transactions on Graphics, 2006, 25 (3): 795- 804
doi: 10.1145/1141911.1141957
9 AGRAWAL A, RASKAR R. Optimal single image capture for motion deblurring [C]// Computer Vision and Pattern Recognition. Miami: IEEE, 2009: 2560-2567.
10 MCCLOSKEY S. Temporally coded flash illumination for motion deblurring [C]// IEEE International Conference on Computer Vision. Barcelona: IEEE, 2011: 683-690.
11 MCCLOSKEY S, DING Y, YU J Design and estimation of coded exposure point spread functions[J]. IEEE Transactions on Pattern Analysis and Machine Intelligence, 2012, 34 (10): 2071- 2077
doi: 10.1109/TPAMI.2012.108
12 TAI Y W, KONG N, LIN S, et al. Coded exposure imaging for projective motion deblurring [C]// Computer Vision and Pattern Recognition. San Francisco: IEEE, 2010: 2408-2415.
13 JEON H G, LEE J Y, HAN Y, et al Multi-image deblurring using complementary sets of fluttering patterns[J]. IEEE Transactions on Image Processing, 2017, 26 (5): 2311- 2326
doi: 10.1109/TIP.2017.2675202
14 HE L, CUI G, FENH H, et al Fast image restoration method based on coded exposure and vibration detection[J]. Optical Engineering, 2015, 54 (10): 536- 545
15 HE L, CUI G, FENH H, et al The optimal code searching method with an improved criterion of coded exposure for remote sensing image restoration[J]. Optics Communications, 2015, 338: 540- 550
doi: 10.1016/j.optcom.2014.11.021
16 李红, 李明伟 编码曝光相机设计与实现[J]. 光电工程, 2016, 43 (9): 72- 77
LI Hong, LI Ming-wei Design and implementation of coded exposure camera[J]. Opto-Electronic Engineering, 2016, 43 (9): 72- 77
doi: 10.3969/j.issn.1003-501X.2016.09.013
17 何富斌. 编码曝光图像的运动去模糊方法研究[D]. 大连: 大连理工大学, 2016.
HE Fu-bin.Research on motion deblurring method for encoding exposure image [D]. Dalian: Dalian University of Technology, 2016.
18 LI X, SUN Y Joint structural similarity and entropy estimation for coded-exposure image restoration[J]. Multimedia Tools and Applications, 2018, 77 (22): 29811- 29828
doi: 10.1007/s11042-018-5773-3
19 崔光茫, 于快快, 叶晓杰, 等 基于Memetic算法的编码曝光最优码字序列搜索方法[J]. 光学学报, 2019, 39 (3): 0311004
CUI Guang-mang, YU Kuai-kuai, YE Xiao-jie, et al Search method of coded exposure optimal codeword sequence based on Memetic algorithm[J]. Acta Optica Sinica, 2019, 39 (3): 0311004
20 王俊臣. 基于小波变换的单幅运动模糊图像恢复[D]. 兰州: 兰州交通大学, 2013.
WANG Jun-chen. Single motion blurred image restoration based on wavelet transform [D]. Lanzhou: Lanzhou Jiaotong University, 2013.
21 朱加豪, 唐岚, 廖若冰 基于RPCA的局部运动模糊图像的复原[J]. 软件, 2017, 38 (8): 54- 58
ZHU Jia-hao, TANG Lan, LIAO Ruo-bing Restoration of local motion blurred images based on RPCA[J]. Computer Engineering and Software, 2017, 38 (8): 54- 58
doi: 10.3969/j.issn.1003-6970.2017.08.010
22 CHO S, LEE S Fast motion deblurring[J]. ACM Transactions on Graphics, 2009, 28 (5): 145
23 TAI Y W, DU H, BROWN M S, et al Correction of spatially varying image and video motion blur using a hybrid camera[J]. IEEE Transactions on Pattern Analysis and Machine Intelligence, 2010, 32 (6): 1012- 1028
doi: 10.1109/TPAMI.2009.97
24 何丽蓉, 黄加紫, 冯华君, 等 基于颤振探测和编码曝光的快速图像复原方法[J]. 光学学报, 2014, 34 (9): 0910003
HE Li-rong, HUANG Jia-zi, FENG Hua-jun, et al Fast image restoration method based on flutter detection and coded exposure[J]. Acta Optica Sinica, 2014, 34 (9): 0910003
25 赵东, 赵宏伟, 于繁华 动态多目标优化的运动物体图像分割[J]. 光学精密工程, 2015, 23 (7): 2109- 2116
ZHAO Dong, ZHAO Hong-wei, YU Fan-hua Dynamic multi-objective optimization for moving object image segmentation[J]. Optics and Precision Engineering, 2015, 23 (7): 2109- 2116
26 ROTH S, BLACK M J Fields of experts[J]. International Journal of Computer Vision, 2009, 82 (2): 205
doi: 10.1007/s11263-008-0197-6
27 WEISS Y, FREEMAN W T. What makes a good model of natural images? [C]// IEEE Conference on Computer Vision and Pattern Recognition. Minneapoli: IEEE, 2007.
28 董文德. 基于光纤陀螺颤振探测的图像复原技术研究[D]. 杭州: 浙江大学, 2013.
DONG Wen-de. Research on image restoration technology based on fiber optic gyroscope flutter detection [D]. Hangzhou: Zhejiang University, 2013.
[1] 李奇 冯华君 徐之海. 用于全数字对焦的点扩散函数性能分析与评价[J]. J4, 2006, 40(6): 1093-1096.