Please wait a minute...
Journal of ZheJiang University (Engineering Science)  2020, Vol. 54 Issue (2): 320-330    DOI: 10.3785/j.issn.1008-973X.2020.02.013
Computer Technology, Information Engineering     
Restoration of local blurred images based on coded exposure and motion prior information
Xiao-jie YE1(),Guang-mang CUI1,2,3,Kuai-kuai YU2,*(),Ju-feng ZHAO1,3,Li-yao ZHU1
1. School of Electronics and Information, Hangzhou Dianzi University, Hangzhou 310018, China
2. Science and Technology on Electro-Optical Information Security Control Laboratory, Tianjin 300308, China
3. Zhejiang Provincial Key Laboratory of Equipment Electronics, Hangzhou 310018, China
Download: HTML     PDF(2616KB) HTML
Export: BibTeX | EndNote (RIS)      

Abstract  

A local blurred image restoration method based on the coded exposure and motion prior information was proposed, aiming at the problem of ill-posed and background damage along with the local motion blurred image restoration process. The theoretical model of coded exposure imaging was analyzed, and the fitness function criterion for the optimum code sequence selection was established. Through the object-image relationship, the preliminary estimation scale parameters of point spread function (PSF) for moving target were obtained as the prior information of motion. The background difference subtraction method was used to extract the motion blurred target area accurately by synthesizing the overlapping characteristics of the motion blurred image by coded exposure. Combined with the prior information, the student-t restoration algorithm based on the Bayesian maximum posteriori probability framework was introduced to estimate and reconstruct the PSF accurately. And the restoration results were obtained by several iterations fast. The experimental simulation system was built and the reconstruct experiments aming at the real motion targets were carried out. Experimental results show that the method can improve the pathological problem of motion blur restoration in traditional exposure effectively, as well as restrain the magnification effect of edge ringing and background noise in the restoration process. The images restored by the proposed method have better subjective and objective evaluation results than that of other methods.



Key wordscoded exposure      motion blur      motion prior information      local restoration      point spread function     
Received: 06 May 2019      Published: 10 March 2020
CLC:  TN 911  
Corresponding Authors: Kuai-kuai YU     E-mail: 13588029671@163.com;oespfa2018@126.com
Cite this article:

Xiao-jie YE,Guang-mang CUI,Kuai-kuai YU,Ju-feng ZHAO,Li-yao ZHU. Restoration of local blurred images based on coded exposure and motion prior information. Journal of ZheJiang University (Engineering Science), 2020, 54(2): 320-330.

URL:

http://www.zjujournals.com/eng/10.3785/j.issn.1008-973X.2020.02.013     OR     http://www.zjujournals.com/eng/Y2020/V54/I2/320


结合编码曝光和运动先验信息的局部模糊图像复原

针对局部运动模糊图像复原的病态性和背景被破坏的问题,提出基于编码曝光和运动先验信息的局部模糊图像复原方法. 分析编码曝光成像理论模型,建立最优码字选取的适应度函数准则. 通过物像关系,获得运动目标的点扩散函数(PSF)像移尺度初步估计参数,作为运动先验信息. 采用背景差分法进行目标提取,综合编码曝光运动模糊图像的叠加特性,实现对运动模糊目标区域的精确提取. 结合先验信息,引入基于贝叶斯最大后验概率框架的student-t复原算法进行PSF精确估计和复原重建,快速迭代得到复原结果. 搭建实验仿真系统,并开展针对实际运动目标的复原实验. 实验结果表明,该方法能有效改善传统曝光中运动模糊复原的病态性问题,抑制复原过程中目标图像边缘振铃及背景噪声的放大效应,所复原图像具有更好的主客观评价结果.


关键词: 编码曝光,  运动模糊,  运动先验信息,  局部复原,  点扩散函数 
Fig.1 Schematic diagram of traditional exposure
Fig.2 Schematic diagram of coded exposure
Fig.3 Implementation flow chart of local blurred image restoration scheme based on coded exposure and motion prior information
Fig.4 Motion blurred target and background
Fig.5 Target template and moving target extraction diagram
Fig.6 Experimental platform of local motion blur image acquisition based on coded exposure
Fig.7 Object extraction results of different methods
Fig.8 Sharp image and partial enlarged image
Fig.9 Coded exposure blurred image and restored results of different methods with PSF motion scale of about 15 pixels
Fig.10 Coded exposure blurred image and restored results of different methods with PSF motion scale of about 20 pixels
Fig.11 Coded exposure blurred image and restored results of different methods with PSF motion scale of about 25 pixels
算法 15像素 20像素 25像素
SSIM PSNR SSIM PSNR SSIM PSNR
RL 0.788 2 34.271 5 0.494 6 30.908 3 0.640 6 32.006 4
Levin 0.840 9 30.839 5 0.696 0 30.651 3 0.750 2 31.301 3
本研究算法 0.876 9 36.513 8 0.697 1 32.145 9 0.770 3 33.852 2
Tab.1 Recovery evaluation indicators for different lengths and algorithms
Fig.12 Restored results under different exposure modes with PSF motion scale of about 15 pixels
Fig.13 Restored results under different exposure modes with PSF motion scale of about 20 pixels
Fig.14 Restored results under different exposure modes with PSF motion scale of about 25 pixels
曝光方式 15像素 20像素 25像素
SSIM PSNR SSIM PSNR SSIM PSNR
传统曝光复原 0.601 6 30.966 2 0.513 8 30.373 9 0.535 2 31.154 0
编码曝光复原 0.876 9 36.513 8 0.697 1 32.145 9 0.770 3 33.852 2
Tab.2 Evaluation indicators for restoration of different exposure modes
Fig.15 Global and local restoration images based on student-t algorithm under coded exposure
Fig.16 Recovery results of different targets with different backgrounds under coded exposure
复原算法 SSIM PSNR
RL 0.533 7 30.127 3
Levin 0.626 8 30.088 3
本研究算法 0.678 5 34.477 9
Tab.3 Evaluation indexs of exercise recovery results in Fig.18
Fig.17 Deblurring results of real moving target
Fig.18 Deblurring results of real moving target for differenet methods(local region zoomed in)
[1]   李东升, 陈春晓, 王章立, 等 基于全局方差和噪声估计的维纳滤波图像的复原方法[J]. 生物医学工程研究, 2017, 36 (4): 331- 335
LI Dong-sheng, CHEN Chun-xiao, WANG Zhang-li, et al Restoration method of Wiener filtered image based on global variance and noise estimation[J]. Journal of Biomedical Engineering Research, 2017, 36 (4): 331- 335
[2]   朱一帆 基于逆滤波、维纳滤波算法的图像复原技术的研究[J]. 中国新通信, 2018, 20 (6): 238
ZHU Yi-fan Research on image restoration technology based on inverse filtering and Wiener filtering algorithm[J]. China New Telecommunications, 2018, 20 (6): 238
doi: 10.3969/j.issn.1673-4866.2018.06.190
[3]   徐岩, 曾祥波 基于红色暗通道先验和逆滤波的水下图像复原[J]. 激光与光电子学进展, 2018, 55 (2): 221- 228
XU Yan, ZENG Xiang-bo Underwater image restoration based on red dark channel prior and inverse filtering[J]. Laser and Optoelectronics Progress, 2018, 55 (2): 221- 228
[4]   肖宿, 韩国强 基于变量分离和加权最小二乘法的图像复原[J]. 计算机应用研究, 2012, (4): 1584- 1587
XIAO Su, HAN Guo-qiang Image restoration based on variable separation and weighted least square method[J]. Application Research of Computers, 2012, (4): 1584- 1587
doi: 10.3969/j.issn.1001-3695.2012.04.108
[5]   叶鹏钊, 冯华君, 徐之海, 等 基于块效应抑制的压缩降质模糊图像盲复原[J]. 浙江大学学报: 工学版, 2018, 52 (2): 406- 412
YE Peng-zhao, FENG Hua-jun, XU Zhi-hai, et al Blind restoration of compressed degraded blurred images based on block effect suppression[J]. Journal of Zhejiang University: Engineering Science, 2018, 52 (2): 406- 412
[6]   苏慧, 冯华君, 徐之海, 等 基于数值保真项优化的TDI遥感图像复原方法[J]. 浙江大学学报: 工学版, 2018, 52 (4): 674- 679
SU Hui, FENG Hua-jun, XU Zhi-hai, et al TDI remote sensing image restoration method based on numerical fidelity optimization[J]. Journal of Zhejiang University: Engineering Science, 2018, 52 (4): 674- 679
[7]   张玉叶, 周胜明, 赵育良, 等 高速运动目标的运动模糊图像复原研究[J]. 红外与激光工程, 2017, (4): 257- 262
ZHANG Yu-ye, ZHOU Sheng-ming, ZHAO Yu-liang, et al Research on motion blurred image restoration of high-speed moving targets[J]. Infrared and Laser Engineering, 2017, (4): 257- 262
[8]   RASKAR R, AGRAWAL A, TUMBLIN J Coded exposure photography: motion deblurring using fluttered shutter[J]. ACM Transactions on Graphics, 2006, 25 (3): 795- 804
doi: 10.1145/1141911.1141957
[9]   AGRAWAL A, RASKAR R. Optimal single image capture for motion deblurring [C]// Computer Vision and Pattern Recognition. Miami: IEEE, 2009: 2560-2567.
[10]   MCCLOSKEY S. Temporally coded flash illumination for motion deblurring [C]// IEEE International Conference on Computer Vision. Barcelona: IEEE, 2011: 683-690.
[11]   MCCLOSKEY S, DING Y, YU J Design and estimation of coded exposure point spread functions[J]. IEEE Transactions on Pattern Analysis and Machine Intelligence, 2012, 34 (10): 2071- 2077
doi: 10.1109/TPAMI.2012.108
[12]   TAI Y W, KONG N, LIN S, et al. Coded exposure imaging for projective motion deblurring [C]// Computer Vision and Pattern Recognition. San Francisco: IEEE, 2010: 2408-2415.
[13]   JEON H G, LEE J Y, HAN Y, et al Multi-image deblurring using complementary sets of fluttering patterns[J]. IEEE Transactions on Image Processing, 2017, 26 (5): 2311- 2326
doi: 10.1109/TIP.2017.2675202
[14]   HE L, CUI G, FENH H, et al Fast image restoration method based on coded exposure and vibration detection[J]. Optical Engineering, 2015, 54 (10): 536- 545
[15]   HE L, CUI G, FENH H, et al The optimal code searching method with an improved criterion of coded exposure for remote sensing image restoration[J]. Optics Communications, 2015, 338: 540- 550
doi: 10.1016/j.optcom.2014.11.021
[16]   李红, 李明伟 编码曝光相机设计与实现[J]. 光电工程, 2016, 43 (9): 72- 77
LI Hong, LI Ming-wei Design and implementation of coded exposure camera[J]. Opto-Electronic Engineering, 2016, 43 (9): 72- 77
doi: 10.3969/j.issn.1003-501X.2016.09.013
[17]   何富斌. 编码曝光图像的运动去模糊方法研究[D]. 大连: 大连理工大学, 2016.
HE Fu-bin.Research on motion deblurring method for encoding exposure image [D]. Dalian: Dalian University of Technology, 2016.
[18]   LI X, SUN Y Joint structural similarity and entropy estimation for coded-exposure image restoration[J]. Multimedia Tools and Applications, 2018, 77 (22): 29811- 29828
doi: 10.1007/s11042-018-5773-3
[19]   崔光茫, 于快快, 叶晓杰, 等 基于Memetic算法的编码曝光最优码字序列搜索方法[J]. 光学学报, 2019, 39 (3): 0311004
CUI Guang-mang, YU Kuai-kuai, YE Xiao-jie, et al Search method of coded exposure optimal codeword sequence based on Memetic algorithm[J]. Acta Optica Sinica, 2019, 39 (3): 0311004
[20]   王俊臣. 基于小波变换的单幅运动模糊图像恢复[D]. 兰州: 兰州交通大学, 2013.
WANG Jun-chen. Single motion blurred image restoration based on wavelet transform [D]. Lanzhou: Lanzhou Jiaotong University, 2013.
[21]   朱加豪, 唐岚, 廖若冰 基于RPCA的局部运动模糊图像的复原[J]. 软件, 2017, 38 (8): 54- 58
ZHU Jia-hao, TANG Lan, LIAO Ruo-bing Restoration of local motion blurred images based on RPCA[J]. Computer Engineering and Software, 2017, 38 (8): 54- 58
doi: 10.3969/j.issn.1003-6970.2017.08.010
[22]   CHO S, LEE S Fast motion deblurring[J]. ACM Transactions on Graphics, 2009, 28 (5): 145
[23]   TAI Y W, DU H, BROWN M S, et al Correction of spatially varying image and video motion blur using a hybrid camera[J]. IEEE Transactions on Pattern Analysis and Machine Intelligence, 2010, 32 (6): 1012- 1028
doi: 10.1109/TPAMI.2009.97
[24]   何丽蓉, 黄加紫, 冯华君, 等 基于颤振探测和编码曝光的快速图像复原方法[J]. 光学学报, 2014, 34 (9): 0910003
HE Li-rong, HUANG Jia-zi, FENG Hua-jun, et al Fast image restoration method based on flutter detection and coded exposure[J]. Acta Optica Sinica, 2014, 34 (9): 0910003
[25]   赵东, 赵宏伟, 于繁华 动态多目标优化的运动物体图像分割[J]. 光学精密工程, 2015, 23 (7): 2109- 2116
ZHAO Dong, ZHAO Hong-wei, YU Fan-hua Dynamic multi-objective optimization for moving object image segmentation[J]. Optics and Precision Engineering, 2015, 23 (7): 2109- 2116
[26]   ROTH S, BLACK M J Fields of experts[J]. International Journal of Computer Vision, 2009, 82 (2): 205
doi: 10.1007/s11263-008-0197-6
[27]   WEISS Y, FREEMAN W T. What makes a good model of natural images? [C]// IEEE Conference on Computer Vision and Pattern Recognition. Minneapoli: IEEE, 2007.
[28]   董文德. 基于光纤陀螺颤振探测的图像复原技术研究[D]. 杭州: 浙江大学, 2013.
DONG Wen-de. Research on image restoration technology based on fiber optic gyroscope flutter detection [D]. Hangzhou: Zhejiang University, 2013.
[1] Zhen JIA,Wen-de DONG,Gui-li XU,Shi-peng ZHU. Image Poisson denoising algorithm based on Markov fields of experts[J]. Journal of ZheJiang University (Engineering Science), 2020, 54(6): 1164-1169.
[2] Yi-xiong FENG,Kang-jie LI,Yi-cong GAO,Hao ZHEN,Jian-rong TAN. Shaft surface defect detection method based on feature and morphology reconstruction[J]. Journal of ZheJiang University (Engineering Science), 2020, 54(3): 427-434.
[3] Lie-jun JIN,Jian-ming ZHAN,Jun-hua CHEN,Tao WANG. Drill pipe fault diagnosis method based on one-dimensional convolutional neural network[J]. Journal of ZheJiang University (Engineering Science), 2020, 54(3): 467-474.
[4] Xiang-long ZHAO,Jie CHEN,Rong-jing HONG,Hua WANG,Yuan-yuan LI. Adaptive feature extraction method for slewing bearing based on Wavelet leader and optimized isometric mapping method[J]. Journal of ZheJiang University (Engineering Science), 2019, 53(11): 2092-2101.
[5] Tian-zhong HU,Jian-bo YU. Life prediction of lithium-ion batteries based on multiscale decomposition and deep learning[J]. Journal of ZheJiang University (Engineering Science), 2019, 53(10): 1852-1864.
[6] Ya-yun ZHANG,Jia-rui LIU,Zhi-yu WANG,Jiong-jiong MO,Fa-xin YU. Implementation of direct digital frequency synthesizer based on three-step rotation coordinate rotation digital computer algorithm[J]. Journal of ZheJiang University (Engineering Science), 2019, 53(10): 2034-2040.
[7] WU Chen-xi, ZHANG Min, WANG Ke-ren. Broadband underdetermined direction of arrival estimation based on two level nested array[J]. Journal of ZheJiang University (Engineering Science), 2017, 51(5): 1016-1023.
[8] XIE Luo feng, XU Hui ning, HUANG Qin yuan, ZHAO Yue, YIN Guo fu. Application of DTCWPT and NCA-LSSVM to inspect internal defects of magnetic tile[J]. Journal of ZheJiang University (Engineering Science), 2017, 51(1): 184-191.
[9] WANG Zhi, ZHU Shi qiang, BU Yan, GUO Zhen min. Stereo matching algorithm using improved guided filtering[J]. Journal of ZheJiang University (Engineering Science), 2016, 50(12): 2262-2269.
[10] CHEN Kuo, FENG Hua-jun, XU Zhi-hai, LI Qi, CHEN Yue-ting. Fast detail-preserving exposure fusion[J]. Journal of ZheJiang University (Engineering Science), 2015, 49(6): 1048-1054.
[11] YU Hui-min, ZENG Xiong. Visual tracking combined with ranking vector SVM[J]. Journal of ZheJiang University (Engineering Science), 2015, 49(6): 1015-1021.
[12] ZHU Zhu, LIU Ji-lin. Real-time Markov random field based ground segmentation of 3D Lidar data[J]. Journal of ZheJiang University (Engineering Science), 2015, 49(3): 464-469.
[13] JIANG Shen-yu, CHEN Kuo, XU Zhi-hai, FENG Hua-jun, LI Qi, CHEN Yue-ting. Multi-exposure image fusion based on well-exposedness assessment[J]. Journal of ZheJiang University (Engineering Science), 2015, 49(3): 470-475.
[14] TONG Ji-jun, ZHANG Guang-lei, CAI Qiang, JIAN Jin-ming, GUO Xi-shan. Application of threshold stochastic resonance in low concentration gas detecting[J]. Journal of ZheJiang University (Engineering Science), 2015, 49(1): 15-19.
[15] LI Jiang, ZHAO Ya-qiong, BAO Ye-hua. Voice processing technique for patients with stroke based on chao theory and surrogate data analysis[J]. Journal of ZheJiang University (Engineering Science), 2015, 49(1): 36-41.