结合大气散射模型的生成对抗网络去雾算法

doi:10.3785/j.issn.1008-973X.2022.02.002

浙江大学学报(工学版)

2022, Vol. 56

Issue (2): 225-235 DOI: 10.3785/j.issn.1008-973X.2022.02.002

计算机与控制工程

结合大气散射模型的生成对抗网络去雾算法

屠杭垚(

),王万良*(

),陈嘉诚,李国庆,吴菲

浙江工业大学计算机科学与技术学院，浙江杭州 310014

Dehazing algorithm combined with atmospheric scattering model based on generative adversarial network

Hang-yao TU(

),Wan-liang WANG*(

),Jia-chen CHEN,Guo-qing LI,Fei WU

School of Computer Science and Technology, Zhejiang University of Technology, Hangzhou 310014, China

全文: PDF(1688 KB) HTML

摘要：

为了提高图像去雾的性能, 提出结合大气散射模型生成对抗网络的去雾算法. 算法在pix2pix GAN基础上进行改进, 将网络的生成器改进成双解码器结构，通过双解码器分别生成无雾图像和透射率图, 并结合大气散射模型还原雾图像，以进一步提高图像分解的质量. 在马尔科夫判别器结构中，采用反向学习机制代替随机裁剪机制，以有效降低因采用随机裁剪算法而导致的判断结果不准确的概率. 在原有的损失函数上，加入雾霾损失函数，提高图像转化的质量. 在STOS和NYU数据集上进行消融实验和对比实验. 大量实验表明所提出方法在PSNR和SSIM指标上比原算法Pix2pix GAN有所提高, 且均优于现有去雾算法，复原图像具有清晰度高、噪声低、纹理丰富的优点.

关键词： 生成对抗网络; 匹配图像; 去雾; 反向学习; 大气散射模型

Abstract:

A dehazing algorithm combined with atmospheric scattering model based on generative adversarial network was proposed in order to improve the performance of image dehazing. The algorithm was improved base on pix2pix GAN. Firstly, the generator is improved to a double decoder structure. The double decoder generates the fog-free image and the transmittance image, separately, then the fog-free image and the transmittance image are combined to restore the fog image by the atmospheric scattering model. The purpose is to improve the quality of decomposition. Secondly, in the Markov discriminator structure, the reverse learning mechanism is used to replace the random cropping mechanism, which aims to reduce the probability of inaccurate judgment caused by the random cropping algorithm. Finally, the haze loss function is added to the original loss function to improve the quality of image translation. The ablation experiments and contrast experiments were applied on STOS and NYU datasets. Experimental results showed that the proposed method was better than the original algorithm pix2pix GAN in terms of PSNR and SSIM, and both were better than the existing dehazing algorithms. The restored images have the advantages of high-resolution, low noise and rich texture.

Key words: generative adversarial network paired images dehazing reverse learning atmospheric scattering mode

收稿日期: 2021-03-29 出版日期: 2022-03-03

CLC:

TP 391

基金资助: 国家自然科学基金资助项目（61873240）

通讯作者: 王万良 E-mail: lewieyao@yeah.net;wwl@zjut.edu.cn

作者简介: 屠杭垚（1995—），男，博士生，从事深度学习、生成对抗网络研究. orcid.org/0000-0001-9947-262X. E-mail: lewieyao@yeah.net

	服务
	把本文推荐给朋友
	加入引用管理器
	E-mail Alert
	作者相关文章
	屠杭垚
	王万良
	陈嘉诚
	李国庆
	吴菲

引用本文:

屠杭垚,王万良,陈嘉诚,李国庆,吴菲. 结合大气散射模型的生成对抗网络去雾算法[J]. 浙江大学学报(工学版), 2022, 56(2): 225-235.

Hang-yao TU,Wan-liang WANG,Jia-chen CHEN,Guo-qing LI,Fei WU. Dehazing algorithm combined with atmospheric scattering model based on generative adversarial network. Journal of ZheJiang University (Engineering Science), 2022, 56(2): 225-235.

链接本文:

https://www.zjujournals.com/eng/CN/10.3785/j.issn.1008-973X.2022.02.002 或 https://www.zjujournals.com/eng/CN/Y2022/V56/I2/225

图 1 本研究所提模型的生成器结构图

表 1 本研究所提模型的生成器网络参数

图 2 一维反向点图

图 3 二维反向点图

图 4 反向学习patchGAN示意图

表 2 判别器网络参数

图 5 结合大气散射模型的生成对抗网络去雾算法框架图

图 6 NYU和SOTS图像数据集示例

图 7 不同λ值效果对比图

表 3 不同λ参数下图像指标的对比

表 4 不同损失函数下图像指标的对比

图 8 不同损失函数下生成图像的对比

图 9 不同网络结构下生成图像的对比

表 5 不同网络结构下图像指标对比

图 10 不同雾霾浓度下生成图像的对比

图 11 不同雾霾浓度下本研究算法的去雾效果指标

图 12 不同算法下生成图像的对比

图 13 不同算法下的去雾效果指标

1	卜令兵, 袁静, 高爱臻, 等基于激光云高仪的雾霾过程探测[J]. 光子学报, 2014, 43 (9): 64- 69 BU Ling-bing, YUAN Jing, GAO Ai-zhen, et al Analysis of haze-fog events based on laser ceilometer[J]. Acta Photonics Sinica, 2014, 43 (9): 64- 69
2	曲晨, 毕笃彦基于懒惰随机游走的雾天含噪图像清晰化[J]. 光学学报, 2018, 38 (4): 103- 112 QU Chen, BI Du-yan Clearness for foggy and noisy image based on lazy random walk[J]. Acta Optica Sinica, 2018, 38 (4): 103- 112
3	MIN X, ZHAI G, GU K, et al Quality evaluation of image dehazing methods using synthetic hazy images[J]. IEEE Transactions on Multimedia, 2019, 21 (9): 2319- 2333 doi: 10.1109/TMM.2019.2902097
4	DOU Z, HAN Y, SHENG W, et al Image dehaze using alternating Laplacian and Beltrami regularizations[J]. Journal of Electronic Imaging, 2015, 24 (2): 023004 doi: 10.1117/1.JEI.24.2.023004
5	覃宏超, 李炎炎, 龙伟, 等基于引导滤波和暗原色先验理论透射率估值的视频去雾算法[J]. 浙江大学学报:工学版, 2018, 52 (7): 1302- 1309 QIN Hong-chao, LI Yan-yan, LONG Wei, et al Real-time video dehazing using guided filtering and transmissivity estimated based on dark channel prior theory[J]. Journal of Zhejiang University: Engineering Science, 2018, 52 (7): 1302- 1309
6	JIN W, MI Z, WU X, et al. Single image de-haze based on a new dark channel estimation method [C]// 2012 IEEE International Conference on Computer Science and Automation Engineering. Zhangjiajie: IEEE , 2012: 791-795.
7	ENGIN D, GENC A, KEMAL E H. Cycle-dehaze: enhanced cyclegan for single image dehazing [C]// Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition Workshops. Salt Lake City: IEEE , 2018: 825-833.
8	LI C, ZHAO X, ZHANG Z, et al Generative adversarial dehaze mapping nets[J]. Pattern Recognition Letters, 2019, 119: 238- 244 doi: 10.1016/j.patrec.2017.11.021
9	YANG D, SUN J. Proximal dehaze-net: a prior learning-based deep network for single image dehazing[C]// Proceedings of the European Conference on Computer Vision. Munich: IEEE, 2018: 702-717.
10	KUANAR S, RAO K R, MAHAPATRA D, et al. Night time haze and glow removal using deep dilated convolutional network [EB/OL]. [2021-03-01]. https://arxiv.53yu.com/ftp/arxiv/papers/1902/1902.00855.pdf.
11	HE K, SUN J, TANG X Single image haze removal using dark channel prior[J]. IEEE Transactions on Pattern Analysis and Machine Intelligence, 2010, 33 (12): 2341- 235
12	金仙力, 张威, 刘林峰基于引导滤波和自适应容差的图像去雾算法[J]. 通信学报, 2020, 41 (5): 27- 36 JIN Xian-li, ZHANG Wei, LIU Lin-feng Image defogging algorithm based on guided filtering and adaptive tolerance[J]. Journal of Communications, 2020, 41 (5): 27- 36 doi: 10.11959/j.issn.1000-436x.2020078
13	LI B, PENG X, WANG Z, et al. Aod-net: all-in-one dehazing network[C]// Proceedings of the IEEE International Conference on Computer Vision. Venice: IEEE, 2017: 4770-4778.
14	肖进胜, 申梦瑶, 雷俊锋, 等基于生成对抗网络的雾霾场景图像转换算法[J]. 计算机学报, 2020, 43 (1): 165- 176 XIAO Jin-sheng, SHEN Meng-yao, LEI Jun-feng, et al Image conversion algorithm for haze scenes based on generative adversarial networks[J]. Chinese Journal of Computers, 2020, 43 (1): 165- 176 doi: 10.11897/SP.J.1016.2020.00165
15	MIRZA M, OSINDERO S. Conditional generative adversarial nets [EB/OL]. [2021-03-01]. https://arxiv.53yu.com/pdf/1411.1784.pdf .
16	ISOLA P, ZHU J Y, ZHOU T, et al. Image-to-image translation with conditional adversarial networks[C]// Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition. Honolulu: IEEE, 2017: 1125-1134.
17	JIA Q, MA Z. Patch-based generative adversarial network for single image haze removal [C]// 2020 International Conference on Computer Engineering and Application. Guangzhou: IEEE, 2020: 882-886.
18	刘国, 吕群波, 刘扬阳基于自适应暗原色的单幅图像去雾算法[J]. 光子学报, 2018, 47 (2): 179- 186 LIU Guo, LV Qun-bo, LIU Yang-yang Single image dehazing algorithm based on adaptive dark channel prior[J]. Acta Photonica Sinica, 2018, 47 (2): 179- 186
19	WANG J B, HE N, ZHANG L L, et al Single image dehazing with a physical model and dark channel prior[J]. Neurocomputing, 2015, 149: 718- 728 doi: 10.1016/j.neucom.2014.08.005
20	PAN J, SUN D, PFISTER H, et al Deblurring images via dark channel prior[J]. IEEE Transactions on Pattern Analysis and Machine Intelligence, 2017, 40 (10): 2315- 2328
21	LI S, REN W, ZHANG J, et al Single image rain removal via a deep decomposition-composition network[J]. Computer Vision and Image Understanding, 2019, 186: 48- 57 doi: 10.1016/j.cviu.2019.05.003
22	RONNEBERGER O, FISCHER P, BROX T. U-net: convolutional networks for biomedical image segmentation [C]// International Conference on Medical Image Computing and Computer-assisted Intervention. Munich: Springer, 2015: 234-241.
23	TIZHOOSH H R. Opposition-based learning: a new scheme for machine intelligence[C]// International Conference on Computational Intelligence for Modelling. Vienna: IEEE, 2005, 1: 695-701.
24	KUMAR A, SHAIKH A M, LI Y, et al Pruning filters with L1-norm and capped L1-norm for CNN compression[J]. Applied Intelligence, 2021, 51 (2): 1152- 1160 doi: 10.1007/s10489-020-01894-y
25	KAMAL U, TONMOY T I, DAS S, et al Automatic traffic sign detection and recognition using SegU-Net and a modified Tversky loss function with L1-constraint[J]. IEEE Transactions on Intelligent Transportation Systems, 2019, 21 (4): 1467- 1479
26	LEE C P, LIN C J A study on L2-loss (squared hinge-loss) multiclass SVM[J]. Neural Computation, 2013, 25 (5): 1302- 1323 doi: 10.1162/NECO_a_00434
27	WANG L T, HOOVER N E, PORTER E H, et al. SSIM: a software levelized compiled-code simulator [C]// Proceedings of the 24th ACM/IEEE Design Automation Conference. New York: ACM, 1987: 2-8.
28	HUYNH-THU Q, GHANBARI M The accuracy of PSNR in predicting video quality for different video scenes and frame rates[J]. Telecommunication Systems, 2012, 49 (1): 35- 48 doi: 10.1007/s11235-010-9351-x
29	GAO Y, HU H M, LI B, et al Detail preserved single image dehazing algorithm based on airlight refinement[J]. IEEE Transactions on Multimedia, 2018, 21 (2): 351- 362
30	CAI B, XU X, JIA K, et al Dehazenet: an end-to-end system for single image haze removal[J]. IEEE Transactions on Image Processing, 2016, 25 (11): 5187- 5198 doi: 10.1109/TIP.2016.2598681
31	REN W, PAN J, ZHANG H, et al Single image dehazing via multi-scale convolutional neural networks with holistic edges[J]. International Journal of Computer Vision, 2020, 128 (1): 240- 259 doi: 10.1007/s11263-019-01235-8

[1]	白文超,韩希先,王金宝. 基于条件生成模型的高效近似查询处理框架[J]. 浙江大学学报(工学版), 2022, 56(5): 995-1005.
[2]	何立,庞善民. 结合年龄监督和人脸先验的语音-人脸图像重建[J]. 浙江大学学报(工学版), 2022, 56(5): 1006-1016.
[3]	陈彤,郭剑锋,韩心中,谢学立,席建祥. 基于生成对抗模型的可见光-红外图像匹配方法[J]. 浙江大学学报(工学版), 2022, 56(1): 63-74.
[4]	陈雪云,黄小巧,谢丽. 基于多尺度条件生成对抗网络血细胞图像分类检测方法[J]. 浙江大学学报(工学版), 2021, 55(9): 1772-1781.
[5]	胡惠雅,盖绍彦,达飞鹏. 基于生成对抗网络的偏转人脸转正[J]. 浙江大学学报(工学版), 2021, 55(1): 116-123.
[6]	刘坤,文熙,黄闽茗,杨欣欣,毛经坤. 基于生成对抗网络的太阳能电池缺陷增强方法[J]. 浙江大学学报(工学版), 2020, 54(4): 684-693.
[7]	段然,周登文,赵丽娟,柴晓亮. 基于多尺度特征映射网络的图像超分辨率重建[J]. 浙江大学学报(工学版), 2019, 53(7): 1331-1339.
[8]	覃宏超, 李炎炎, 龙伟, 赵瑞朋. 基于引导滤波和暗原色先验理论透射率估值的视频去雾算法[J]. 浙江大学学报(工学版), 2018, 52(7): 1302-1309.
[9]	傅蔚阳, 刘以安, 薛松. 基于灰狼算法与小波神经网络的目标威胁评估[J]. 浙江大学学报(工学版), 2018, 52(4): 680-686.

Viewed

Full text

Abstract

Cited

Shared

Discussed