Dehazing algorithm combined with atmospheric scattering model based on generative adversarial network

doi:10.3785/j.issn.1008-973X.2022.02.002

Journal of ZheJiang University (Engineering Science)

2022, Vol. 56

Issue (2): 225-235 DOI: 10.3785/j.issn.1008-973X.2022.02.002

Dehazing algorithm combined with atmospheric scattering model based on generative adversarial network

Hang-yao TU(

),Wan-liang WANG*(

),Jia-chen CHEN,Guo-qing LI,Fei WU

School of Computer Science and Technology, Zhejiang University of Technology, Hangzhou 310014, China

Download:

HTML

PDF(1688KB) HTML
Export: BibTeX | EndNote (RIS)

Abstract

A dehazing algorithm combined with atmospheric scattering model based on generative adversarial network was proposed in order to improve the performance of image dehazing. The algorithm was improved base on pix2pix GAN. Firstly, the generator is improved to a double decoder structure. The double decoder generates the fog-free image and the transmittance image, separately, then the fog-free image and the transmittance image are combined to restore the fog image by the atmospheric scattering model. The purpose is to improve the quality of decomposition. Secondly, in the Markov discriminator structure, the reverse learning mechanism is used to replace the random cropping mechanism, which aims to reduce the probability of inaccurate judgment caused by the random cropping algorithm. Finally, the haze loss function is added to the original loss function to improve the quality of image translation. The ablation experiments and contrast experiments were applied on STOS and NYU datasets. Experimental results showed that the proposed method was better than the original algorithm pix2pix GAN in terms of PSNR and SSIM, and both were better than the existing dehazing algorithms. The restored images have the advantages of high-resolution, low noise and rich texture.

Key words： generative adversarial network paired images dehazing reverse learning atmospheric scattering mode

Received: 29 March 2021 Published: 03 March 2022

CLC:

TP 391

Corresponding Authors: Wan-liang WANG E-mail: lewieyao@yeah.net;wwl@zjut.edu.cn

	Service
	E-mail this article
	Add to my bookshelf
	Add to citation manager
	E-mail Alert
	RSS
	Articles by authors
	Hang-yao TU
	Wan-liang WANG
	Jia-chen CHEN
	Guo-qing LI
	Fei WU

Cite this article:

Hang-yao TU,Wan-liang WANG,Jia-chen CHEN,Guo-qing LI,Fei WU. Dehazing algorithm combined with atmospheric scattering model based on generative adversarial network. Journal of ZheJiang University (Engineering Science), 2022, 56(2): 225-235.

URL:

https://www.zjujournals.com/eng/10.3785/j.issn.1008-973X.2022.02.002 OR https://www.zjujournals.com/eng/Y2022/V56/I2/225

结合大气散射模型的生成对抗网络去雾算法

为了提高图像去雾的性能, 提出结合大气散射模型生成对抗网络的去雾算法. 算法在pix2pix GAN基础上进行改进, 将网络的生成器改进成双解码器结构，通过双解码器分别生成无雾图像和透射率图, 并结合大气散射模型还原雾图像，以进一步提高图像分解的质量. 在马尔科夫判别器结构中，采用反向学习机制代替随机裁剪机制，以有效降低因采用随机裁剪算法而导致的判断结果不准确的概率. 在原有的损失函数上，加入雾霾损失函数，提高图像转化的质量. 在STOS和NYU数据集上进行消融实验和对比实验. 大量实验表明所提出方法在PSNR和SSIM指标上比原算法Pix2pix GAN有所提高, 且均优于现有去雾算法，复原图像具有清晰度高、噪声低、纹理丰富的优点.

关键词： 生成对抗网络, 匹配图像, 去雾, 反向学习, 大气散射模型

Fig.1 Generator block framework of proposed model

Tab.1 Network parameters of generator of proposed model

Fig.2 One dimensional inverse point

Fig.3 Two dimensional inverse point

Fig.4 Schematic diagram of OBL-patchGAN

Tab.2 Discriminator network parameters

Fig.5 Framework of dehazing algorithm combined with atmospheric scattering model based on generative adversarial network

Fig.6 Examples of NYU and SOTS image datasets

Fig.7 Comparison of effects of different λ

Tab.3 Comparison of image indicators in different λ parameters

Tab.4 Comparison of image indicator in different loss functions

Fig.8 Comparison of images generated in different loss functions

Fig.9 Comparison of images generated in different network structure

Tab.5 Comparison of image indicators in different network structures

Fig.10 Comparison of generated images in different haze concentrations

Fig.11 Dehazing effect indicators of proposed algorithm in different haze concentrations

Fig.12 Comparison of generated images in different algorithms

Fig.13 Dehazing effect indicators in different algorithms


[1]	卜令兵, 袁静, 高爱臻, 等基于激光云高仪的雾霾过程探测[J]. 光子学报, 2014, 43 (9): 64- 69 BU Ling-bing, YUAN Jing, GAO Ai-zhen, et al Analysis of haze-fog events based on laser ceilometer[J]. Acta Photonics Sinica, 2014, 43 (9): 64- 69

[2]	曲晨, 毕笃彦基于懒惰随机游走的雾天含噪图像清晰化[J]. 光学学报, 2018, 38 (4): 103- 112 QU Chen, BI Du-yan Clearness for foggy and noisy image based on lazy random walk[J]. Acta Optica Sinica, 2018, 38 (4): 103- 112

[3]	MIN X, ZHAI G, GU K, et al Quality evaluation of image dehazing methods using synthetic hazy images[J]. IEEE Transactions on Multimedia, 2019, 21 (9): 2319- 2333 doi: 10.1109/TMM.2019.2902097

[4]	DOU Z, HAN Y, SHENG W, et al Image dehaze using alternating Laplacian and Beltrami regularizations[J]. Journal of Electronic Imaging, 2015, 24 (2): 023004 doi: 10.1117/1.JEI.24.2.023004

[5]	覃宏超, 李炎炎, 龙伟, 等基于引导滤波和暗原色先验理论透射率估值的视频去雾算法[J]. 浙江大学学报:工学版, 2018, 52 (7): 1302- 1309 QIN Hong-chao, LI Yan-yan, LONG Wei, et al Real-time video dehazing using guided filtering and transmissivity estimated based on dark channel prior theory[J]. Journal of Zhejiang University: Engineering Science, 2018, 52 (7): 1302- 1309

[6]	JIN W, MI Z, WU X, et al. Single image de-haze based on a new dark channel estimation method [C]// 2012 IEEE International Conference on Computer Science and Automation Engineering. Zhangjiajie: IEEE , 2012: 791-795.

[7]	ENGIN D, GENC A, KEMAL E H. Cycle-dehaze: enhanced cyclegan for single image dehazing [C]// Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition Workshops. Salt Lake City: IEEE , 2018: 825-833.

[8]	LI C, ZHAO X, ZHANG Z, et al Generative adversarial dehaze mapping nets[J]. Pattern Recognition Letters, 2019, 119: 238- 244 doi: 10.1016/j.patrec.2017.11.021

[9]	YANG D, SUN J. Proximal dehaze-net: a prior learning-based deep network for single image dehazing[C]// Proceedings of the European Conference on Computer Vision. Munich: IEEE, 2018: 702-717.

[10]	KUANAR S, RAO K R, MAHAPATRA D, et al. Night time haze and glow removal using deep dilated convolutional network [EB/OL]. [2021-03-01]. https://arxiv.53yu.com/ftp/arxiv/papers/1902/1902.00855.pdf.

[11]	HE K, SUN J, TANG X Single image haze removal using dark channel prior[J]. IEEE Transactions on Pattern Analysis and Machine Intelligence, 2010, 33 (12): 2341- 235

[12]	金仙力, 张威, 刘林峰基于引导滤波和自适应容差的图像去雾算法[J]. 通信学报, 2020, 41 (5): 27- 36 JIN Xian-li, ZHANG Wei, LIU Lin-feng Image defogging algorithm based on guided filtering and adaptive tolerance[J]. Journal of Communications, 2020, 41 (5): 27- 36 doi: 10.11959/j.issn.1000-436x.2020078

[13]	LI B, PENG X, WANG Z, et al. Aod-net: all-in-one dehazing network[C]// Proceedings of the IEEE International Conference on Computer Vision. Venice: IEEE, 2017: 4770-4778.

[14]	肖进胜, 申梦瑶, 雷俊锋, 等基于生成对抗网络的雾霾场景图像转换算法[J]. 计算机学报, 2020, 43 (1): 165- 176 XIAO Jin-sheng, SHEN Meng-yao, LEI Jun-feng, et al Image conversion algorithm for haze scenes based on generative adversarial networks[J]. Chinese Journal of Computers, 2020, 43 (1): 165- 176 doi: 10.11897/SP.J.1016.2020.00165

[15]	MIRZA M, OSINDERO S. Conditional generative adversarial nets [EB/OL]. [2021-03-01]. https://arxiv.53yu.com/pdf/1411.1784.pdf .

[16]	ISOLA P, ZHU J Y, ZHOU T, et al. Image-to-image translation with conditional adversarial networks[C]// Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition. Honolulu: IEEE, 2017: 1125-1134.

[17]	JIA Q, MA Z. Patch-based generative adversarial network for single image haze removal [C]// 2020 International Conference on Computer Engineering and Application. Guangzhou: IEEE, 2020: 882-886.

[18]	刘国, 吕群波, 刘扬阳基于自适应暗原色的单幅图像去雾算法[J]. 光子学报, 2018, 47 (2): 179- 186 LIU Guo, LV Qun-bo, LIU Yang-yang Single image dehazing algorithm based on adaptive dark channel prior[J]. Acta Photonica Sinica, 2018, 47 (2): 179- 186

[19]	WANG J B, HE N, ZHANG L L, et al Single image dehazing with a physical model and dark channel prior[J]. Neurocomputing, 2015, 149: 718- 728 doi: 10.1016/j.neucom.2014.08.005

[20]	PAN J, SUN D, PFISTER H, et al Deblurring images via dark channel prior[J]. IEEE Transactions on Pattern Analysis and Machine Intelligence, 2017, 40 (10): 2315- 2328

[21]	LI S, REN W, ZHANG J, et al Single image rain removal via a deep decomposition-composition network[J]. Computer Vision and Image Understanding, 2019, 186: 48- 57 doi: 10.1016/j.cviu.2019.05.003

[22]	RONNEBERGER O, FISCHER P, BROX T. U-net: convolutional networks for biomedical image segmentation [C]// International Conference on Medical Image Computing and Computer-assisted Intervention. Munich: Springer, 2015: 234-241.

[23]	TIZHOOSH H R. Opposition-based learning: a new scheme for machine intelligence[C]// International Conference on Computational Intelligence for Modelling. Vienna: IEEE, 2005, 1: 695-701.

[24]	KUMAR A, SHAIKH A M, LI Y, et al Pruning filters with L1-norm and capped L1-norm for CNN compression[J]. Applied Intelligence, 2021, 51 (2): 1152- 1160 doi: 10.1007/s10489-020-01894-y

[25]	KAMAL U, TONMOY T I, DAS S, et al Automatic traffic sign detection and recognition using SegU-Net and a modified Tversky loss function with L1-constraint[J]. IEEE Transactions on Intelligent Transportation Systems, 2019, 21 (4): 1467- 1479

[26]	LEE C P, LIN C J A study on L2-loss (squared hinge-loss) multiclass SVM[J]. Neural Computation, 2013, 25 (5): 1302- 1323 doi: 10.1162/NECO_a_00434

[27]	WANG L T, HOOVER N E, PORTER E H, et al. SSIM: a software levelized compiled-code simulator [C]// Proceedings of the 24th ACM/IEEE Design Automation Conference. New York: ACM, 1987: 2-8.

[28]	HUYNH-THU Q, GHANBARI M The accuracy of PSNR in predicting video quality for different video scenes and frame rates[J]. Telecommunication Systems, 2012, 49 (1): 35- 48 doi: 10.1007/s11235-010-9351-x

[29]	GAO Y, HU H M, LI B, et al Detail preserved single image dehazing algorithm based on airlight refinement[J]. IEEE Transactions on Multimedia, 2018, 21 (2): 351- 362

[30]	CAI B, XU X, JIA K, et al Dehazenet: an end-to-end system for single image haze removal[J]. IEEE Transactions on Image Processing, 2016, 25 (11): 5187- 5198 doi: 10.1109/TIP.2016.2598681

[31]	REN W, PAN J, ZHANG H, et al Single image dehazing via multi-scale convolutional neural networks with holistic edges[J]. International Journal of Computer Vision, 2020, 128 (1): 240- 259 doi: 10.1007/s11263-019-01235-8

[1]	Wen-chao BAI,Xi-xian HAN,Jin-bao WANG. Efficient approximate query processing framework based on conditional generative model[J]. Journal of ZheJiang University (Engineering Science), 2022, 56(5): 995-1005.

[2]	Li HE,Shan-min PANG. Face reconstruction from voice based on age-supervised learning and face prior information[J]. Journal of ZheJiang University (Engineering Science), 2022, 56(5): 1006-1016.

[3]	Pei-zhi WEN,Jun-mou CHEN,Yan-nan XIAO,Ya-yuan WEN,Wen-ming HUANG. Underwater image enhancement algorithm based on GAN and multi-level wavelet CNN[J]. Journal of ZheJiang University (Engineering Science), 2022, 56(2): 213-224.

[4]	Tong CHEN,Jian-feng GUO,Xin-zhong HAN,Xue-li XIE,Jian-xiang XI. Visible and infrared image matching method based on generative adversarial model[J]. Journal of ZheJiang University (Engineering Science), 2022, 56(1): 63-74.

[5]	Xue-yun CHEN,Xiao-qiao HUANG,Li XIE. Classification and detection method of blood cells images based on multi-scale conditional generative adversarial network[J]. Journal of ZheJiang University (Engineering Science), 2021, 55(9): 1772-1781.

[6]	Hui-ya HU,Shao-yan GAI,Fei-peng DA. Face frontalization based on generative adversarial network[J]. Journal of ZheJiang University (Engineering Science), 2021, 55(1): 116-123.

[7]	Kun LIU,Xi WEN,Min-ming HUANG,Xin-xin YANG,Jing-kun MAO. Solar cell defect enhancement method based on generative adversarial network[J]. Journal of ZheJiang University (Engineering Science), 2020, 54(4): 684-693.

[8]	Ran DUAN,Deng-wen ZHOU,Li-juan ZHAO,Xiao-liang CHAI. Image super-resolution reconstruction based on multi-scale feature mapping network[J]. Journal of ZheJiang University (Engineering Science), 2019, 53(7): 1331-1339.

[9]	Wan-liang WANG,Sheng-lan YANG,Yan-wei ZHAO,Zhuo-rong LI. Estimation of river surface flow velocity based on conditional boundary equilibrium generative adversarial network[J]. Journal of ZheJiang University (Engineering Science), 2019, 53(11): 2118-2128.

[10]	QIN Hong-chao, LI Yan-yan, LONG Wei, ZHAO Rui-peng. Real-time video dehazing using guided filtering and transmissivity estimated based on dark channel prior theory[J]. Journal of ZheJiang University (Engineering Science), 2018, 52(7): 1302-1309.

Viewed

Full text

Abstract

Cited

Shared

Discussed