Please wait a minute...
浙江大学学报(工学版)
Journal of ZheJiang University (Engineering Science)  2024, Vol. 58 Issue (2): 268-278    DOI: 10.3785/j.issn.1008-973X.2024.02.005
    
Real-world dehazing method with invariant learning
Xiaozhe MENG(),Yuxin FENG,Zhuo SU*(),Fan ZHOU
School of Computer Science and Engineering, Research Institute of Sun Yat-sen University in Shenzhen, Sun Yat-sen University, Guangzhou 510000, China
Download: HTML     PDF(7951KB) HTML
Export: BibTeX | EndNote (RIS)      

Abstract  

A real hazy image removal method based on invariant learning was proposed to solve the problem of quality interference in image dehazing. The Fourier feature transform was used to linearize the features extracted by the network. Then global weighting was performed and covariance was solved for the linearized features. The correlation between the features was removed. Invariant learning makes the network more concerned about the essential relationship between features and the dehazing image, which can enable the network to obtain stable cross-domain features. The different roles of the data and the proposed method were analyzed and explained. The improvement of the atmospheric scattering model was realized, and a new dataset suitable for real haze scenarios was constructed. The real-world dehazing effect of the method was verified through extensive experiments.

Key words： image dehazing      quality interference      invariant learning      feature correlation     
Received: 20 July 2023      Published: 23 January 2024
CLC:  TP 391  
Fund:  深圳市科技计划资助项目（JCYJ20200109142612234）；广东省基础与应用基础研究基金资助项目（2021A1515012313）.
Corresponding Authors: Zhuo SU     E-mail: mengxzh5@mail2.sysu.edu.cn;suzhuo3@mail.sysu.edu.cn
Service
E-mail this article
Add to my bookshelf
Add to citation manager
E-mail Alert
RSS
Articles by authors
Xiaozhe MENG
Yuxin FENG
Zhuo SU
Fan ZHOU
Cite this article:

Xiaozhe MENG,Yuxin FENG,Zhuo SU,Fan ZHOU. Real-world dehazing method with invariant learning. Journal of ZheJiang University (Engineering Science), 2024, 58(2): 268-278.

URL:

https://www.zjujournals.com/eng/10.3785/j.issn.1008-973X.2024.02.005     OR     https://www.zjujournals.com/eng/Y2024/V58/I2/268


基于不变学习的真实雾霾去除方法

针对有监督去雾方法面临的质量干扰问题,提出基于不变学习的真实雾霾图像去除方法.该方法使用傅里叶特征变换将网络提取的特征线性化表示，针对线性化特征进行全局加权并求解协方差，去除特征之间的相关性.不变学习使网络更加关注特征与去雾图像之间的本质关系，可以使网络获得稳定的跨域特征.分析并解释了数据和所提出方法的不同作用.该方法既实现了对大气光散射模型的改进，又构建了适用于真实雾霾场景的新数据集.通过大量实验，验证了该方法在真实世界的良好去雾效果.


关键词： 图像去雾,  质量干扰,  不变学习,  特征相关 
Fig.1 Results of different methods on real data
Fig.2 Differences comparison of synthesized data using ASM
Fig.3 Structure of the invariant learning dehazing network
Fig.4 Test results of different methods on Fattal’s data
Fig.5 Test results of different methods on URHI and RTTS datasets
方法NIMA[34]BIQME[32]PM2.5[33]
输入4.3250.504198.250
MSBDN[19]4.1400.548166.783
FFA[6]3.7180.524189.193
DehazeFlow[5]4.6870.530177.369
SGID[3]3.6940.545178.359
DAD[23]4.0050.561108.403
D4[31]4.6310.555163.386
IDE[21]4.5860.553150.240
PSD[22]4.3450.514151.900
SLA[24]4.5980.524138.242
本文方法4.7030.577137.436
Tab.1 No-reference image quality evaluation of different dehazing results methods on RTTS dataset
Fig.6 Comparison of dehazing results of different methods considering image brightness loss
Fig.7 Test results of different methods on SOTS dataset and proposed DarkHaze dataset
方法DarkHazeSOTS
PSNR/dBSSIMPSNR/dBSSIM
MSBDN[19]16.050.733030.930.9780
SGID[3]17.020.753630.200.9754
D4[31]16.160.675625.820.9445
本文方法26.830.930126.040.9511
Tab.2 PSNR and SSIM of different methods on DarkHaze and SOTS test datasets
方法Np/106FLOPs/109
MSBDN[19]36.85137.90
PSD[22]33.11220.60
SGID[3]13.87108.40
D4[31]10.702.25
本文方法31.4034.17
Tab.3 Complexity comparison of different methods
Fig.8 Training loss and PSNR variation under different settings
Fig.9 Dehazing result of different settings on real-world data
Fig.10 Comparison of results of alternating image enhancement and dehazing of input image
[1]   ARYAN M, MURARI M, PRATIK N, et al ReViewNet: a fast and resource optimized network for enabling safe autonomous driving in hazy weather conditions[J]. IEEE Transactions on Intelligent Transportation Systems, 2021, 22: 4256- 4266
doi: 10.1109/TITS.2020.3013099
[2]   CAO Z, QIN Y, JIA L, et al Haze removal of railway monitoring images using multi-scale residual network[J]. IEEE Transactions on Intelligent Transportation Systems, 2020, 22 (12): 7460- 7473
[3]   BAI H, PAN J, XIANG X, et al Self-guided image dehazing using progressive feature fusion[J]. IEEE Transactions on Image Processing, 2022, 31: 1217- 1229
doi: 10.1109/TIP.2022.3140609
[4]   WU H, QU Y, LIN S, et al. Contrastive learning for compact single image dehazing [C]// Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition . [S. l. ]: IEEE, 2021: 10551-10560.
[5]   LI H, LI J, ZHAO D, et al. Dehazeflow: multi-scale conditional flow network for single image dehazing [C]// Proceedings of the 29th ACM International Conference on Multimedia . New York: ACM, 2021: 2577-2585.
[6]   QIN X, WANG Z, BAI Y, et al. FFA-Net: feature fusion attention network for single image dehazing [C]// Proceedings of the AAAI Conference on Artificial Intelligence . New York: AAAI, 2020: 11908-11915.
[7]   MA L, MA T, LIU R, et al. Toward fast, flexible, and robust low-light image enhancement [C]// Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition . New Orleans: IEEE, 2022: 5637-5646.
[8]   ARJOVSKY M, BOTTOU L, GULRAJANI I, et al. Invariant risk minimization [EB/OL]. (2019-07-05). https://arxiv.org/pdf/1907.02893.pdf.
[9]   WANG J, LAN C, LIU C, et al Generalizing to unseen domains: a survey on domain generalization[J]. IEEE Transactions on Knowledge and Data Engineering, 2023, 35 (8): 8052- 8072
[10]   MUANDET K, BALDUZZI D, SCHÖLKOPF B. Domain generalization via invariant feature representation [C]/ /International Conference on Machine Learning . Atlanta: ACM, 2013: 10-18.
[11]   KUANG K, CUI P, ATHEY S, et al. Stable prediction across unknown environments [C]// Proceedings of the 24th ACM SIGKDD International Conference on Knowledge Discovery and Data Mining . London: ACM, 2018: 1617-1626.
[12]   KUANG K, XIONG R, CUI P, et al. Stable prediction with model misspecification and agnostic distribution shift [C]// Proceedings of the AAAI Conference on Artificial Intelligence . New York: AAAI, 2020: 4485-4492.
[13]   ZHANG X, CUI P, XU R, et al. Deep stable learning for out-of-distribution generalization [C]// Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition .[S. l. ]: IEEE, 2021: 5372-5382.
[14]   HE K, SUN J, TANG X Single image haze removal using dark channel prior[J]. IEEE Transactions on Pattern Analysis and Machine Intelligence, 2010, 33 (12): 2341- 2353
[15]   FATTAL R Dehazing using color-lines[J]. ACM Transactions on Graphics, 2014, 34 (1): 1- 14
[16]   BERMAN D, AVIDAN S. Non-local image dehazing [C]// Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition . Las Vegas: IEEE, 2016: 1674-1682.
[17]   LI B, REN W, FU D, et al Benchmarking single-image dehazing and beyond[J]. IEEE Transactions on Image Processing, 2018, 28 (1): 492- 505
[18]   LIU X, MA Y, SHI Z, et al. Griddehazenet: attention-based multi-scale network for image dehazing [C]// Proceedings of the IEEE/CVF International Conference on Computer Vision . Montreal: IEEE, 2019: 7314-7323.
[19]   DONG H, PAN J, XIANG L, et al. Multi-scale boosted dehazing network with dense feature fusion [C]// Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition . Seattle: IEEE, 2020: 2157-2167.
[20]   KIM G, KWON J Deep illumination-aware dehazing with low-light and detail enhancement[J]. IEEE Transactions on Intelligent Transportation Systems, 2021, 23 (3): 2494- 2508
[21]   JU M, DING C, REN W, et al IDE: image dehazing and exposure using an enhanced atmospheric scattering model[J]. IEEE Transactions on Image Processing, 2021, 30: 2180- 2192
doi: 10.1109/TIP.2021.3050643
[22]   CHEN Z, WANG Y, YANG Y, et al. PSD: principled synthetic-to-real dehazing guided by physical priors [C]// Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition . [S. l. ]: IEEE, 2021: 7180-7189.
[23]   SHAO Y, LI L, REN W, et al. Domain adaptation for image dehazing [C]// Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition . Seattle: IEEE, 2020: 2808-2817.
[24]   LIANG Y, WANG B, ZUO W, et al. Self-supervised learning and adaptation for single image dehazing [C]// Proceedings of the 31st International Joint Conference on Artificial Intelligence . Vienna: Morgan Kaufmann, 2022: 1-15.
[25]   LI R, PAN J, LI Z, et al. Single image dehazing via conditional generative adversarial network [C]// Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition . Salt Lake City: IEEE, 2018: 8202-8211.
[26]   NARASIMHAN S G, NAYAR S K Vision and the atmosphere[J]. International Journal of Computer Vision, 2002, 48 (3): 233
doi: 10.1023/A:1016328200723
[27]   YANG W, WANG W, HUANG H, et al Sparse gradient regularized deep Retinex network for robust low-light image enhancement[J]. IEEE Transactions on Image Processing, 2021, 30: 2072- 2086
doi: 10.1109/TIP.2021.3050850
[28]   ZEILER M D, FERGUS R. Visualizing and understanding convolutional networks [C]// European Conference Computer Vision . Zurich: Springer, 2014: 818-833.
[29]   GRAY R M, GOODMAN J W. Fourier transforms: an introduction for engineers [M]. Berlin: Springer, 2012: 53-113.
[30]   LI L, DONG Y, REN W, et al Semi-supervised image dehazing[J]. IEEE Transactions on Image Processing, 2019, 29: 2766- 2779
[31]   YANG Y, WANG C, LIU R, et al. Self-augmented unpaired image dehazing via density and depth decomposition [C]// Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition . New Orleans: IEEE, 2022: 2037-2046.
[32]   GU K, TAO D, QIAO J F, et al Learning a no-reference quality assessment model of enhanced images with big data[J]. IEEE Transactions on Neural Networks and Learning Systems, 2017, 29 (4): 1301- 1313
[33]   GU K, QIAO J, LI X Highly efficient picture-based prediction of PM2.5 concentration[J]. IEEE Transactions on Industrial Electronics, 2018, 66 (4): 3176- 3184
[1] JIA Wen-chao, HU Rong-gui, SHI Fan, XU Cheng-xi. Injection vulnerability threat detection method with multi-feature correlation[J]. Journal of ZheJiang University (Engineering Science), 2018, 52(3): 524-530.