Please wait a minute...
浙江大学学报(工学版)  2024, Vol. 58 Issue (2): 268-278    DOI: 10.3785/j.issn.1008-973X.2024.02.005
计算机技术、通信技术     
基于不变学习的真实雾霾去除方法
孟小哲(),冯钰新,苏卓*(),周凡
中山大学 计算机学院,中山大学深圳研究院,广东 广州 510000
Real-world dehazing method with invariant learning
Xiaozhe MENG(),Yuxin FENG,Zhuo SU*(),Fan ZHOU
School of Computer Science and Engineering, Research Institute of Sun Yat-sen University in Shenzhen, Sun Yat-sen University, Guangzhou 510000, China
 全文: PDF(7951 KB)   HTML
摘要:

针对有监督去雾方法面临的质量干扰问题,提出基于不变学习的真实雾霾图像去除方法.该方法使用傅里叶特征变换将网络提取的特征线性化表示,针对线性化特征进行全局加权并求解协方差,去除特征之间的相关性.不变学习使网络更加关注特征与去雾图像之间的本质关系,可以使网络获得稳定的跨域特征.分析并解释了数据和所提出方法的不同作用.该方法既实现了对大气光散射模型的改进,又构建了适用于真实雾霾场景的新数据集.通过大量实验,验证了该方法在真实世界的良好去雾效果.

关键词: 图像去雾质量干扰不变学习特征相关    
Abstract:

A real hazy image removal method based on invariant learning was proposed to solve the problem of quality interference in image dehazing. The Fourier feature transform was used to linearize the features extracted by the network. Then global weighting was performed and covariance was solved for the linearized features. The correlation between the features was removed. Invariant learning makes the network more concerned about the essential relationship between features and the dehazing image, which can enable the network to obtain stable cross-domain features. The different roles of the data and the proposed method were analyzed and explained. The improvement of the atmospheric scattering model was realized, and a new dataset suitable for real haze scenarios was constructed. The real-world dehazing effect of the method was verified through extensive experiments.

Key words: image dehazing    quality interference    invariant learning    feature correlation
收稿日期: 2023-07-20 出版日期: 2024-01-23
CLC:  TP 391  
基金资助: 深圳市科技计划资助项目(JCYJ20200109142612234);广东省基础与应用基础研究基金资助项目(2021A1515012313).
通讯作者: 苏卓     E-mail: mengxzh5@mail2.sysu.edu.cn;suzhuo3@mail.sysu.edu.cn
作者简介: 孟小哲(1990—),男,博士生,从事图像复原、迁移学习的研究. orcid.org/0009-0005-5734-1097. E-mail:mengxzh5@mail2.sysu.edu.cn
服务  
把本文推荐给朋友
加入引用管理器
E-mail Alert
作者相关文章  
孟小哲
冯钰新
苏卓
周凡

引用本文:

孟小哲,冯钰新,苏卓,周凡. 基于不变学习的真实雾霾去除方法[J]. 浙江大学学报(工学版), 2024, 58(2): 268-278.

Xiaozhe MENG,Yuxin FENG,Zhuo SU,Fan ZHOU. Real-world dehazing method with invariant learning. Journal of ZheJiang University (Engineering Science), 2024, 58(2): 268-278.

链接本文:

https://www.zjujournals.com/eng/CN/10.3785/j.issn.1008-973X.2024.02.005        https://www.zjujournals.com/eng/CN/Y2024/V58/I2/268

图 1  不同去雾方法在真实数据上的去雾结果
图 2  使用ASM合成的数据差异对比
图 3  基于不变学习的去雾网络结构图
图 4  不同方法在Fattal数据上的测试结果
图 5  不同方法在URHI和RTTS上的测试结果
方法NIMA[34]BIQME[32]PM2.5[33]
输入4.3250.504198.250
MSBDN[19]4.1400.548166.783
FFA[6]3.7180.524189.193
DehazeFlow[5]4.6870.530177.369
SGID[3]3.6940.545178.359
DAD[23]4.0050.561108.403
D4[31]4.6310.555163.386
IDE[21]4.5860.553150.240
PSD[22]4.3450.514151.900
SLA[24]4.5980.524138.242
本文方法4.7030.577137.436
表 1  不同方法在RTTS数据集上去雾结果的无参考图像质量评价
图 6  去雾时考虑图像亮度损失方法的去雾结果对比
图 7  不同方法在SOTS数据集和提出的DarkHaze数据集上的测试结果
方法DarkHazeSOTS
PSNR/dBSSIMPSNR/dBSSIM
MSBDN[19]16.050.733030.930.9780
SGID[3]17.020.753630.200.9754
D4[31]16.160.675625.820.9445
本文方法26.830.930126.040.9511
表 2  不同方法在DarkHaze和SOTS测试数据集上的PSNR和SSIM
方法Np/106FLOPs/109
MSBDN[19]36.85137.90
PSD[22]33.11220.60
SGID[3]13.87108.40
D4[31]10.702.25
本文方法31.4034.17
表 3  不同方法的复杂度对比
图 8  不同设置下的训练损失和PSNR变化情况
图 9  不同设定下的真实去雾结果
图 10  对输入图像交替进行增强和去雾的结果对比
1 ARYAN M, MURARI M, PRATIK N, et al ReViewNet: a fast and resource optimized network for enabling safe autonomous driving in hazy weather conditions[J]. IEEE Transactions on Intelligent Transportation Systems, 2021, 22: 4256- 4266
doi: 10.1109/TITS.2020.3013099
2 CAO Z, QIN Y, JIA L, et al Haze removal of railway monitoring images using multi-scale residual network[J]. IEEE Transactions on Intelligent Transportation Systems, 2020, 22 (12): 7460- 7473
3 BAI H, PAN J, XIANG X, et al Self-guided image dehazing using progressive feature fusion[J]. IEEE Transactions on Image Processing, 2022, 31: 1217- 1229
doi: 10.1109/TIP.2022.3140609
4 WU H, QU Y, LIN S, et al. Contrastive learning for compact single image dehazing [C]// Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition . [S. l. ]: IEEE, 2021: 10551-10560.
5 LI H, LI J, ZHAO D, et al. Dehazeflow: multi-scale conditional flow network for single image dehazing [C]// Proceedings of the 29th ACM International Conference on Multimedia . New York: ACM, 2021: 2577-2585.
6 QIN X, WANG Z, BAI Y, et al. FFA-Net: feature fusion attention network for single image dehazing [C]// Proceedings of the AAAI Conference on Artificial Intelligence . New York: AAAI, 2020: 11908-11915.
7 MA L, MA T, LIU R, et al. Toward fast, flexible, and robust low-light image enhancement [C]// Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition . New Orleans: IEEE, 2022: 5637-5646.
8 ARJOVSKY M, BOTTOU L, GULRAJANI I, et al. Invariant risk minimization [EB/OL]. (2019-07-05). https://arxiv.org/pdf/1907.02893.pdf.
9 WANG J, LAN C, LIU C, et al Generalizing to unseen domains: a survey on domain generalization[J]. IEEE Transactions on Knowledge and Data Engineering, 2023, 35 (8): 8052- 8072
10 MUANDET K, BALDUZZI D, SCHÖLKOPF B. Domain generalization via invariant feature representation [C]/ /International Conference on Machine Learning . Atlanta: ACM, 2013: 10-18.
11 KUANG K, CUI P, ATHEY S, et al. Stable prediction across unknown environments [C]// Proceedings of the 24th ACM SIGKDD International Conference on Knowledge Discovery and Data Mining . London: ACM, 2018: 1617-1626.
12 KUANG K, XIONG R, CUI P, et al. Stable prediction with model misspecification and agnostic distribution shift [C]// Proceedings of the AAAI Conference on Artificial Intelligence . New York: AAAI, 2020: 4485-4492.
13 ZHANG X, CUI P, XU R, et al. Deep stable learning for out-of-distribution generalization [C]// Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition .[S. l. ]: IEEE, 2021: 5372-5382.
14 HE K, SUN J, TANG X Single image haze removal using dark channel prior[J]. IEEE Transactions on Pattern Analysis and Machine Intelligence, 2010, 33 (12): 2341- 2353
15 FATTAL R Dehazing using color-lines[J]. ACM Transactions on Graphics, 2014, 34 (1): 1- 14
16 BERMAN D, AVIDAN S. Non-local image dehazing [C]// Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition . Las Vegas: IEEE, 2016: 1674-1682.
17 LI B, REN W, FU D, et al Benchmarking single-image dehazing and beyond[J]. IEEE Transactions on Image Processing, 2018, 28 (1): 492- 505
18 LIU X, MA Y, SHI Z, et al. Griddehazenet: attention-based multi-scale network for image dehazing [C]// Proceedings of the IEEE/CVF International Conference on Computer Vision . Montreal: IEEE, 2019: 7314-7323.
19 DONG H, PAN J, XIANG L, et al. Multi-scale boosted dehazing network with dense feature fusion [C]// Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition . Seattle: IEEE, 2020: 2157-2167.
20 KIM G, KWON J Deep illumination-aware dehazing with low-light and detail enhancement[J]. IEEE Transactions on Intelligent Transportation Systems, 2021, 23 (3): 2494- 2508
21 JU M, DING C, REN W, et al IDE: image dehazing and exposure using an enhanced atmospheric scattering model[J]. IEEE Transactions on Image Processing, 2021, 30: 2180- 2192
doi: 10.1109/TIP.2021.3050643
22 CHEN Z, WANG Y, YANG Y, et al. PSD: principled synthetic-to-real dehazing guided by physical priors [C]// Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition . [S. l. ]: IEEE, 2021: 7180-7189.
23 SHAO Y, LI L, REN W, et al. Domain adaptation for image dehazing [C]// Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition . Seattle: IEEE, 2020: 2808-2817.
24 LIANG Y, WANG B, ZUO W, et al. Self-supervised learning and adaptation for single image dehazing [C]// Proceedings of the 31st International Joint Conference on Artificial Intelligence . Vienna: Morgan Kaufmann, 2022: 1-15.
25 LI R, PAN J, LI Z, et al. Single image dehazing via conditional generative adversarial network [C]// Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition . Salt Lake City: IEEE, 2018: 8202-8211.
26 NARASIMHAN S G, NAYAR S K Vision and the atmosphere[J]. International Journal of Computer Vision, 2002, 48 (3): 233
doi: 10.1023/A:1016328200723
27 YANG W, WANG W, HUANG H, et al Sparse gradient regularized deep Retinex network for robust low-light image enhancement[J]. IEEE Transactions on Image Processing, 2021, 30: 2072- 2086
doi: 10.1109/TIP.2021.3050850
28 ZEILER M D, FERGUS R. Visualizing and understanding convolutional networks [C]// European Conference Computer Vision . Zurich: Springer, 2014: 818-833.
29 GRAY R M, GOODMAN J W. Fourier transforms: an introduction for engineers [M]. Berlin: Springer, 2012: 53-113.
30 LI L, DONG Y, REN W, et al Semi-supervised image dehazing[J]. IEEE Transactions on Image Processing, 2019, 29: 2766- 2779
31 YANG Y, WANG C, LIU R, et al. Self-augmented unpaired image dehazing via density and depth decomposition [C]// Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition . New Orleans: IEEE, 2022: 2037-2046.
32 GU K, TAO D, QIAO J F, et al Learning a no-reference quality assessment model of enhanced images with big data[J]. IEEE Transactions on Neural Networks and Learning Systems, 2017, 29 (4): 1301- 1313
33 GU K, QIAO J, LI X Highly efficient picture-based prediction of PM2.5 concentration[J]. IEEE Transactions on Industrial Electronics, 2018, 66 (4): 3176- 3184
[1] 曾铮,贾晓红,辛士庆,严冬明. 精度可控的边界表示模型网格生成[J]. 浙江大学学报(工学版), 2024, 58(2): 257-267.
[2] 宋一然,周千寓,邵志文,易冉,马利庄. 基于动态采样对偶可变形网络的实时视频实例分割[J]. 浙江大学学报(工学版), 2024, 58(2): 247-256.
[3] 周姿含,王叙萌,陈为. 基于迁移学习的交互时序数据可视化生成方法[J]. 浙江大学学报(工学版), 2024, 58(2): 239-246.
[4] 曹寅,秦俊平,马千里,孙昊,闫凯,王磊,任家琪. 文本生成图像研究综述[J]. 浙江大学学报(工学版), 2024, 58(2): 219-238.
[5] 秦思怡,盖绍彦,达飞鹏. 混合采样下多级特征聚合的视频目标检测算法[J]. 浙江大学学报(工学版), 2024, 58(1): 10-19.
[6] 杨豚,郭永存,王爽,马鑫. 煤矿井下无人驾驶轨道电机车障碍物识别[J]. 浙江大学学报(工学版), 2024, 58(1): 29-39.
[7] 孙雪菲,张瑞峰,关欣,李锵. 强化先验骨架结构的轻量型高效人体姿态估计[J]. 浙江大学学报(工学版), 2024, 58(1): 50-60.
[8] 陈健松,蔡艺军. 面向垃圾分类场景的轻量化目标检测方案[J]. 浙江大学学报(工学版), 2024, 58(1): 71-77.
[9] 陈巧红,孙佳锦,漏杨波,方志坚. 基于多任务学习与层叠 Transformer 的多模态情感分析模型[J]. 浙江大学学报(工学版), 2023, 57(12): 2421-2429.
[10] 刘慧,王秀丽,沈跃,徐婕. 基于三维激光点云的苗圃场景多目标分类方法[J]. 浙江大学学报(工学版), 2023, 57(12): 2430-2438.
[11] 侯小虎,贾晓芬,赵佰亭. 眼底病变OCT图像的轻量化识别算法[J]. 浙江大学学报(工学版), 2023, 57(12): 2448-2455.
[12] 廖嘉文,庞彦伟,聂晶,孙汉卿,曹家乐. 基于特征融合和一致性损失的双目低光照增强[J]. 浙江大学学报(工学版), 2023, 57(12): 2456-2466.
[13] 杨恒,李卓,康忠元,田兵,董青. 基于循环神经网络的双目视觉物体6D位姿估计[J]. 浙江大学学报(工学版), 2023, 57(11): 2179-2187.
[14] 许丽丽,詹燕,鲁建厦,郎一丁. 四向穿梭车仓储系统复合作业调度优化[J]. 浙江大学学报(工学版), 2023, 57(11): 2188-2199.
[15] 顾桂梅,贾耀华,赵岩浩,张文辉,闫炳旭. 基于组合零样本学习的接触网吊弦线缺陷识别[J]. 浙江大学学报(工学版), 2023, 57(11): 2285-2293.