Please wait a minute...
浙江大学学报(工学版)  2025, Vol. 59 Issue (6): 1119-1129    DOI: 10.3785/j.issn.1008-973X.2025.06.003
计算机技术     
基于多维协同注意力的双支特征联合去雾网络
杨燕(),晁丽鹏
兰州交通大学 电子与信息工程学院,甘肃 兰州 730070
A two-branch feature joint dehazing network based on multidimensional collaborative attention
Yan YANG(),Lipeng CHAO
School of Electronic and Information Engineering, Lanzhou Jiaotong University, Lanzhou 730070, China
 全文: PDF(4406 KB)   HTML
摘要:

针对雾天图像复原中单一特征提取网络难以协同增强内容与边缘特征的问题,将去雾任务分为内容信息提取和边缘特征恢复2个子任务,提出基于多维协同注意力的双支特征联合去雾网络. 在第1个分支构建密集连接的卷积块提取有雾图像中的多层次内容信息;在第2个分支采用级联的多尺度残差卷积块对图像的纹理细节进行恢复;通过图像重构模块对2种特征进行多尺度重构,实现不同特征信息的交换和增强,提升去雾效果. 在网络中引入注意力机制,同时在空间和像素上进行注意力交互建模,使网络能够高效率地学习有雾图像的主要特征. 实验结果表明,所提网络在多个数据集上的客观指标均优于大多数现有算法的;在去雾视觉效果上,所提网络能够实现高内容还原度,并完整保留纹理细节.

关键词: 图像去雾图像恢复神经网络多维协同注意力特征重建    
Abstract:

The dehazing task was divided into two subtasks, content information extraction and edge feature restoration, and a two-branch feature joint dehazing network based on multidimensional collaborative attention was proposed, aiming at the problems that the single-feature extraction network had difficulty in collaboratively enhancing content and edge features in hazy image restoration. In the first branch of the network, densely connected convolutional blocks were constructed to extract multi-level content information from hazy images. In the second branch, cascaded multi-scale residual convolutional blocks were used to restore the texture details. Then the image reconstruction module was used to perform the multi-scale reconstruction on the two types of features, achieving the exchange and enhancement of different feature information, and improving the dehazing effect. Attention mechanism was introduced into the network and attention interactions were modeled in space and pixels to efficiently learn the main features of hazy images. The experimental results show that the proposed network outperforms most existing algorithms in objective metrics on multiple datasets, with high content restoration and complete texture details in dehazing visual effects.

Key words: image dehazing    image restoration    neural network    multidimensional collaborative attention    feature reconstruction
收稿日期: 2024-04-02 出版日期: 2025-05-30
CLC:  TP 391.4  
基金资助: 国家自然科学基金资助项目(61561030,62063014);甘肃省高等学校产业支撑计划资助项目(2021CYZC-04);兰州交通大学研究生教改项目(JG201928).
作者简介: 杨燕(1972—)女,教授,博士, 研究数字图像处理、语音信号处理和智能信息处理. orcid.org/0000-0001-5338-0762. E-mail:yangyantd@mail.lzjtu.cn
服务  
把本文推荐给朋友
加入引用管理器
E-mail Alert
作者相关文章  
杨燕
晁丽鹏

引用本文:

杨燕,晁丽鹏. 基于多维协同注意力的双支特征联合去雾网络[J]. 浙江大学学报(工学版), 2025, 59(6): 1119-1129.

Yan YANG,Lipeng CHAO. A two-branch feature joint dehazing network based on multidimensional collaborative attention. Journal of ZheJiang University (Engineering Science), 2025, 59(6): 1119-1129.

链接本文:

https://www.zjujournals.com/eng/CN/10.3785/j.issn.1008-973X.2025.06.003        https://www.zjujournals.com/eng/CN/Y2025/V59/I6/1119

图 1  基于多维协同注意力的双支特征联合去雾网络的框架图
图 2  空洞卷积密集块(DRDB)的结构图
图 3  多维协同注意力(MCA)的结构图
图 4  级联卷积块(CCB)结构图
图 5  多尺度特征重构模块(MFEB)的结构图
模型PSNRSSIMM/106
base23.2510.9320.0202
base-523.3840.9310.0335
base-723.279.0.9290.0469
base-e26.9870.9440.0271
base-r27.9200.9560.1021
base-s24.5870.9430.1022
本研究算法28.3690.9610.1022
表 1  本研究方法不同模块消融实验的结果
图 6  消融实验去雾效果局部放大对比
模型PSNRSSIM
Model-R22.2510.958
Model-P24.3840.910
Model-C26.7450.934
本研究算法28.3690.961
表 2  不同边缘提取算子消融实验的结果
图 7  不同边缘特征算子下的去雾效果
图 8  不同算法在SOTS-outdoor上的去雾效果对比
图 9  不同算法在SOTS-indoor上的去雾效果对比
图 10  不同算法在Haze-RD上的去雾效果对比
图 11  不同算法在真实有雾图像上的去雾效果对比
图 12  本研究算法去雾效果的边缘细节对比图
方法SOTS-indoorSOTS-outdoorHaze-RD
PSNRSSIMPSNRSSIMPSNRSSIM
DCP15.7450.79115.2410.74511.3730.741
DehazeNet19.5700.86122.0780.88314.3600.763
AOD-Net16.1830.82318.7210.85415.2220.744
GCA-Net25.6520.95627.3070.90212.2860.602
UHD16.2400.78316.1540.80714.0430.778
SGID18.3600.90721.7430.66814.4730.782
DehazeFormer22.7580.89922.5720.94214.2420.780
DEA-Net27.2940.90719.2950.80613.2180.750
本研究算法23.4290.91028.3690.96116.0230.791
表 3  不同方法的客观评价指标对比
方法t/sM/106
DCP2.132(Matlab)
DehazeNet1.532(Matlab)0.008
AOD-Net0.0170.002
GCA-Net0.1630.703
SGID0.81413.867
DehazeFormer0.4622.514
本研究算法0.1030.102
表 4  不同方法的去雾时间及模型复杂度对比
1 吴迪, 朱青松 图像去雾的最新研究进展[J]. 自动化学报, 2015, 41 (2): 221- 239
WU Di, ZHU Qingsong The latest research progress of image dehazing[J]. Acta Automatica Sinica, 2015, 41 (2): 221- 239
2 TAREL J P, HAUTIÈRE N. Fast visibility restoration from a single color or gray level image [C]// IEEE 12th International Conference on Computer Vision. Kyoto: IEEE, 2009: 2201–2208.
3 MENG G, WANG Y, DUAN J, et al. Efficient image dehazing with boundary constraint and contextual regularization [C]// IEEE International Conference on Computer Vision. Sydney: IEEE, 2013: 617–624.
4 覃宏超, 李炎炎, 龙伟, 等 基于引导滤波和暗原色先验理论透射率估值的视频去雾算法[J]. 浙江大学学报: 工学版, 2018, 52 (7): 1302- 1309
QIN Hongchao, LI Yanyan, LONG Wei, et al Real-time video dehazing using guided filtering and transmissivity estimated based on dark channel prior theory[J]. Journal of Zhejiang University: Engineering Science, 2018, 52 (7): 1302- 1309
5 NAYAR S K, NARASIMHAN S G. Vision in bad weather [C]// 7th IEEE International Conference on Computer Vision. Kerkyra: IEEE, 1999: 820–827.
6 NARASIMHAN S G, NAYAR S K. Removing weather effects from monochrome images [C]// IEEE Computer Society Conference on Computer Vision and Pattern Recognition. Kauai: IEEE, 2001: II.
7 LIHE Z, HE J, YUAN Q, et al PhDnet: a novel physic-aware dehazing network for remote sensing images[J]. Information Fusion, 2024, 106: 102277
doi: 10.1016/j.inffus.2024.102277
8 HE K, SUN J, TANG X. Single image haze removal using dark channel prior [C]// IEEE Conference on Computer Vision and Pattern Recognition. Miami: IEEE, 2009: 1956–1963.
9 ZHU Q, MAI J, SHAO L A fast single image haze removal algorithm using color attenuation prior[J]. IEEE Transactions on Image Processing, 2015, 24 (11): 3522- 3533
doi: 10.1109/TIP.2015.2446191
10 CAI B, XU X, JIA K, et al DehazeNet: an end-to-end system for single image haze removal[J]. IEEE Transactions on Image Processing, 2016, 25 (11): 5187- 5198
11 LI B, PENG X, WANG Z, et al. AOD-net: all-in-one dehazing network [C]// IEEE International Conference on Computer Vision. Venice: IEEE, 2017: 4780–4788.
12 HE K, SUN J, TANG X Guided image filtering[J]. IEEE Transactions on Pattern Analysis and Machine Intelligence, 2013, 35 (6): 1397- 1409
doi: 10.1109/TPAMI.2012.213
13 张登银, 鞠铭烨, 王雪梅 一种基于暗通道先验的快速图像去雾算法[J]. 电子学报, 2015, 43 (7): 1437- 1443
ZHANG Dengyin, JU Mingye, WANG Xuemei A fast image daze removal algorithm using dark channel prior[J]. Acta Electronica Sinica, 2015, 43 (7): 1437- 1443
doi: 10.3969/j.issn.0372-2112.2015.07.029
14 JU M, DING C, GUO Y J, et al IDGCP: image dehazing based on gamma correction prior[J]. IEEE Transactions on Image Processing, 2019, 29: 3104- 3118
15 ZHANG J, TAO D FAMED-net: a fast and accurate multi-scale end-to-end dehazing network[J]. IEEE Transactions on Image Processing, 2020, 29: 72- 84
doi: 10.1109/TIP.2019.2922837
16 REN W, MA L, ZHANG J, et al. Gated fusion network for single image dehazing [C]// IEEE/CVF Conference on Computer Vision and Pattern Recognition. Salt Lake City: IEEE, 2018: 3253–3261.
17 LIU X, MA Y, SHI Z, et al. GridDehazeNet: attention-based multi-scale network for image dehazing [C]// IEEE/CVF International Conference on Computer Vision. Seoul: IEEE, 2019: 7313–7322.
18 DAS S D, DUTTA S. Fast deep multi-patch hierarchical network for nonhomogeneous image dehazing [C]// IEEE/CVF Conference on Computer Vision and Pattern Recognition Workshops. Seattle: IEEE, 2020: 1994–2001.
19 BALLA P K, KUMAR A, PANDEY R A 4-channelled hazy image input generation and deep learning-based single image dehazing[J]. Journal of Visual Communication and Image Representation, 2024, 100: 104099
doi: 10.1016/j.jvcir.2024.104099
20 QIN X, WANG Z, BAI Y, et al FFA-net: feature fusion attention network for single image dehazing[J]. AAAI Conference on Artificial Intelligence, 2020, 34 (7): 11908- 11915
doi: 10.1609/aaai.v34i07.6865
21 SHAO Y, LI L, REN W, et al. Domain adaptation for image dehazing [C]// IEEE/CVF Conference on Computer Vision and Pattern Recognition. Seattle: IEEE, 2020: 2805–2814.
22 SONG Y, HE Z, QIAN H, et al Vision transformers for single image dehazing[J]. IEEE Transactions on Image Processing, 2023, 32: 1927- 1941
doi: 10.1109/TIP.2023.3256763
23 YU Y, ZHANG Y, CHENG Z, et al MCA: multidimensional collaborative attention in deep convolutional neural networks for image recognition[J]. Engineering Applications of Artificial Intelligence, 2023, 126: 107079
doi: 10.1016/j.engappai.2023.107079
24 ZHANG Y, TIAN Y, KONG Y, et al. Residual dense network for image super-resolution [C]// IEEE/CVF Conference on Computer Vision and Pattern Recognition. Salt Lake City: IEEE, 2018: 2472–2481.
25 YAN Q, GONG D, SHI Q, et al. Attention-guided network for ghost-free high dynamic range imaging [C]// IEEE/CVF Conference on Computer Vision and Pattern Recognition. Long Beach: IEEE, 2019: 1751–1760.
26 杨爱萍, 刘瑾, 邢金娜, 等 基于内容特征和风格特征融合的单幅图像去雾网络[J]. 自动化学报, 2023, 49 (4): 769- 777
YANG Aiping, LIU Jin, XING Jinna, et al Content feature and style feature fusion network for single image dehazing[J]. Acta Automatica Sinica, 2023, 49 (4): 769- 777
27 WANG Z, BOVIK A C, SHEIKH H R, et al Image quality assessment: from error visibility to structural similarity[J]. IEEE Transactions on Image Processing, 2004, 13 (4): 600- 612
doi: 10.1109/TIP.2003.819861
28 HE T, ZHANG Z, ZHANG H, et al. Bag of tricks for image classification with convolutional neural networks [C]// IEEE/CVF Conference on Computer Vision and Pattern Recognition. Long Beach: IEEE, 2019: 558–567.
29 KINGMA D P, BA J, HAMMAD M M. Adam: a method for stochastic optimization [EB/OL]. (2014−12−22)[2024−02−03]. https://arxiv.org/abs/1412.6980v9.
30 LI B, REN W, FU D, et al Benchmarking single image dehazing and beyond[J]. IEEE Transactions on Image Processing, 2018, 1: 492- 505
31 MANNOS J, SAKRISON D The effects of a visual fidelity criterion of the encoding of images[J]. IEEE Transactions on Information Theory, 1974, 20 (4): 525- 536
doi: 10.1109/TIT.1974.1055250
32 ZHANG Y, DING L, SHARMA G. HazeRD: an outdoor scene dataset and benchmark for single image dehazing [C]// IEEE International Conference on Image Processing. Beijing: IEEE, 2017: 3205–3209.
33 CHEN D, HE M, FAN Q, et al. Gated context aggregation network for image dehazing and deraining [C]// IEEE Winter Conference on Applications of Computer Vision. Waikoloa Village: IEEE, 2019: 1375–1383.
34 ZHENG Z, REN W, CAO X, et al. Ultra-high-definition image dehazing via multi-guided bilateral learning [C]// IEEE/CVF Conference on Computer Vision and Pattern Recognition. Nashville: IEEE, 2021: 16180–16189.
35 BAI H, PAN J, XIANG X, et al Self-guided image dehazing using progressive feature fusion[J]. IEEE Transactions on Image Processing, 2022, 31: 1217- 1229
doi: 10.1109/TIP.2022.3140609
[1] 王爽,孙守锁,郭永存,胡泽永. 电磁混合式耦合器调隙装置多目标参数优化[J]. 浙江大学学报(工学版), 2025, 59(5): 1007-1017.
[2] 张梦瑶,周杰,李文婷,赵勇. 结合全局信息和局部信息的三维网格分割框架[J]. 浙江大学学报(工学版), 2025, 59(5): 912-919.
[3] 钱新宇,谢清林,陶功权,温泽峰. 基于多结构数据驱动的车轮扁疤定量识别方法[J]. 浙江大学学报(工学版), 2025, 59(4): 688-697.
[4] 袁沛,蒋君侠,马飞,金杰峰,来建良. 基于PCA和自联想神经网络的核环境冷挤压切割刀具状态监测[J]. 浙江大学学报(工学版), 2025, 59(3): 606-615.
[5] 刘宜成,杨迦凌,唐瑞,程靖. 柔性空间机器人预定义时间自适应滑模控制[J]. 浙江大学学报(工学版), 2025, 59(2): 351-361.
[6] 杨冰,徐楚阳,姚金良,向学勤. 基于单目RGB图像的三维手部姿态估计方法[J]. 浙江大学学报(工学版), 2025, 59(1): 18-26.
[7] 吴菲,陈嘉诚,王万良. 基于并行计算的计算智能综述[J]. 浙江大学学报(工学版), 2025, 59(1): 27-38.
[8] 赖凌轩,柳景青,周一粟,李秀娟. 基于时频卷积神经网络的供水管道漏损识别[J]. 浙江大学学报(工学版), 2025, 59(1): 196-204.
[9] 王海军,王涛,俞慈君. 基于递归量化分析的CFRP超声检测缺陷识别方法[J]. 浙江大学学报(工学版), 2024, 58(8): 1604-1617.
[10] 李劲业,李永强. 融合知识图谱的时空多图卷积交通流量预测[J]. 浙江大学学报(工学版), 2024, 58(7): 1366-1376.
[11] 徐满,张冬梅,余想,李江,吴益平. 基于多重分形的改进GRU滑坡位移预测模型[J]. 浙江大学学报(工学版), 2024, 58(7): 1407-1416.
[12] 宋娟,贺龙喜,龙会平. 基于深度学习的隧道衬砌多病害检测算法[J]. 浙江大学学报(工学版), 2024, 58(6): 1161-1173.
[13] 邢志伟,朱书杰,李彪. 基于改进图卷积神经网络的航空行李特征感知[J]. 浙江大学学报(工学版), 2024, 58(5): 941-950.
[14] 刘议丹,朱小飞,尹雅博. 基于异质图卷积神经网络的论点对抽取模型[J]. 浙江大学学报(工学版), 2024, 58(5): 900-907.
[15] 王骏骋,王法慧. 基于变权重PSO-Elman神经网络的路面附着系数估计[J]. 浙江大学学报(工学版), 2024, 58(3): 622-634.