Please wait a minute...
浙江大学学报(工学版)  2026, Vol. 60 Issue (4): 791-799    DOI: 10.3785/j.issn.1008-973X.2026.04.011
计算机技术     
基于掩模和非局部注意力的双阶段去雨网络
侯玉珍1(),沈晓红1,*(),李莉1,杨明源1,张彩明2,3
1. 山东财经大学 计算机与人工智能学院,山东 济南 250014
2. 山东大学 软件学院,山东 济南 250101
3. 山东工商学院 山东省未来智能金融工程实验室,山东 烟台 264003
Dual-stage deraining network based on mask and non-local attention
Yuzhen HOU1(),Xiaohong SHEN1,*(),Li LI1,Mingyuan YANG1,Caiming ZHANG2,3
1. School of Computing and Artificial Intelligence, Shandong University of Finance and Economics, Jinan 250014, China
2. School of Software, Shandong University, Jinan 250101, China
3. Shandong Future Intelligent Finance Engineering Laboratory, Shandong Technology and Business University, Yantai 264003, China
 全文: PDF(4767 KB)   HTML
摘要:

针对单图像去雨网络中雨纹噪声干扰严重与现有注意力机制空间全局建模能力不足的问题,提出基于雨纹掩模抑制和非局部重建协同的双阶段图像去雨网络. 第1阶段构建雨纹掩模注意力机制,通过形态学操作生成雨纹掩模,在特征提取时选择性遮蔽有雨区域,提高模型抑制雨纹干扰的能力;第2阶段设计非局部注意力机制,利用基于特征聚类的非局部相似性度量方法引导像素重排,打破空间约束,增强滑动窗口注意力的远距离建模能力,提升去雨效果. 双阶段设计采用“雨纹抑制-细节重建”的递进优化,实现无雨图像的高质量重建. 在多个公开数据集上的实验表明,与其他网络相比,所提网络的峰值信噪比与结构相似性指标显著提升,在有效去除雨纹的同时更好地保留了图像细节信息,能获得视觉效果更自然、细节纹理更丰富的高质量复原图像.

关键词: 图像去雨Transformer雨纹掩模非局部注意力特征聚类    
Abstract:

A dual-stage image deraining network based on rain streak mask suppression and non-local reconstruction collaboration was proposed to address severe rain streak noise interference and insufficient spatial global modeling capability of existing attention mechanisms in single-image deraining networks. In the first stage of the network, a rain streak mask attention mechanism was designed, in which rain streak masks were generated through morphological operations, to enhance the model’s ability to suppress rain streak interference by selectively masking rain-affected regions during feature extraction. In the second stage, a non-local attention mechanism was devised by employing a feature clustering-based non-local similarity measurement method to guide pixel rearrangement, which broke spatial constraints, thereby augmenting the long-range modeling capability of the sliding window attention mechanism and improving the deraining performance. Through progressive optimization based on the dual-stage “rain streak suppression-detail reconstruction” process, high-quality reconstruction of rain-free images was achieved. Experimental results on multiple public datasets demonstrate that the proposed network achieves significant improvements in both PSNR and SSIM metrics compared to other networks, effectively removing rain streaks while better preserving image details and producing high-quality restored results with natural-looking appearance and fine-grained texture representations.

Key words: image deraining    Transformer    rain streak mask    non-local attention    feature clustering
收稿日期: 2025-07-14 出版日期: 2026-03-19
CLC:  TP 391.4  
基金资助: 国家自然科学基金资助项目(62202268); 中央引导地方科技发展资金资助项目(YDZX2023079); 教育部人文社科资助项目(22YJA630086); 山东省重点研发计划资助项目(2024TSGC0118).
通讯作者: 沈晓红     E-mail: houyuzhen921@163.com;xhshen@sdufe.edu.cn
作者简介: 侯玉珍(2001—),女,硕士生,从事图像处理研究. orcid.org/0009-0004-1967-7668. E-mail:houyuzhen921@163.com
服务  
把本文推荐给朋友
加入引用管理器
E-mail Alert
作者相关文章  
侯玉珍
沈晓红
李莉
杨明源
张彩明

引用本文:

侯玉珍,沈晓红,李莉,杨明源,张彩明. 基于掩模和非局部注意力的双阶段去雨网络[J]. 浙江大学学报(工学版), 2026, 60(4): 791-799.

Yuzhen HOU,Xiaohong SHEN,Li LI,Mingyuan YANG,Caiming ZHANG. Dual-stage deraining network based on mask and non-local attention. Journal of ZheJiang University (Engineering Science), 2026, 60(4): 791-799.

链接本文:

https://www.zjujournals.com/eng/CN/10.3785/j.issn.1008-973X.2026.04.011        https://www.zjujournals.com/eng/CN/Y2026/V60/I4/791

图 1  基于掩模和非局部注意力的双阶段去雨网络框架图
图 2  雨纹掩模生成模块
图 3  雨纹掩模注意力模块(MAB)结构图
图 4  非局部注意力模块(NAB)结构图
方法Rain200LRain200HDID-DataDDN-Data平均指标
PSNR/dBSSIMPSNR/dBSSIMPSNR/dBSSIMPSNR/dBSSIMPSNR/dBSSIM
GMM (2015)28.660.865214.500.416425.810.834427.550.847924.130.7410
DSC (2016)27.160.866314.730.381524.240.827927.310.837323.360.7333
DDN (2017)34.680.967126.050.805630.970.911630.000.904130.430.8971
PreNet (2019)37.800.981429.040.899133.170.948132.600.945933.150.9436
RCDNet (2020)39.170.988530.240.904834.080.953233.040.947234.130.9484
DualGCN (2021)40.730.988631.150.912534.370.962033.010.948934.810.9530
SPDNet (2021)40.500.987531.280.920734.570.956033.150.945734.880.9525
Restormer (2022)40.990.989032.000.934435.290.964134.200.957135.620.9612
DRSgormer (2023)41.230.989432.170.932635.350.964634.350.958835.780.9614
Regformer (2024)41.510.990032.460.935335.430.965134.380.959135.950.9624
NeRD-Rain(2024)41.710.990332.400.937335.530.965934.450.959636.020.9630
MMamba (2025)41.490.989532.430.934535.410.965534.460.959335.950.9622
本研究算法41.710.990733.230.934135.590.968034.540.961836.260.9636
表 1  不同方法的客观评价指标对比
图 5  不同算法在 DID-Data数据集上的去雨效果对比
图 6  不同算法在 Rain200H数据集上的去雨效果对比
模型编码器解码器PSNR/dBSSIM
不分阶段模型1STBSTB40.690.9852
模型2MABMAB40.990.9851
模型3NABNAB41.100.9862
分阶段模型4MABSTB41.410.9869
模型5STBNAB41.370.9871
本模型MABNAB41.710.9907
表 2  本研究方法不同模块消融实验的结果
图 7  消融实验去雨效果对比
2 汤红忠, 王翔, 张小刚, 等 面向单幅图像去雨的非相干字典学习及其稀疏表示研究[J]. 通信学报, 2017, 38 (7): 28- 35
TANG Hongzhong, WANG Xiang, ZHANG Xiaogang, et al Incoherent dictionary learning and sparse representation for single-image rain removal[J]. Journal on Communications, 2017, 38 (7): 28- 35
doi: 10.11959/j.issn.1000-436x.2017149
3 YANG W, TAN R T, WANG S, et al Single image deraining: from model-based to data-driven and beyond[J]. IEEE Transactions on Pattern Analysis and Machine Intelligence, 2021, 43 (11): 4059- 4077
doi: 10.1109/TPAMI.2020.2995190
4 CARION N, MASSA F, SYNNAEVE G, et al. End-to-end object detection with transformers [C]// The 16th European Conference on Computer Vision. Cham: Springer, 2020: 213–229.
5 HONG D, HAN Z, YAO J, et al SpectralFormer: rethinking hyperspectral image classification with transformers[J]. IEEE Transactions on Geoscience and Remote Sensing, 2022, 60: 1- 15
6 杨军, 张琛 基于边界点估计与稀疏卷积神经网络的三维点云语义分割[J]. 浙江大学学报: 工学版, 2024, 58 (6): 1121- 1132
YANG Jun, ZHANG Chen Semantic segmentation of 3D point cloud based on boundary point estimation and sparse convolution neural network[J]. Journal of Zhejiang University: Engineering Science, 2024, 58 (6): 1121- 1132
7 LI B, ZHANG Z, ZHENG H, et al. Diving deep into regions: exploiting regional information Transformer for single image deraining [EB/OL]. (2024-08-04) [2025 -07-01]. https://arxiv.org/abs/2402.16033.
8 ZENG K, LIN H, YAN Z, et al Non-local self-attention network for image super-resolution[J]. Applied Intelligence, 2024, 54 (7): 5336- 5352
doi: 10.1007/s10489-024-05343-y
9 ZHENG X, LIAO Y, GUO W, et al. Single-image-based rain and snow removal using multi-guided filter [C]// International Conference Neural Information Processing. Berlin, Heidelberg: Springer, 2013.
10 KIM J H, LEE C, SIM J Y, et al. Single-image deraining using an adaptive nonlocal means filter [C]// 2013 IEEE International Conference on Image Processing. Melbourne: IEEE, 2013: 914–917.
11 CHEN D Y, CHEN C C, KANG L W Visual depth guided color image rain streaks removal using sparse coding[J]. IEEE Transactions on Circuits and Systems for Video Technology, 2014, 24 (8): 1430- 1455
doi: 10.1109/TCSVT.2014.2308627
1 LIN C Y, TAO Z, XU A S, et al Sequential dual attention network for rain streak removal in a single image[J]. IEEE Transactions on Image Processing, 2020, 29: 6250- 6265
12 LUO Y, XU Y, JI H. Removing rain from a single image via discriminative sparse coding [C]// 2015 IEEE International Conference on Computer Vision (ICCV). Santiago: IEEE, 2015: 3397–3405.
13 LI Y, TAN R T, GUO X, et al. Rain streak removal using layer priors [C]// 2016 IEEE Conference on Computer Vision and Pattern Recognition (CVPR). Las Vegas: IEEE, 2016: 2736–2744.
14 FU X, HUANG J, DING X, et al Clearing the skies: a deep network architecture for single-image rain removal[J]. IEEE Transactions on Image Processing, 2017, 26 (6): 2944- 2956
doi: 10.1109/TIP.2017.2691802
15 FU X, HUANG J, ZENG D, et al. Removing rain from single images via a deep detail network [C]// 2017 IEEE Conference on Computer Vision and Pattern Recognition (CVPR). Honolulu: IEEE, 2017: 1715–1723.
16 REN D, ZUO W, HU Q, et al. Progressive image deraining networks: a better and simpler baseline [C]// 2019 IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR). Long Beach: IEEE, 2019: 3932-3941.
17 YANG W, TAN R T, FENG J, et al. Deep joint rain detection and removal from a single image [C]// 2017 IEEE Conference on Computer Vision and Pattern Recognition (CVPR). Honolulu: IEEE, 2017: 1685–1694.
18 WANG H, XIE Q, ZHAO Q, et al. A model-driven deep neural network for single image rain removal [C]// 2020 IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR). Seattle: IEEE, 2020: 3100–3109.
19 FU X, QI Q, ZHA Z, et al. Rain streak removal via dual graph convolutional network [C]// Proceedings of the AAAI Conference on Artificial Intelligence. [S. l. ]: AAAI Press, 2021: 1352–1360.
20 YI Q, LI J, DAI Q, et al. Structure-preserving deraining with residue channel prior guidance [C]// 2021 IEEE/CVF International Conference on Computer Vision (ICCV). Montreal: IEEE, 2021: 4218–4227.
21 DOAOVUTSKIY A, BEYER L, KOLESNIKOV A, et al. An image is worth 16x16 words: transformers for image recognition at scale [EB/OL]. (2021-06-03) [2025-07-01]. https://arxiv.org/abs/2010.11929.
22 LIU Z, LIN Y, CAO Y, et al. Swin transformer: hierarchical vision transformer using shifted windows [C]// 2021 IEEE/CVF International Conference on Computer Vision (ICCV). Montreal: IEEE, 2021: 9992–10002.
23 ZAMIR S W, ARORA A, KHAN S, et al. Restormer: efficient transformer for high-resolution image restoration [C]// 2022 IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR). New Orleans: IEEE, 2022: 5718–5729.
24 CHEN X, LI H, LI M, et al. Learning a sparse transformer network for effective image deraining [C]// 2023 IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR). Vancouver: IEEE, 2023: 5896–5905.
25 CHEN X, PAN J, DONG J. Bidirectional multi-scale implicit neural representations for image deraining [C]// 2024 IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR). Seattle: IEEE, 2024: 25627–25636.
26 OUYANG Z, LI W MMamba: enhancing image deraining with Morton curve-driven locality learning[J]. Neurocomputing, 2025, 638: 130161
doi: 10.1016/j.neucom.2025.130161
27 HE K, ZHANG X, REN S, et al. Deep residual learning for image recognition [C]// 2016 IEEE Conference on Computer Vision and Pattern Recognition (CVPR). Las Vegas: IEEE, 2016: 770–778.
28 SHI W, CABALLERO J, HUSZÁR F, et al. Real-time single image and video super-resolution using an efficient sub-pixel convolutional neural network [C]// 2016 IEEE Conference on Computer Vision and Pattern Recognition (CVPR). Las Vegas: IEEE, 2016: 1874–1883.
[1] 包晓安,彭书友,张娜,涂小妹,张庆琪,吴彪. 基于多方位感知深度融合检测头的目标检测算法[J]. 浙江大学学报(工学版), 2026, 60(1): 32-42.
[2] 孟璇,张雪英,孙颖,周雅茹. 基于电极排列和Transformer的脑电情感识别[J]. 浙江大学学报(工学版), 2025, 59(9): 1872-1880.
[3] 刘杰,吴优,田佳禾,韩轲. 改进Transformer的肺部CT图像超分辨率重建[J]. 浙江大学学报(工学版), 2025, 59(7): 1434-1442.
[4] 蔡永青,韩成,权巍,陈兀迪. 基于注意力机制的视觉诱导晕动症评估模型[J]. 浙江大学学报(工学版), 2025, 59(6): 1110-1118.
[5] 张梦瑶,周杰,李文婷,赵勇. 结合全局信息和局部信息的三维网格分割框架[J]. 浙江大学学报(工学版), 2025, 59(5): 912-919.
[6] 张德军,白燕子,曹锋,吴亦奇,徐战亚. 面向密集预测任务的点云Transformer适配器[J]. 浙江大学学报(工学版), 2025, 59(5): 920-928.
[7] 马莉,王永顺,胡瑶,范磊. 预训练长短时空交错Transformer在交通流预测中的应用[J]. 浙江大学学报(工学版), 2025, 59(4): 669-678.
[8] 张振利,胡新凯,李凡,冯志成,陈智超. 基于CNN和Efficient Transformer的多尺度遥感图像语义分割算法[J]. 浙江大学学报(工学版), 2025, 59(4): 778-786.
[9] 贾晓芬,王子祥,赵佰亭,梁镇洹,胡锐. 双维度交叉融合驱动的图像超分辨率重建方法[J]. 浙江大学学报(工学版), 2025, 59(12): 2516-2526.
[10] 杨燕,贾存鹏. 代理注意力下域特征交互的高效图像去雾算法[J]. 浙江大学学报(工学版), 2025, 59(12): 2527-2538.
[11] 刘宇轩,刘毅志,廖祝华,邹正标,汤璟昕. 面向动态交通流量预测的自适应图注意Transformer[J]. 浙江大学学报(工学版), 2025, 59(12): 2585-2592.
[12] 杨冰,徐楚阳,姚金良,向学勤. 基于单目RGB图像的三维手部姿态估计方法[J]. 浙江大学学报(工学版), 2025, 59(1): 18-26.
[13] 马现伟,范朝辉,聂为之,李东,朱逸群. 对失效传感器具备鲁棒性的故障诊断方法[J]. 浙江大学学报(工学版), 2024, 58(7): 1488-1497.
[14] 范康,钟铭恩,谭佳威,詹泽辉,冯妍. 联合语义分割和深度估计的交通场景感知算法[J]. 浙江大学学报(工学版), 2024, 58(4): 684-695.
[15] 温绍杰,吴瑞刚,冯超文,刘英莉. 基于Transformer的多模态级联文档布局分析网络[J]. 浙江大学学报(工学版), 2024, 58(2): 317-324.