Please wait a minute...
浙江大学学报(工学版)  2023, Vol. 57 Issue (10): 1966-1976    DOI: 10.3785/j.issn.1008-973X.2023.10.006
计算机技术、自动化技术     
用于遥感图像变化检测的深度监督网络
袁小平(),王小倩,何祥,胡杨明
中国矿业大学 信息与控制工程学院,江苏 徐州 221116
Deep supervised network for change detection of remote sensing image
Xiao-ping YUAN(),Xiao-qian WANG,Xiang HE,Yang-ming HU
School of Information and Control Engineering, China University of Mining and Technology, Xuzhou 221116, China
 全文: PDF(1529 KB)   HTML
摘要:

为了解决大多数全卷积网络出现的特征提取单一、遥感图像变化检测(CD)能力差的问题,借助Unet++网络构建用于遥感图像CD的深度监督网络(DSNet). 设计多尺度残差模块替换传统卷积层,通过融合孪生网络双分支上的空间与光谱特性,获取遥感图像在不同层次间的语义信息,有效解决了特征提取单一的问题. 在模型解码端设计横向输出层,实现节点从低级向高级特征聚合的深度监督过程. 将具备信息差异化的不同特征融合结果传输至基于归一化的注意力模块(NAM)中. 在不引入额外参数的前提下增强了变化区域的信息权重. 实验结果表明,所提模型在遥感图像CD任务中的召回率和精度分别为90.39%和92.04%,模型的参数量和计算量为6.38 M和60 G. 与不同网络模型的对比表明,该方法具有检测精度高、速度快和轻量化等优点.

关键词: 图像分割注意力模块深度监督横向输出层轻量化孪生网络    
Abstract:

A deep supervised network (DSNet) for remote sensing image change detection (CD) was built with the Unet++ to address the inadequate feature extraction and poor ability of remote sensing image CD in most fullly convolutional networks. A multi-scale residual module was designed to replace the conventional convolution layer. The problem of single feature extraction was resolved by combining the spatial and spectral properties of twin networks to obtain the semantic information of remote sensing images at various levels. The horizontal output layer of model was built to implement the deep supervision process of node feature aggregation from shallow to deep features at the decoding end of the model. The fusion results of different features with information differentiation were transmitted to the normalization-based attention module (NAM). The information weight of the changed region was enhanced without introducing additional parameters. Experimental results showed that the recall rate and accuracy of the proposed model in the remote sensing image CD task were 90.39% and 92.04%, respectively, and the parameter quantity and calculation quantity of the model were 6.38 M and 60 G. Comparise with different network models indicates that the proposed method has the advantages of high detection accuracy, fast speed and lightweight.

Key words: image segmentation    attention module    deep supervision    transverse output layer    lightweight    twin network
收稿日期: 2022-04-29 出版日期: 2023-10-18
CLC:  TP 391  
基金资助: 国家科技支撑计划资助项目(2013BAK06B08);江苏省研究生科研与实践创新计划资助项目(SJCX20_0812)
作者简介: 袁小平(1966—),男,教授,从事图像处理研究. orcid.org/0000-0002-7936-0070. E-mail: 1941@cumt.edu.cn
服务  
把本文推荐给朋友
加入引用管理器
E-mail Alert
作者相关文章  
袁小平
王小倩
何祥
胡杨明

引用本文:

袁小平,王小倩,何祥,胡杨明. 用于遥感图像变化检测的深度监督网络[J]. 浙江大学学报(工学版), 2023, 57(10): 1966-1976.

Xiao-ping YUAN,Xiao-qian WANG,Xiang HE,Yang-ming HU. Deep supervised network for change detection of remote sensing image. Journal of ZheJiang University (Engineering Science), 2023, 57(10): 1966-1976.

链接本文:

https://www.zjujournals.com/eng/CN/10.3785/j.issn.1008-973X.2023.10.006        https://www.zjujournals.com/eng/CN/Y2023/V57/I10/1966

图 1  DSNet算法的总体结构图
图 2  DSNet网络的编码器结构图
图 3  多尺度残差特征提取模块的结构图
图 4  X0,4特征图的构建
图 5  解码器生成的特征差异图
图 6  深度监督结构图
图 7  LEVIR_CD数据集图像裁剪示意图
方法 PR RE F1 Kappa par
FC-EF[8] 0.784 3 0.746 7 0.765 0 0.747 8 1.35
FC-Siam-conc[8] 0.841 2 0.836 4 0.838 7 0.820 1 1.55
FC-Siam-diff[8] 0.864 6 0.824 7 0.843 1 0.834 3 1.35
DASNet[19] 0.873 7 0.857 5 0.865 4 0.857 2 16.25
IFN[20] 0.903 5 0.881 3 0.892 1 0.877 8 35.72
SNUNet-C32[14] 0.910 1 0.890 1 0.899 9 0.894 5 12.03
DSNet(本研究) 0.920 4 0.903 9 0.912 0 0.891 3 6.38
表 1  LEVIR数据集上不同网络的变化检测结果
图 8  不同模型的F1值
图 9  不同方法在LEVIR数据集上的建筑物提取结果
基线方法 Unet++ block 深度融合监督 PR RE F1
Unet++ × × 0.874 6 0.864 7 0.869 6
Unet++ × 0.889 1 0.878 9 0.883 9
Unet++ × 0.903 5 0.889 4 0.896 3
Unet++ 0.920 4 0.903 9 0.912 0
表 2  MultiRes block和深度融合监督对变化检测指标的影响
图 10  不同深度监督方式下的F1
图 11  不同算法的计算量对比结果
图 12  在LEVIR上训练集和验证集的F1
1 GAMBA P, DEll'ACQUA F, LISINI G Change detection of multitemporal SAR data in urban areas combining feature-based and pixel-based techniques[J]. IEEE Transactions on Geoscience and Remote Sensing, 2006, 44 (10): 2820- 2827
doi: 10.1109/TGRS.2006.879498
2 HUSSAIN M, CHEN D, CHENG A, et al Change detection from remotely sensed images: from pixel-based to object-based approaches[J]. ISPRS Journal of Photogrammetry and Remote Sensing, 2013, 80: 91- 106
doi: 10.1016/j.isprsjprs.2013.03.006
3 CHEN J, CHEN J, LIAO A, et al Global land cover mapping at 30 m resolution: a POK-based operational approach[J]. ISPRS Journal of Photogrammetry and Remote Sensing, 2015, 103: 7- 27
doi: 10.1016/j.isprsjprs.2014.09.002
4 VOLPI M, TUIA D, BOVOLO F, et al Supervised change detection in VHR images using contextual information and support vector machines[J]. International Journal of Applied Earth Observation and Geoinformation, 2013, 20: 77- 85
doi: 10.1016/j.jag.2011.10.013
5 FENG W, SUI H, TU J, et al A novel change detection approach based on visual saliency and random forest from multi-temporal high-resolution remote-sensing images[J]. International Journal of Remote Sensing, 2018, 39 (22): 7998- 8021
doi: 10.1080/01431161.2018.1479794
6 YAN J, WANG L, SONG W, et al A time-series classification approach based on change detection for rapid land cover mapping[J]. ISPRS Journal of Photogrammetry and Remote Sensing, 2019, 158: 249- 262
doi: 10.1016/j.isprsjprs.2019.10.003
7 GU W, LV Z, HAO M Change detection method for remote sensing images based on an improved Markov random field[J]. Multimedia Tools and Applications, 2017, 76 (17): 17719- 17734
doi: 10.1007/s11042-015-2960-3
8 DAUDT R C, LE S B, BOULCH A. Fully convolutional siamese networks for change detection [C]// 25th IEEE International Conference on Image Processing. Athens: IEEE, 2018: 4063-4067.
9 PENG D, ZHANG Y, GUAN H End-to-end change detection for high resolution satellite images using improved UNet++[J]. Remote Sensing, 2019, 11 (11): 1382
doi: 10.3390/rs11111382
10 PENG X, ZHONG R, LI Z, et al Optical remote sensing image change detection based on attention mechanism and image difference[J]. IEEE Transactions on Geoscience and Remote Sensing, 2020, 59 (9): 7296- 7307
11 CHOPRA S, HADSELL R, LECUN Y. Learning a similarity metric discriminatively, with application to face verification [C]// IEEE Computer Society Conference on Computer Vision and Pattern Recognition. San Diego: IEEE, 2005, 1: 539-546.
12 ZHANG X, YUE Y, GAO W, et al. DifUnet++: a satellite images change detection network based on Unet++ and differential pyramid [EB/OL]. [2021-01-22]. http://dx.chinadoi.cn/10.1109/LGRS.2021.3049370.
13 郭海涛, 卢俊, 袁洲, 等 Siam-DeepLabv3+网络遥感影像语义变化检测方法[J]. 测绘科学技术学报, 2021, 38 (6): 597- 603
GUO Hai-tao, LU Jun, YUAN Zhou, et al Semantic change detection method for remote sensing images with Siam-DeepLabv3+ network[J]. Journal of Surveying and Mapping Science and Technology, 2021, 38 (6): 597- 603
doi: 10.3969/j.issn.1673-6338.2021.06.008
14 FANG S, LI K, SHAO J, et al. SNUNet-CD: a densely connected siamese network for change detection of VHR images[EB/OL]. [2021-02-17]. http://dx.chinadoi.cn/10.1109/LGRS.2021.3056416.
15 DONG J, ZHAO W, WANG S. Multiscale context aggregation network for building change detection using high resolution remote sensing images [EB/OL]. [2021-10-18]. https://ieeexplore.ieee.org/document/9578997.
16 SHI Q, LIU M, LI S, et al A deeply supervised attention metric-based network and an open aerial image dataset for remote sensing change detection[J]. IEEE Transactions on Geoscience and Remote Sensing, 2021, 60: 1- 16
17 AFAQ Y, MANOCHA A Analysis on change detection techniques for remote sensing applications: a review[J]. Ecological Informatics, 2021, 63: 101310
doi: 10.1016/j.ecoinf.2021.101310
18 LIU Y, SHAO Z, TENG Y, et al. Nam: normalization-based attention module [EB/OL]. [2021-11-25]. https://arxiv.org/abs/2111.12419.
19 CHEN J, YUAN Z, PENG J, et al DASNet: dual attentive fully convolutional siamese networks for change detection in high-resolution satellite images[J]. IEEE Journal of Selected Topics in Applied Earth Observations and Remote Sensing, 2020, 14: 1194- 1206
20 ZHANG C, YUE P, TAPETE D, et al A deeply supervised image fusion network for change detection in high resolution bi-temporal remote sensing images[J]. ISPRS Journal of Photogrammetry and Remote Sensing, 2020, 166: 183- 200
doi: 10.1016/j.isprsjprs.2020.06.003
21 MOLCHANOV P, TYREE S, KARRAS T, et al. Pruning convolutional neural networks for resource efficient inference [EB/OL]. [2022-04-29]. https://arxiv.org/abs/1611.06440.
[1] 何卫隆,王平,张爱华,梁婷婷,马强杰. 基于目标检测的焊接电弧形态在线定量检测[J]. 浙江大学学报(工学版), 2023, 57(9): 1903-1914.
[2] 金鑫,庄建军,徐子恒. 轻量化YOLOv5s网络车底危险物识别算法[J]. 浙江大学学报(工学版), 2023, 57(8): 1516-1526.
[3] 李晓艳,王鹏,郭嘉,李雪,孙梦宇. 基于双注意力机制的多分支孪生网络目标跟踪[J]. 浙江大学学报(工学版), 2023, 57(7): 1307-1316.
[4] 张艳,孙晶雪,孙叶美,刘树东,王传启. 基于分割注意力与线性变换的轻量化目标检测[J]. 浙江大学学报(工学版), 2023, 57(6): 1195-1204.
[5] 马庆禄,鲁佳萍,唐小垚,段学锋. 改进YOLOv5s的公路隧道烟火检测方法[J]. 浙江大学学报(工学版), 2023, 57(4): 784-794.
[6] 艾青林,杨佳豪,崔景瑞. 基于自适应增殖数据增强与全局特征融合的小目标行人检测[J]. 浙江大学学报(工学版), 2023, 57(10): 1933-1944.
[7] 王海军,马文来,张圣燕. 注意力聚集无锚框的孪生网络无人机跟踪算法[J]. 浙江大学学报(工学版), 2023, 57(10): 1945-1954.
[8] 王万良,王铁军,陈嘉诚,尤文波. 融合多尺度和多头注意力的医疗图像分割方法[J]. 浙江大学学报(工学版), 2022, 56(9): 1796-1805.
[9] 郝琨,王阔,王贝贝. 基于改进Mobilenet-YOLOv3的轻量级水下生物检测算法[J]. 浙江大学学报(工学版), 2022, 56(8): 1622-1632.
[10] 褚晶辉,史李栋,井佩光,吕卫. 适用于目标检测的上下文感知知识蒸馏网络[J]. 浙江大学学报(工学版), 2022, 56(3): 503-509.
[11] 李飞,胡坤,张勇,王文善,蒋浩. 基于混合域注意力YOLOv4的输送带纵向撕裂多维度检测[J]. 浙江大学学报(工学版), 2022, 56(11): 2156-2167.
[12] 张云佐,郭威,蔡昭权,李文博. 联合多尺度与注意力机制的遥感图像目标检测[J]. 浙江大学学报(工学版), 2022, 56(11): 2215-2223.
[13] 张融,张为. 基于改进GhostNet-FCOS的火灾检测算法[J]. 浙江大学学报(工学版), 2022, 56(10): 1891-1899.
[14] 杨栋杰,高贤君,冉树浩,张广斌,王萍,杨元维. 基于多重多尺度融合注意力网络的建筑物提取[J]. 浙江大学学报(工学版), 2022, 56(10): 1924-1934.
[15] 袁小平,何祥,王小倩,胡杨明. 基于多层级特征自适应融合的图像分割算法[J]. 浙江大学学报(工学版), 2022, 56(10): 1958-1966.