Please wait a minute...
Journal of ZheJiang University (Engineering Science)  2023, Vol. 57 Issue (10): 1966-1976    DOI: 10.3785/j.issn.1008-973X.2023.10.006
    
Deep supervised network for change detection of remote sensing image
Xiao-ping YUAN(),Xiao-qian WANG,Xiang HE,Yang-ming HU
School of Information and Control Engineering, China University of Mining and Technology, Xuzhou 221116, China
Download: HTML     PDF(1529KB) HTML
Export: BibTeX | EndNote (RIS)      

Abstract  

A deep supervised network (DSNet) for remote sensing image change detection (CD) was built with the Unet++ to address the inadequate feature extraction and poor ability of remote sensing image CD in most fullly convolutional networks. A multi-scale residual module was designed to replace the conventional convolution layer. The problem of single feature extraction was resolved by combining the spatial and spectral properties of twin networks to obtain the semantic information of remote sensing images at various levels. The horizontal output layer of model was built to implement the deep supervision process of node feature aggregation from shallow to deep features at the decoding end of the model. The fusion results of different features with information differentiation were transmitted to the normalization-based attention module (NAM). The information weight of the changed region was enhanced without introducing additional parameters. Experimental results showed that the recall rate and accuracy of the proposed model in the remote sensing image CD task were 90.39% and 92.04%, respectively, and the parameter quantity and calculation quantity of the model were 6.38 M and 60 G. Comparise with different network models indicates that the proposed method has the advantages of high detection accuracy, fast speed and lightweight.



Key wordsimage segmentation      attention module      deep supervision      transverse output layer      lightweight      twin network     
Received: 29 April 2022      Published: 18 October 2023
CLC:  TP 391  
Fund:  国家科技支撑计划资助项目(2013BAK06B08);江苏省研究生科研与实践创新计划资助项目(SJCX20_0812)
Cite this article:

Xiao-ping YUAN,Xiao-qian WANG,Xiang HE,Yang-ming HU. Deep supervised network for change detection of remote sensing image. Journal of ZheJiang University (Engineering Science), 2023, 57(10): 1966-1976.

URL:

https://www.zjujournals.com/eng/10.3785/j.issn.1008-973X.2023.10.006     OR     https://www.zjujournals.com/eng/Y2023/V57/I10/1966


用于遥感图像变化检测的深度监督网络

为了解决大多数全卷积网络出现的特征提取单一、遥感图像变化检测(CD)能力差的问题,借助Unet++网络构建用于遥感图像CD的深度监督网络(DSNet). 设计多尺度残差模块替换传统卷积层,通过融合孪生网络双分支上的空间与光谱特性,获取遥感图像在不同层次间的语义信息,有效解决了特征提取单一的问题. 在模型解码端设计横向输出层,实现节点从低级向高级特征聚合的深度监督过程. 将具备信息差异化的不同特征融合结果传输至基于归一化的注意力模块(NAM)中. 在不引入额外参数的前提下增强了变化区域的信息权重. 实验结果表明,所提模型在遥感图像CD任务中的召回率和精度分别为90.39%和92.04%,模型的参数量和计算量为6.38 M和60 G. 与不同网络模型的对比表明,该方法具有检测精度高、速度快和轻量化等优点.


关键词: 图像分割,  注意力模块,  深度监督,  横向输出层,  轻量化,  孪生网络 
Fig.1 Overall structure of DSNet algorithm
Fig.2 Encoder structure diagram of DSNet
Fig.3 Structure diagram of MultiRes feature extraction module
Fig.4 Construction of X0,4 characteristic graph
Fig.5 Feature difference map generated by decoder
Fig.6 Structure diagram of deep supervision
Fig.7 Schematic diagram of LEVIR_CD dataset image clipping
方法 PR RE F1 Kappa par
FC-EF[8] 0.784 3 0.746 7 0.765 0 0.747 8 1.35
FC-Siam-conc[8] 0.841 2 0.836 4 0.838 7 0.820 1 1.55
FC-Siam-diff[8] 0.864 6 0.824 7 0.843 1 0.834 3 1.35
DASNet[19] 0.873 7 0.857 5 0.865 4 0.857 2 16.25
IFN[20] 0.903 5 0.881 3 0.892 1 0.877 8 35.72
SNUNet-C32[14] 0.910 1 0.890 1 0.899 9 0.894 5 12.03
DSNet(本研究) 0.920 4 0.903 9 0.912 0 0.891 3 6.38
Tab.1 Change detection results of different networks on LEVIR dataset
Fig.8 F1 values of different models
Fig.9 Building extraction results of different methods on LEVIR dataset
基线方法 Unet++ block 深度融合监督 PR RE F1
Unet++ × × 0.874 6 0.864 7 0.869 6
Unet++ × 0.889 1 0.878 9 0.883 9
Unet++ × 0.903 5 0.889 4 0.896 3
Unet++ 0.920 4 0.903 9 0.912 0
Tab.2 Influence of MultiRes block and deep fusion supervision on change detection indicators
Fig.10 F1 under different depth supervision modes
Fig.11 Comparison results of calculation amount of different algorithms
Fig.12 F1 of training and verification set on LEVIR dataset
[1]   GAMBA P, DEll'ACQUA F, LISINI G Change detection of multitemporal SAR data in urban areas combining feature-based and pixel-based techniques[J]. IEEE Transactions on Geoscience and Remote Sensing, 2006, 44 (10): 2820- 2827
doi: 10.1109/TGRS.2006.879498
[2]   HUSSAIN M, CHEN D, CHENG A, et al Change detection from remotely sensed images: from pixel-based to object-based approaches[J]. ISPRS Journal of Photogrammetry and Remote Sensing, 2013, 80: 91- 106
doi: 10.1016/j.isprsjprs.2013.03.006
[3]   CHEN J, CHEN J, LIAO A, et al Global land cover mapping at 30 m resolution: a POK-based operational approach[J]. ISPRS Journal of Photogrammetry and Remote Sensing, 2015, 103: 7- 27
doi: 10.1016/j.isprsjprs.2014.09.002
[4]   VOLPI M, TUIA D, BOVOLO F, et al Supervised change detection in VHR images using contextual information and support vector machines[J]. International Journal of Applied Earth Observation and Geoinformation, 2013, 20: 77- 85
doi: 10.1016/j.jag.2011.10.013
[5]   FENG W, SUI H, TU J, et al A novel change detection approach based on visual saliency and random forest from multi-temporal high-resolution remote-sensing images[J]. International Journal of Remote Sensing, 2018, 39 (22): 7998- 8021
doi: 10.1080/01431161.2018.1479794
[6]   YAN J, WANG L, SONG W, et al A time-series classification approach based on change detection for rapid land cover mapping[J]. ISPRS Journal of Photogrammetry and Remote Sensing, 2019, 158: 249- 262
doi: 10.1016/j.isprsjprs.2019.10.003
[7]   GU W, LV Z, HAO M Change detection method for remote sensing images based on an improved Markov random field[J]. Multimedia Tools and Applications, 2017, 76 (17): 17719- 17734
doi: 10.1007/s11042-015-2960-3
[8]   DAUDT R C, LE S B, BOULCH A. Fully convolutional siamese networks for change detection [C]// 25th IEEE International Conference on Image Processing. Athens: IEEE, 2018: 4063-4067.
[9]   PENG D, ZHANG Y, GUAN H End-to-end change detection for high resolution satellite images using improved UNet++[J]. Remote Sensing, 2019, 11 (11): 1382
doi: 10.3390/rs11111382
[10]   PENG X, ZHONG R, LI Z, et al Optical remote sensing image change detection based on attention mechanism and image difference[J]. IEEE Transactions on Geoscience and Remote Sensing, 2020, 59 (9): 7296- 7307
[11]   CHOPRA S, HADSELL R, LECUN Y. Learning a similarity metric discriminatively, with application to face verification [C]// IEEE Computer Society Conference on Computer Vision and Pattern Recognition. San Diego: IEEE, 2005, 1: 539-546.
[12]   ZHANG X, YUE Y, GAO W, et al. DifUnet++: a satellite images change detection network based on Unet++ and differential pyramid [EB/OL]. [2021-01-22]. http://dx.chinadoi.cn/10.1109/LGRS.2021.3049370.
[13]   郭海涛, 卢俊, 袁洲, 等 Siam-DeepLabv3+网络遥感影像语义变化检测方法[J]. 测绘科学技术学报, 2021, 38 (6): 597- 603
GUO Hai-tao, LU Jun, YUAN Zhou, et al Semantic change detection method for remote sensing images with Siam-DeepLabv3+ network[J]. Journal of Surveying and Mapping Science and Technology, 2021, 38 (6): 597- 603
doi: 10.3969/j.issn.1673-6338.2021.06.008
[14]   FANG S, LI K, SHAO J, et al. SNUNet-CD: a densely connected siamese network for change detection of VHR images[EB/OL]. [2021-02-17]. http://dx.chinadoi.cn/10.1109/LGRS.2021.3056416.
[15]   DONG J, ZHAO W, WANG S. Multiscale context aggregation network for building change detection using high resolution remote sensing images [EB/OL]. [2021-10-18]. https://ieeexplore.ieee.org/document/9578997.
[16]   SHI Q, LIU M, LI S, et al A deeply supervised attention metric-based network and an open aerial image dataset for remote sensing change detection[J]. IEEE Transactions on Geoscience and Remote Sensing, 2021, 60: 1- 16
[17]   AFAQ Y, MANOCHA A Analysis on change detection techniques for remote sensing applications: a review[J]. Ecological Informatics, 2021, 63: 101310
doi: 10.1016/j.ecoinf.2021.101310
[18]   LIU Y, SHAO Z, TENG Y, et al. Nam: normalization-based attention module [EB/OL]. [2021-11-25]. https://arxiv.org/abs/2111.12419.
[19]   CHEN J, YUAN Z, PENG J, et al DASNet: dual attentive fully convolutional siamese networks for change detection in high-resolution satellite images[J]. IEEE Journal of Selected Topics in Applied Earth Observations and Remote Sensing, 2020, 14: 1194- 1206
[20]   ZHANG C, YUE P, TAPETE D, et al A deeply supervised image fusion network for change detection in high resolution bi-temporal remote sensing images[J]. ISPRS Journal of Photogrammetry and Remote Sensing, 2020, 166: 183- 200
doi: 10.1016/j.isprsjprs.2020.06.003
[21]   MOLCHANOV P, TYREE S, KARRAS T, et al. Pruning convolutional neural networks for resource efficient inference [EB/OL]. [2022-04-29]. https://arxiv.org/abs/1611.06440.
[1] Wei-long HE,Ping WANG,Ai-hua ZHANG,Ting-ting LIANG,Qiang-jie MA. On-line quantitative detection of welding arc shape based on object detection[J]. Journal of ZheJiang University (Engineering Science), 2023, 57(9): 1903-1914.
[2] Xin JIN,Jian-jun ZHUANG,Zi-heng XU. Lightweight YOLOv5s network-based algorithm for identifying hazardous objects under vehicles[J]. Journal of ZheJiang University (Engineering Science), 2023, 57(8): 1516-1526.
[3] Hao-ran GUO,Ji-chang GUO,Yu-dong WANG. Lightweight semantic segmentation network for underwater image[J]. Journal of ZheJiang University (Engineering Science), 2023, 57(7): 1278-1286.
[4] Yan ZHANG,Jing-xue SUN,Ye-mei SUN,Shu-dong LIU,Chuan-qi WANG. Lightweight object detection based on split attention and linear transformation[J]. Journal of ZheJiang University (Engineering Science), 2023, 57(6): 1195-1204.
[5] Qing-lu MA,Jia-ping LU,Xiao-yao TANG,Xue-feng DUAN. Improved YOLOv5s flame and smoke detection method in road tunnels[J]. Journal of ZheJiang University (Engineering Science), 2023, 57(4): 784-794.
[6] Yao ZENG,Fa-qin GAO. Surface defect detection algorithm of electronic components based on improved YOLOv5[J]. Journal of ZheJiang University (Engineering Science), 2023, 57(3): 455-465.
[7] Qing-lin AI,Jia-hao YANG,Jing-rui CUI. Small target pedestrian detection based on adaptive proliferation data enhancement and global feature fusion[J]. Journal of ZheJiang University (Engineering Science), 2023, 57(10): 1933-1944.
[8] Wan-liang WANG,Tie-jun WANG,Jia-cheng CHEN,Wen-bo YOU. Medical image segmentation method combining multi-scale and multi-head attention[J]. Journal of ZheJiang University (Engineering Science), 2022, 56(9): 1796-1805.
[9] Kun HAO,Kuo WANG,Bei-bei WANG. Lightweight underwater biological detection algorithm based on improved Mobilenet-YOLOv3[J]. Journal of ZheJiang University (Engineering Science), 2022, 56(8): 1622-1632.
[10] Fei LI,Kun HU,Yong ZHANG,Wen-shan WANG,Hao JIANG. Multi-dimensional detection of longitudinal tearing of conveyor belt based on YOLOv4 of hybrid domain attention[J]. Journal of ZheJiang University (Engineering Science), 2022, 56(11): 2156-2167.
[11] Yun-zuo ZHANG,Wei GUO,Zhao-quan CAI,Wen-bo LI. Remote sensing image target detection combining multi-scale and attention mechanism[J]. Journal of ZheJiang University (Engineering Science), 2022, 56(11): 2215-2223.
[12] Rong ZHANG,Wei ZHANG. Fire detection algorithm based on improved GhostNet-FCOS[J]. Journal of ZheJiang University (Engineering Science), 2022, 56(10): 1891-1899.
[13] Dong-jie YANG,Xian-jun GAO,Shu-hao RAN,Guang-bin ZHANG,Ping WANG,Yuan-wei YANG. Building extraction based on multiple multiscale-feature fusion attention network[J]. Journal of ZheJiang University (Engineering Science), 2022, 56(10): 1924-1934.
[14] Xiao-ping YUAN,Xiang HE,Xiao-qian WANG,Yang-ming HU. Image segmentation algorithm based on multi-level feature adaptive fusion[J]. Journal of ZheJiang University (Engineering Science), 2022, 56(10): 1958-1966.
[15] Jiang-bo XU,Yuan-zhi WANG,Yu QI,Bao-hua CAO,Yong-zhen LUO,Chang-gen YAN,Xiao-hua YANG,Han BAO,Yu-zhou XIANG. Deformation characteristics of fiber-reinforced foam lightweight soil under cyclic loading and unloading[J]. Journal of ZheJiang University (Engineering Science), 2022, 56(1): 111-117.