Please wait a minute...
Journal of ZheJiang University (Engineering Science)  2024, Vol. 58 Issue (6): 1133-1141    DOI: 10.3785/j.issn.1008-973X.2024.06.004
    
Recognition of complex power lines based on novel encoder-decoder network
Yuntang LI(),Hengjie LI,Kun ZHANG,Binrui WANG,Shanyue GUAN,Yuan CHEN
College of Mechanical and Electrical Engineering, China Jiliang University, Hangzhou 310018, China
Download: HTML     PDF(2429KB) HTML
Export: BibTeX | EndNote (RIS)      

Abstract  

Accurate and quick recognition of multiple and intersecting complex power lines from images was achieved by constructing a novel encoder-decoder network. The first 16 layers of regular MobileNetV3 were taken by the encoder to reduce network parameters, and the coordinate attention mechanism was used to replace the squeeze and excitation attention mechanism of regular MobileNetV3 to obtain the channel and position information of feature maps. The multi-scale feature information of power lines was obtained by the decoder through the pyramid pooling module to improve the recognition accuracy. The encoder feature maps of 2nd, 4th, 7th, 11th and 13th layers were processed by the sharpened kernel convolution and were connected to the corresponding layers of decoder, enhancing the extraction of complex power lines edge features. The hybrid loss function was used to resolve the imbalance between classes of images with fewer power lines and more background pixels. The speed of network training was accelerated by transfer learning. Experimental results indicated that the mean pixel accuracy (MPA), mean intersection over union (MIOU) and recognition speed of the novel encoder-decoder network were 92.18%, 84.27%, and 32 frames per second, respectively, which were superior to those of PSPNet, U2Net, and other power lines recognition networks.



Key wordscomplex power lines      encoder-decoder network      MobileNetV3      attention mechanism      loss function     
Received: 14 August 2023      Published: 25 May 2024
CLC:  TP 391  
Fund:  浙江省属高校基本科研业务费专项资金资助项目(2020YW29).
Cite this article:

Yuntang LI,Hengjie LI,Kun ZHANG,Binrui WANG,Shanyue GUAN,Yuan CHEN. Recognition of complex power lines based on novel encoder-decoder network. Journal of ZheJiang University (Engineering Science), 2024, 58(6): 1133-1141.

URL:

https://www.zjujournals.com/eng/10.3785/j.issn.1008-973X.2024.06.004     OR     https://www.zjujournals.com/eng/Y2024/V58/I6/1133


基于新型编码解码网络的复杂输电线识别

构建新型编码解码网络实现图像中多根交叉复杂输电线的高精度快速识别. 为了减少网络参数,编码器取常规MobileNetV3的前16层,并且采用坐标注意力机制代替常规MobileNetV3的挤压和激励注意力机制,获取特征图的通道信息和位置信息;解码器通过金字塔池化模块获取输电线多尺度特征信息,提高识别精度;采用跳跃链接将编码器第2、4、7、11和13层特征图经锐化核卷积处理后分别与解码器的特征图堆叠,加强复杂输电线边缘特征提取;引入混合损失函数解决图像中输电线像素少、背景像素多的类别不平衡问题;利用迁移学习加快网络训练速度. 实验结果表明,新型编码解码网络的平均像素精度(MPA)、平均交并比(MIOU)和识别速度分别为92.18%、84.27%和32帧/s,优于PSPNet、U2Net和其他输电线识别网络的.


关键词: 复杂输电线,  编码解码网络,  MobileNetV3,  注意力机制,  损失函数 
Fig.1 Novel encoder-decoder network
Fig.2 Regular MobileNetV3
Fig.3 Bneck module
Fig.4 Coordinate attention module
Fig.5 Bneck-CA module
Fig.6 Pyramid pooling module
Fig.7 Visualization of feature maps at different depths
Fig.8 Mosaic data augmentation
Fig.9 Image mark of complex power lines
训练参数数值
Batch_size8
Initial_lr0.0001
Epoch500
CudaTrue
Tab.1 Training parameters of novel encoder-decoder network
Fig.10 Loss changing of network training
方法MPA/%MIOU/%FPS/(帧·s?1)
方法189.6381.7722
方法288.1780.9228
方法388.9381.3827
方法488.2780.8735
方法589.6882.1534
方法690.7282.7732
方法791.5883.3431
方法891.9784.0231
方法992.1884.2732
Tab.2 Comparison of ablation experiment results
Fig.11 Power line recognition results of five types of networks
网络MPA/%MIOU/%FPS/(帧·s?1)
PSPNet81.6173.789
U2Net89.7282.318
文献[12]90.2582.7529
文献[13]87.3779.6221
新型编码解码网络92.1884.2732
Tab.3 Comparison of five network recognition results
[22]   周登文, 田金月, 马路遥, 等 基于多级特征并联的轻量级图像语义分割[J]. 浙江大学学报:工学版, 2020, 54 (8): 1516- 1524
ZHOU Dengwen, TIAN Jinyue, MA luyao, et al Lightweight image semantic segmentation based on multi-level feature cascaded network[J]. Journal of Zhejiang University: Engineering Science, 2020, 54 (8): 1516- 1524
[23]   柴玉梅, 员武莲, 王黎明, 等 基于双注意力机制和迁移学习的跨领域推荐模型[J]. 计算机学报, 2020, 43 (10): 1924- 1942
CHAI Yumei, YUN Wulian, WANG Liming, et al A cross-domain recommendation model based on dual attention mechanism and transfer learning[J]. Chinese Journal of Computers, 2020, 43 (10): 1924- 1942
doi: 10.11897/SP.J.1016.2020.01924
[24]   QIN X, ZHANG Z, HUANG C, et al U2-Net: going deeper with nested U-structure for salient object detection[J]. Pattern Recognition, 2020, 106: 1- 12
[1]   YANG L, FAN J, LIU Y, et al A review on state-of-the-art power line inspection techniques[J]. IEEE Transactions on Instrumentation and Measurement, 2020, 69 (12): 9350- 9365
doi: 10.1109/TIM.2020.3031194
[2]   XU B, ZHAO Y, WANG T, et al Development of power transmission line detection technology based on unmanned aerial vehicle image vision[J]. SN Applied Sciences, 2023, 5 (3): 1- 15
[3]   张从新, 赵乐, 王先培 复杂地物背景下电力线的快速提取算法[J]. 武汉大学学报:工学版, 2018, 51 (8): 732- 739
ZHANG Congxin, ZHAO Le, WANG Xianpei Research on fast extraction algorithm of power line in complex ground object background[J]. Engineering Journal of Wuhan University, 2018, 51 (8): 732- 739
[4]   赵乐, 王先培, 代荡荡, 等 复杂背景下电力线自动提取算法[J]. 高电压技术, 2019, 45 (1): 218- 227
ZHAO Le, WANG Xianpei, DAI Dangdang, et al Automatic extraction algorithm of power line in complex background[J]. High Voltage Engineering, 2019, 45 (1): 218- 227
[5]   周封, 任贵新 基于颜色空间变量的输电线图像分类及特征提取[J]. 电力系统保护与控制, 2018, 46 (5): 89- 98
ZHOU Feng, REN Guixin Image classification and feature extraction of transmission line based on color space variable[J]. Power System Protection and Control, 2018, 46 (5): 89- 98
doi: 10.7667/PSPC170283
[6]   LI P, QIU R, WANG M, et al Online monitoring of overhead power lines against tree intrusion via a low-cost camera and mobile edge computing approach[J]. Journal of Physics: Conference Series, 2023, 2422 (1): 1- 17
[7]   ZHAO L, WANG X, YAO H, et al Power line extraction from aerial images using object-based markov random field with anisotropic weighted penalty[J]. IEEE Access, 2019, 7: 125333- 125356
doi: 10.1109/ACCESS.2019.2939025
[8]   田萱, 王亮, 丁琪 基于深度学习的图像语义分割方法综述[J]. 软件学报, 2019, 30 (2): 440- 468
TIAN Xuan, WANG Liang, DING Qi Review of image semantic segmentation based on deep learning[J]. Journal of Software, 2019, 30 (2): 440- 468
[9]   ZHOU Y, REN Y, XU E, et al Supervised semantic segmentation based on deep learning: a survey[J]. Multimedia Tools and Applications, 2022, 81 (20): 29283- 29304
doi: 10.1007/s11042-022-12842-y
[10]   许刚, 李果 轻量化航拍图像电力线语义分割[J]. 中国图象图形学报, 2021, 26 (11): 2605- 2618
XU Gang, LI Guo Research on lightweight neural network of aerial powerline image segmentation[J]. Journal of Image and Graphics, 2021, 26 (11): 2605- 2618
[11]   黄巨挺, 高宏力, 戴志坤 基于编码解码结构的移动端电力线语义分割方法[J]. 计算机应用, 2021, 41 (10): 2952- 2958
HUANG Juting, GAO Hongli, DAI Zhikun Semantic segmentation method of power line on mobile terminals based on encoder-decoder structure[J]. Journal of Computer Applications, 2021, 41 (10): 2952- 2958
doi: 10.11772/j.issn.1001-9081.2020122037
[12]   李运堂, 詹叶君, 王鹏峰, 等 基于新型编解码网络的复杂背景航拍图像输电线识别[J]. 传感技术学报, 2022, 35 (8): 1057- 1064
LI Yuntang, ZHAN Yejun, WANG Pengfeng, et al Power line recognition from aerial images with complex background based on new codec network[J]. Chinese Journal of Sensors and Actuators, 2022, 35 (8): 1057- 1064
[13]   JAFFARI R, HASHMANI M A, REYES-ALDASORO C C A novel focal phi loss for power line segmentation with auxiliary classifier U-Net[J]. Sensors, 2021, 21 (8): 1- 23
doi: 10.1109/JSEN.2021.3063942
[14]   LI B, CHEN C, DONG S, et al Transmission line detection in aerial images: an instance segmentation approach based on multitask neural networks[J]. Signal Processing: Image Communication, 2021, 96: 1- 9
[15]   YANG L, FAN J, XU S, et al Vision-based power line segmentation with an attention fusion network[J]. IEEE Sensors Journal, 2022, 22 (8): 8196- 8205
doi: 10.1109/JSEN.2022.3157336
[16]   YANG Y, HAN J Real-time object detector based MobileNetV3 for UAV applications[J]. Multimedia Tools and Applications, 2022, 82 (12): 18709- 18725
[17]   HOU Q, ZHOU D, FENG J. Coordinate attention for efficient mobile network design [C]// 2021 IEEE/CVF Conference on Computer Vision and Pattern Recognition . Nashville: IEEE, 2021: 13713−13722.
[18]   ZHANG Y, SIDIBE D, MOREL O, et al Deep multimodal fusion for semantic image segmentation: a survey[J]. Image and Vision Computing, 2021, 105: 1- 17
[19]   LIN T, GOYAL P, GIRSHICK R, et al Focal loss for dense object detection[J]. IEEE Transactions on Pattern Analysis and Machine Intelligence, 2020, 42 (2): 318- 327
doi: 10.1109/TPAMI.2018.2858826
[20]   KANG D, PARK S, PAIK J SdBAN: salient object detection using bilateral attention network with dice coefficient loss[J]. IEEE Access, 2020, 8: 104357- 104370
doi: 10.1109/ACCESS.2020.2999627
[1] Jun YANG,Chen ZHANG. Semantic segmentation of 3D point cloud based on boundary point estimation and sparse convolution neural network[J]. Journal of ZheJiang University (Engineering Science), 2024, 58(6): 1121-1132.
[2] Zhiwei XING,Shujie ZHU,Biao LI. Airline baggage feature perception based on improved graph convolutional neural network[J]. Journal of ZheJiang University (Engineering Science), 2024, 58(5): 941-950.
[3] Yi LIU,Yidan CHEN,Lin GAO,Jiao HONG. Lightweight road extraction model based on multi-scale feature fusion[J]. Journal of ZheJiang University (Engineering Science), 2024, 58(5): 951-959.
[4] Cuiting WEI,Weijian ZHAO,Bochao SUN,Yunyi LIU. Intelligent rebar inspection based on improved Mask R-CNN and stereo vision[J]. Journal of ZheJiang University (Engineering Science), 2024, 58(5): 1009-1019.
[5] Hai HUAN,Yu SHENG,Chenxi GU. Global guidance multi-feature fusion network based on remote sensing image road extraction[J]. Journal of ZheJiang University (Engineering Science), 2024, 58(4): 696-707.
[6] Mingjun SONG,Wen YAN,Yizhao DENG,Junran ZHANG,Haiyan TU. Light-weight algorithm for real-time robotic grasp detection[J]. Journal of ZheJiang University (Engineering Science), 2024, 58(3): 599-610.
[7] Zhaoyi JIANG,Wenqin ZOU,Shenghao ZHENG,Chao SONG,Bailin YANG. Variable rate compression of point cloud based on scene flow[J]. Journal of ZheJiang University (Engineering Science), 2024, 58(2): 279-287.
[8] Xinhua YAO,Tao YU,Senwen FENG,Zijian MA,Congcong LUAN,Hongyao SHEN. Recognition method of parts machining features based on graph neural network[J]. Journal of ZheJiang University (Engineering Science), 2024, 58(2): 349-359.
[9] Siyi QIN,Shaoyan GAI,Feipeng DA. Video object detection algorithm based on multi-level feature aggregation under mixed sampler[J]. Journal of ZheJiang University (Engineering Science), 2024, 58(1): 10-19.
[10] Hai-feng LI,Xue-ying ZHANG,Shu-fei DUAN,Hai-rong JIA,Hui-zhi LIANG. Fusing generative adversarial network and temporal convolutional network for Mandarin emotion recognition[J]. Journal of ZheJiang University (Engineering Science), 2023, 57(9): 1865-1875.
[11] Xiao-qiang ZHAO,Ze WANG,Zhao-yang SONG,Hong-mei JIANG. Image super-resolution reconstruction based on dynamic attention network[J]. Journal of ZheJiang University (Engineering Science), 2023, 57(8): 1487-1494.
[12] Hui-xin WANG,Xiang-rong TONG. Research progress of recommendation system based on knowledge graph[J]. Journal of ZheJiang University (Engineering Science), 2023, 57(8): 1527-1540.
[13] Xiu-lan SONG,Zhao-hang DONG,Hang-guan SHAN,Wei-jie LU. Vehicle trajectory prediction based on temporal-spatial multi-head attention mechanism[J]. Journal of ZheJiang University (Engineering Science), 2023, 57(8): 1636-1643.
[14] Xiao-yan LI,Peng WANG,Jia GUO,Xue LI,Meng-yu SUN. Multi branch Siamese network target tracking based on double attention mechanism[J]. Journal of ZheJiang University (Engineering Science), 2023, 57(7): 1307-1316.
[15] Wei QUAN,Yong-qing CAI,Chao WANG,Jia SONG,Hong-kai SUN,Lin-xuan LI. VR sickness estimation model based on 3D-ResNet two-stream network[J]. Journal of ZheJiang University (Engineering Science), 2023, 57(7): 1345-1353.