Please wait a minute...
Journal of ZheJiang University (Engineering Science)  2023, Vol. 57 Issue (11): 2285-2293    DOI: 10.3785/j.issn.1008-973X.2023.11.016
    
Defect identification for catenary dropper line based on compositional zero-shot learning
Gui-mei GU1(),Yao-hua JIA1,Yan-hao ZHAO2,Wen-hui ZHANG2,Bing-xu YAN3
1. School of Automation and Electrical Engineering, Lanzhou Jiaotong University, Lanzhou 730070, China
2. China Railway Lanzhou Bureau Group Co. Ltd, Lanzhou 730030, China
3. China Railway Zhengzhou Bureau Group Co. Ltd, Zhengzhou 450015, China
Download: HTML     PDF(1081KB) HTML
Export: BibTeX | EndNote (RIS)      

Abstract  

Defect identification method for catenary dropper line based on compositional zero-shot learning was proposed, aiming at the problem of insufficient learning of model features and difficulty in effectively improving the recognition accuracy caused by the serious lack of image of catenary defects on site. The visual feature extraction module using ResNet-50 as the backbone network was used to extract image visual features. The pre-trained Word2Vec word vector was used to initialize the node features in the label combination graph. The dependence relationship between the nodes in the label combination graph was learned through the 2-layer graph convolutional networks, thereby optimizing the semantic features of the combined label nodes and improving the final recognition effect. The extracted visual features were matched with the semantic features of the optimized combined label nodes, and the similarity function was constructed to calculate the similarity score between the visual features of the image and the semantic features of the combined label. The prediction of the combined label was completed through the cross-entropy loss. The simulation results show that the proposed method has an average class detection accuracy of 93.5% for seen samples and 86.5% for unseen samples.



Key wordscatenary dropper      defect identification      compositional zero-shot learning      ResNet-50 network      graph convolution network      word vector     
Received: 12 January 2023      Published: 11 December 2023
CLC:  U 225.4  
  TP 391.9  
Fund:  甘肃省科技计划资助项目(20JR10RA216)
Cite this article:

Gui-mei GU,Yao-hua JIA,Yan-hao ZHAO,Wen-hui ZHANG,Bing-xu YAN. Defect identification for catenary dropper line based on compositional zero-shot learning. Journal of ZheJiang University (Engineering Science), 2023, 57(11): 2285-2293.

URL:

https://www.zjujournals.com/eng/10.3785/j.issn.1008-973X.2023.11.016     OR     https://www.zjujournals.com/eng/Y2023/V57/I11/2285


基于组合零样本学习的接触网吊弦线缺陷识别

目前现场接触网吊弦缺陷图像严重不足,导致模型特征学习不充分,识别准确率难以得到有效提高,为此提出基于组合零样本学习的接触网吊弦线缺陷识别方法. 采用以ResNet-50作为主干网络的视觉特征提取模块提取图像视觉特征;使用预训练的Word2Vec词向量对标签组合图中的节点特征进行初始化,并通过2层图卷积网络学习标签组合图中各节点之间的依赖关系,从而优化组合标签节点的语义特征,改善最终的识别效果;将提取到的视觉特征和优化后的组合标签节点的语义特征相对齐,构建相似度函数计算图像视觉特征与组合标签语义特征之间的相似度得分,并通过交叉熵损失完成图像组合标签的预测. 仿真实验结果表明:所提方法对可见类样本的类平均检测准确率为93.5%,对不可见类样本的类平均检测准确率为86.5%.


关键词: 接触网吊弦,  缺陷识别,  组合零样本学习,  ResNet-50网络,  图卷积网络,  词向量 
Fig.1 Framework of compositional zero-shot learning (CZSL) method
Fig.2 Label combination diagram based on data set of this study
Fig.3 Original dropping image and its histogram distribution
Fig.4 Dropping image after CLAHE enhancement and its histogram distribution
样本名称 样本类型 N
训练集 验证集 测试集
正常吊弦(normal dropper) 可见类样本 1500 200 500
松弛绞线(slack cable) 可见类样本 1500 200 500
断裂绞线(broken cable) 可见类样本 1500 200 500
松弛吊弦(slack dropper) 不可见类样本 0 100 100
断裂吊弦(broken dropper) 不可见类样本 0 100 100
总计 4500 800 1700
Tab.1 Sample types and quantities of dataset
网络层 参数 输出大小
conv1 7×7,64×64,stride2 112×112
conv2_x 3×3Max Pool,stride2 56×56
$ \left[\begin{array}{l}1\times 1,64\\ 3\times 3,64\\ 1\times 1,256\end{array}\right]\times 3 $
conv3_x $ \left[\begin{array}{l}1\times 1,128\\ 3\times 3,128\\ 1\times 1,512\end{array}\right]\times 4 $ 28×28
conv4_x $ \left[\begin{array}{l}1\times 1,256\\ 3\times 3,256\\ 1\times 1,1\;024\end{array}\right]\times 6 $ 14×14
conv5_x $ \left[\begin{array}{l}1\times 1,512\\ 3\times 3,512\\ 1\times 1,2\;048\end{array}\right]\times 3 $ 7×7
Tab.2 ResNet-50 backbone network parameters
α Accs/% Accu/% α Accs/% Accu/%
0 88.0 82.9 2 93.5 86.5
1 89.3 84.2 3 91.7 85.3
Tab.3 Comparison of algorithm performance under different self connected weights
Fig.5 Visualization diagram of adjacency matrix
L Accs/% Accu/%
2 93.5 86.5
4 77.2 63.1
6 56.6 46.7
Tab.4 Comparison of algorithm performance under different GCN layers
网络 Accs/% Accu/% M t/ms
ResNet-18 87.6 80.2 13 511 232 5.6
ResNet-50 93.5 86.5 27 022 408 17.2
ResNet-101 94.1 86.9 46 014 528 40.5
Vgg-16 88.2 78.3 152 274 752 55.4
Tab.5 Comparison of algorithm performance for different visual feature extraction networks
算法 Accs/% Accu/% H/%
TMN 87.0 61.3 71.9
LE+ 90.5 73.7 81.2
AttOp 88.5 75.7 81.6
SymNet 91.5 82.7 86.9
CZSL 93.5 86.5 89.9
Tab.6 Comparison of detection accuracy between CZSL and other algorithms
Fig.6 Training set loss curve between CZSL and other algorithms
算法 M t/ms
AttOp 24 394 837 14.7
LE+ 24 665 132 14.6
SymNet 26 434 081 20.4
TMN 24 337 733 30.0
CZSL 27 022 408 17.2
Tab.7 Comparison of network parameters between CZSL and other algorithms
Fig.7 Qualitative analysis of CZSL detection effect
[1]   胡碟. 基于深度学习的铁路接触网吊弦检测与识别[D]. 成都: 西南交通大学, 2020: 2.
HU Die. Detection and recognition of railway catenary dropper based on deep learning [D]. Chengdu: Southwest Jiaotong University, 2020: 2.
[2]   齐冬莲, 钱佳莹, 闫云凤, 等 一种基于 RefineDet 网络和霍夫变换的高速铁路接触网吊弦状态多尺度检测方法[J]. 电子与 信息学报, 2021, 43 (7): 2014- 2022
QI Dong-lian, QIAN Jia-ying, YAN Yun-feng, et al A multi-scale detection method for dropper states in high-speed-railway contact network based on RefineDet network and Hough transform[J]. Journal of Electronics and Information Technology, 2021, 43 (7): 2014- 2022
[3]   陈强, 彭继慎, 闫云凤, 等 基于 FCOS 和 ResNet50-F 的吊弦不受力识别方法[J]. 铁道学报, 2021, 43 (10): 36- 42
CHEN Qiang, PENG Ji-shen, YAN Yun-feng, et al Method based on FCOS and ResNet50-FL for identifying stressfree dropper[J]. Journal of the China Railway Society, 2021, 43 (10): 36- 42
[4]   余晓宁, 顾桂梅, 王阳萍, 等 基于Faster R-CNN的接触网吊弦故障检测方法[J]. 兰州交通大学学报, 2021, 40 (2): 58- 65
YU Xiao-ning, GU Gui-mei, WANG Yang-ping, et al Catenary dropper fault detection method based on faster R-CNN[J]. Journal of Lanzhou Jiaotong University, 2021, 40 (2): 58- 65
[5]   LAROCHELLE H, ERHAN D, BENGIO Y. Zerodata learning of new tasks [C]// Proceedings of the 23rd National Conference on Artificial Intelligence. Chicago: AAAI, 2008: 646–651.
[6]   LAMPERT C H, NICKISCH H, HARMELING S. Attribute based classification for zeroshot visual object categorization [J]. IEEE Transactions on Pattern Analysis and Machine Intelligence, 2014, 36(3): 453465.
[7]   FROME A, CORRADO G S, SHLENS J, et al. Devise: a deep visualsemantic embedding model [C]// Proceedings of the 26th International Conference on Neural Information Processing Systems. New York: NIPS, 2013, 2121-2129.
[8]   MIKOLOY T, SUTSKEVER I, CHEN K, et al. Distributed representations of words and phrases and their compositionality [C]// Proceedings of the 26th International Conference on Neural Information Processing Systems. New York: NIPS, 2013, 3111-3119.
[9]   KINGMA D P, WELLING M. Autoencoding variational bayes [EB/OL]. [2022-11-17]. https://arxiv.org/pdf/1312.6114.pdf.
[10]   HOFFMAN D D, RICHARDS W A Parts of recognition[J]. Cognition, 1984, 18 (1): 65- 96
[11]   BIEDERMAN I Recognition-by-components: a theory of human image understanding[J]. Psychological Review, 1987, 94 (2): 115
doi: 10.1037/0033-295X.94.2.115
[12]   MISRA I, GUPTA A, HEBERT M. From red wine to red tomato: composition with context [C]// 2017 IEEE Conference on Computer Vision and Pattern Recognition. Honolulu: IEEE, 2017: 1160-1169.
[13]   PURUSHWKAKAM S, NICKEL M, GUPTA A, et al. Task-driven modular networks for zero-shot compositional learning [C]// 2019 IEEE/CVF International Conference on Computer Vision. Seoul: IEEE, 2019: 3592-3601.
[14]   NAGARAJAN T, GRAUMAN K. Attributes as operators: factorizing unseen attribute-object compositions [C]// 2018 European Conference on Computer Vision. Munich: ECCV, 2018: 172-190.
[15]   LI Y L, XU Y, MAO X H, et al. Symmetry and group in attribute-object compositions [C]// 2020 IEEE/CVF Conference on Computer Vision and Pattern Recognition. Seattle: IEEE, 2020: 11313-11322.
[16]   KIPF T N, WELLING M. Semi-supervised classification with graph convolutional networks [EB/OL]. [2022-11-19]. https://arxiv.org/pdf/1609.02907.pdf.
[17]   王雪松, 荣小龙, 程玉虎, 等 基于自适应多尺度图卷积网络的多标签图像识别[J]. 控制与决策, 2022, 37 (7): 1737- 1744
WANG Xue-song, RONG Xiao-long, CHENG Yu-hu, et al Multi-label image recognition based on adaptive multi-scale graph convolutional network[J]. Control and Decision, 2022, 37 (7): 1737- 1744
[18]   GAO H Y, JI S W Graph U-Nets[J]. IEEE Transactions on Pattern Analysis and Machine Intelligence, 2019, 44 (9): 4948- 4960
[19]   PIZER S M, AMBURN E P, AUSTIN J D, et al Adaptive histogram equalization and its variations[J]. Computer Vision, Graphics, and Image Processing, 1987, 39 (3): 355- 368
doi: 10.1016/S0734-189X(87)80186-X
[20]   HAN Z Y, FU Z Y, CHEN S, et al. Contrastive embedding for generalized zero-shot learning[C]// 2021 IEEE/ CVF Conference on Computer Vision and Pattern Recognition. Nashville: IEEE, 2021: 2371-2381.
[21]   XIAN Y Q, SCHIELE B, AKATA Z. Zero-Shot Learning: the good, the bad and the ugly [C]// 2017 IEEE Conference on Computer Vision and Pattern Recognition. Honolulu: IEEE, 2017: 3077-3086.
[22]   胡文博, 邱实, 许馨月, 等 基于深度学习的钢轨伤损超声检测与分类[J]. 铁道学报, 2021, 43 (4): 108- 116
HU Wen-bo, QIU Shi, XU Xin-yue, et al Ultrasonic detection and classification for internal defect of rail based on deep learning[J]. Journal of the China Railway Society, 2021, 43 (4): 108- 116
[1] Kun LIU,Xiao-song YANG. Surface defect identification of cross scene strip based on unsupervised domain adaptation[J]. Journal of ZheJiang University (Engineering Science), 2023, 57(3): 477-485.
[2] Yan-nan ZHANG,Xiao-hong HUANG,Yan MA,Qun CONG. Method with recording text classification based on deep learning[J]. Journal of ZheJiang University (Engineering Science), 2020, 54(7): 1264-1271.
[3] WANG Kai-can, XU Ji, ZHAI Guo-fu. Defect identification method for aluminum plate based on electromagnetic acoustic technique[J]. Journal of ZheJiang University (Engineering Science), 2014, 48(11): 2031-2038.