Please wait a minute...
浙江大学学报(工学版)  2023, Vol. 57 Issue (11): 2285-2293    DOI: 10.3785/j.issn.1008-973X.2023.11.016
电气工程     
基于组合零样本学习的接触网吊弦线缺陷识别
顾桂梅1(),贾耀华1,赵岩浩2,张文辉2,闫炳旭3
1. 兰州交通大学 自动化与电气工程学院,甘肃 兰州 730070
2. 中国铁路兰州局集团有限公司,甘肃 兰州 730030
3. 中国铁路郑州局集团有限公司,河南 郑州 450015
Defect identification for catenary dropper line based on compositional zero-shot learning
Gui-mei GU1(),Yao-hua JIA1,Yan-hao ZHAO2,Wen-hui ZHANG2,Bing-xu YAN3
1. School of Automation and Electrical Engineering, Lanzhou Jiaotong University, Lanzhou 730070, China
2. China Railway Lanzhou Bureau Group Co. Ltd, Lanzhou 730030, China
3. China Railway Zhengzhou Bureau Group Co. Ltd, Zhengzhou 450015, China
 全文: PDF(1081 KB)   HTML
摘要:

目前现场接触网吊弦缺陷图像严重不足,导致模型特征学习不充分,识别准确率难以得到有效提高,为此提出基于组合零样本学习的接触网吊弦线缺陷识别方法. 采用以ResNet-50作为主干网络的视觉特征提取模块提取图像视觉特征;使用预训练的Word2Vec词向量对标签组合图中的节点特征进行初始化,并通过2层图卷积网络学习标签组合图中各节点之间的依赖关系,从而优化组合标签节点的语义特征,改善最终的识别效果;将提取到的视觉特征和优化后的组合标签节点的语义特征相对齐,构建相似度函数计算图像视觉特征与组合标签语义特征之间的相似度得分,并通过交叉熵损失完成图像组合标签的预测. 仿真实验结果表明:所提方法对可见类样本的类平均检测准确率为93.5%,对不可见类样本的类平均检测准确率为86.5%.

关键词: 接触网吊弦缺陷识别组合零样本学习ResNet-50网络图卷积网络词向量    
Abstract:

Defect identification method for catenary dropper line based on compositional zero-shot learning was proposed, aiming at the problem of insufficient learning of model features and difficulty in effectively improving the recognition accuracy caused by the serious lack of image of catenary defects on site. The visual feature extraction module using ResNet-50 as the backbone network was used to extract image visual features. The pre-trained Word2Vec word vector was used to initialize the node features in the label combination graph. The dependence relationship between the nodes in the label combination graph was learned through the 2-layer graph convolutional networks, thereby optimizing the semantic features of the combined label nodes and improving the final recognition effect. The extracted visual features were matched with the semantic features of the optimized combined label nodes, and the similarity function was constructed to calculate the similarity score between the visual features of the image and the semantic features of the combined label. The prediction of the combined label was completed through the cross-entropy loss. The simulation results show that the proposed method has an average class detection accuracy of 93.5% for seen samples and 86.5% for unseen samples.

Key words: catenary dropper    defect identification    compositional zero-shot learning    ResNet-50 network    graph convolution network    word vector
收稿日期: 2023-01-12 出版日期: 2023-12-11
CLC:  U 225.4  
基金资助: 甘肃省科技计划资助项目(20JR10RA216)
作者简介: 顾桂梅(1970—),女,教授,从事接触网智能监测和故障监测诊断研究. orcid.org/0000-0003-0485-5535. E-mail: 386509464@qq.com
服务  
把本文推荐给朋友
加入引用管理器
E-mail Alert
作者相关文章  
顾桂梅
贾耀华
赵岩浩
张文辉
闫炳旭

引用本文:

顾桂梅,贾耀华,赵岩浩,张文辉,闫炳旭. 基于组合零样本学习的接触网吊弦线缺陷识别[J]. 浙江大学学报(工学版), 2023, 57(11): 2285-2293.

Gui-mei GU,Yao-hua JIA,Yan-hao ZHAO,Wen-hui ZHANG,Bing-xu YAN. Defect identification for catenary dropper line based on compositional zero-shot learning. Journal of ZheJiang University (Engineering Science), 2023, 57(11): 2285-2293.

链接本文:

https://www.zjujournals.com/eng/CN/10.3785/j.issn.1008-973X.2023.11.016        https://www.zjujournals.com/eng/CN/Y2023/V57/I11/2285

图 1  组合零样本学习(CZSL)方法框架图
图 2  基于数据集的标签组合图
图 3  原始吊弦图像及其直方图分布
图 4  CLAHE增强后吊弦图像及其直方图分布
样本名称 样本类型 N
训练集 验证集 测试集
正常吊弦(normal dropper) 可见类样本 1500 200 500
松弛绞线(slack cable) 可见类样本 1500 200 500
断裂绞线(broken cable) 可见类样本 1500 200 500
松弛吊弦(slack dropper) 不可见类样本 0 100 100
断裂吊弦(broken dropper) 不可见类样本 0 100 100
总计 4500 800 1700
表 1  数据集样本类型及数量
网络层 参数 输出大小
conv1 7×7,64×64,stride2 112×112
conv2_x 3×3Max Pool,stride2 56×56
$ \left[\begin{array}{l}1\times 1,64\\ 3\times 3,64\\ 1\times 1,256\end{array}\right]\times 3 $
conv3_x $ \left[\begin{array}{l}1\times 1,128\\ 3\times 3,128\\ 1\times 1,512\end{array}\right]\times 4 $ 28×28
conv4_x $ \left[\begin{array}{l}1\times 1,256\\ 3\times 3,256\\ 1\times 1,1\;024\end{array}\right]\times 6 $ 14×14
conv5_x $ \left[\begin{array}{l}1\times 1,512\\ 3\times 3,512\\ 1\times 1,2\;048\end{array}\right]\times 3 $ 7×7
表 2  ResNet-50主干网络参数
α Accs/% Accu/% α Accs/% Accu/%
0 88.0 82.9 2 93.5 86.5
1 89.3 84.2 3 91.7 85.3
表 3  不同自连接权重下算法性能对比
图 5  邻接矩阵可视化图
L Accs/% Accu/%
2 93.5 86.5
4 77.2 63.1
6 56.6 46.7
表 4  不同GCN层数时算法性能对比
网络 Accs/% Accu/% M t/ms
ResNet-18 87.6 80.2 13 511 232 5.6
ResNet-50 93.5 86.5 27 022 408 17.2
ResNet-101 94.1 86.9 46 014 528 40.5
Vgg-16 88.2 78.3 152 274 752 55.4
表 5  不同视觉特征提取网络下的算法性能对比
算法 Accs/% Accu/% H/%
TMN 87.0 61.3 71.9
LE+ 90.5 73.7 81.2
AttOp 88.5 75.7 81.6
SymNet 91.5 82.7 86.9
CZSL 93.5 86.5 89.9
表 6  CZSL与其他算法的检测准确率对比
图 6  CZSL与其他算法的训练集损失曲线
算法 M t/ms
AttOp 24 394 837 14.7
LE+ 24 665 132 14.6
SymNet 26 434 081 20.4
TMN 24 337 733 30.0
CZSL 27 022 408 17.2
表 7  CZSL与其他算法的网络参数对比
图 7  CZSL检测效果定性分析
1 胡碟. 基于深度学习的铁路接触网吊弦检测与识别[D]. 成都: 西南交通大学, 2020: 2.
HU Die. Detection and recognition of railway catenary dropper based on deep learning [D]. Chengdu: Southwest Jiaotong University, 2020: 2.
2 齐冬莲, 钱佳莹, 闫云凤, 等 一种基于 RefineDet 网络和霍夫变换的高速铁路接触网吊弦状态多尺度检测方法[J]. 电子与 信息学报, 2021, 43 (7): 2014- 2022
QI Dong-lian, QIAN Jia-ying, YAN Yun-feng, et al A multi-scale detection method for dropper states in high-speed-railway contact network based on RefineDet network and Hough transform[J]. Journal of Electronics and Information Technology, 2021, 43 (7): 2014- 2022
3 陈强, 彭继慎, 闫云凤, 等 基于 FCOS 和 ResNet50-F 的吊弦不受力识别方法[J]. 铁道学报, 2021, 43 (10): 36- 42
CHEN Qiang, PENG Ji-shen, YAN Yun-feng, et al Method based on FCOS and ResNet50-FL for identifying stressfree dropper[J]. Journal of the China Railway Society, 2021, 43 (10): 36- 42
4 余晓宁, 顾桂梅, 王阳萍, 等 基于Faster R-CNN的接触网吊弦故障检测方法[J]. 兰州交通大学学报, 2021, 40 (2): 58- 65
YU Xiao-ning, GU Gui-mei, WANG Yang-ping, et al Catenary dropper fault detection method based on faster R-CNN[J]. Journal of Lanzhou Jiaotong University, 2021, 40 (2): 58- 65
5 LAROCHELLE H, ERHAN D, BENGIO Y. Zerodata learning of new tasks [C]// Proceedings of the 23rd National Conference on Artificial Intelligence. Chicago: AAAI, 2008: 646–651.
6 LAMPERT C H, NICKISCH H, HARMELING S. Attribute based classification for zeroshot visual object categorization [J]. IEEE Transactions on Pattern Analysis and Machine Intelligence, 2014, 36(3): 453465.
7 FROME A, CORRADO G S, SHLENS J, et al. Devise: a deep visualsemantic embedding model [C]// Proceedings of the 26th International Conference on Neural Information Processing Systems. New York: NIPS, 2013, 2121-2129.
8 MIKOLOY T, SUTSKEVER I, CHEN K, et al. Distributed representations of words and phrases and their compositionality [C]// Proceedings of the 26th International Conference on Neural Information Processing Systems. New York: NIPS, 2013, 3111-3119.
9 KINGMA D P, WELLING M. Autoencoding variational bayes [EB/OL]. [2022-11-17]. https://arxiv.org/pdf/1312.6114.pdf.
10 HOFFMAN D D, RICHARDS W A Parts of recognition[J]. Cognition, 1984, 18 (1): 65- 96
11 BIEDERMAN I Recognition-by-components: a theory of human image understanding[J]. Psychological Review, 1987, 94 (2): 115
doi: 10.1037/0033-295X.94.2.115
12 MISRA I, GUPTA A, HEBERT M. From red wine to red tomato: composition with context [C]// 2017 IEEE Conference on Computer Vision and Pattern Recognition. Honolulu: IEEE, 2017: 1160-1169.
13 PURUSHWKAKAM S, NICKEL M, GUPTA A, et al. Task-driven modular networks for zero-shot compositional learning [C]// 2019 IEEE/CVF International Conference on Computer Vision. Seoul: IEEE, 2019: 3592-3601.
14 NAGARAJAN T, GRAUMAN K. Attributes as operators: factorizing unseen attribute-object compositions [C]// 2018 European Conference on Computer Vision. Munich: ECCV, 2018: 172-190.
15 LI Y L, XU Y, MAO X H, et al. Symmetry and group in attribute-object compositions [C]// 2020 IEEE/CVF Conference on Computer Vision and Pattern Recognition. Seattle: IEEE, 2020: 11313-11322.
16 KIPF T N, WELLING M. Semi-supervised classification with graph convolutional networks [EB/OL]. [2022-11-19]. https://arxiv.org/pdf/1609.02907.pdf.
17 王雪松, 荣小龙, 程玉虎, 等 基于自适应多尺度图卷积网络的多标签图像识别[J]. 控制与决策, 2022, 37 (7): 1737- 1744
WANG Xue-song, RONG Xiao-long, CHENG Yu-hu, et al Multi-label image recognition based on adaptive multi-scale graph convolutional network[J]. Control and Decision, 2022, 37 (7): 1737- 1744
18 GAO H Y, JI S W Graph U-Nets[J]. IEEE Transactions on Pattern Analysis and Machine Intelligence, 2019, 44 (9): 4948- 4960
19 PIZER S M, AMBURN E P, AUSTIN J D, et al Adaptive histogram equalization and its variations[J]. Computer Vision, Graphics, and Image Processing, 1987, 39 (3): 355- 368
doi: 10.1016/S0734-189X(87)80186-X
20 HAN Z Y, FU Z Y, CHEN S, et al. Contrastive embedding for generalized zero-shot learning[C]// 2021 IEEE/ CVF Conference on Computer Vision and Pattern Recognition. Nashville: IEEE, 2021: 2371-2381.
21 XIAN Y Q, SCHIELE B, AKATA Z. Zero-Shot Learning: the good, the bad and the ugly [C]// 2017 IEEE Conference on Computer Vision and Pattern Recognition. Honolulu: IEEE, 2017: 3077-3086.
22 胡文博, 邱实, 许馨月, 等 基于深度学习的钢轨伤损超声检测与分类[J]. 铁道学报, 2021, 43 (4): 108- 116
HU Wen-bo, QIU Shi, XU Xin-yue, et al Ultrasonic detection and classification for internal defect of rail based on deep learning[J]. Journal of the China Railway Society, 2021, 43 (4): 108- 116
[1] 申自浩,唐雨雨,王辉,刘沛骞,刘琨. 基于聚类和深度学习的车联网轨迹隐私保护机制[J]. 浙江大学学报(工学版), 2024, 58(1): 20-28.
[2] 孟闯,王慧. 多信息融合的时空图卷积交通流量预测模型[J]. 浙江大学学报(工学版), 2023, 57(8): 1541-1550.
[3] 程艳芬,吴家俊,何凡. 基于关系门控图卷积网络的方面级情感分析[J]. 浙江大学学报(工学版), 2023, 57(3): 437-445.
[4] 刘坤,杨晓松. 基于无监督域适应的跨场景带钢表面缺陷识别[J]. 浙江大学学报(工学版), 2023, 57(3): 477-485.
[5] 张京京,张兆功,许鑫. 融合图增强和采样策略的图卷积协同过滤模型[J]. 浙江大学学报(工学版), 2023, 57(2): 243-251.
[6] 侯越,韩成艳,郑鑫,邓志远. 基于时空融合图卷积的交通流数据修复方法[J]. 浙江大学学报(工学版), 2022, 56(7): 1394-1403.
[7] 郭策,曾志文,朱鹏铭,周智千,卢惠民. 基于图卷积模仿学习的分布式群集控制[J]. 浙江大学学报(工学版), 2022, 56(6): 1055-1061.
[8] 王友卫,童爽,凤丽洲,朱建明,李洋,陈福. 基于图卷积网络的归纳式微博谣言检测新方法[J]. 浙江大学学报(工学版), 2022, 56(5): 956-966.
[9] 王婷,朱小飞,唐顾. 基于知识增强的图卷积神经网络的文本分类[J]. 浙江大学学报(工学版), 2022, 56(2): 322-328.
[10] 陆佳炜,郑嘉弘,李端倪,徐俊,肖刚. 面向服务聚类的短文本优化主题模型[J]. 浙江大学学报(工学版), 2022, 56(12): 2416-2425.
[11] 钟帆,柏正尧. 采用动态残差图卷积的3D点云超分辨率[J]. 浙江大学学报(工学版), 2022, 56(11): 2251-2259.
[12] 张彦楠,黄小红,马严,丛群. 基于深度学习的录音文本分类方法[J]. 浙江大学学报(工学版), 2020, 54(7): 1264-1271.
[13] 叶刚,李毅波,马逐曦,成杰. 基于ViBe的端到端铝带表面缺陷检测识别方法[J]. 浙江大学学报(工学版), 2020, 54(10): 1906-1914.
[14] 陈思,蔡晓东,侯珍珍,李波. 基于非均匀邻居节点采样的聚合式图嵌入方法[J]. 浙江大学学报(工学版), 2019, 53(11): 2163-2167.
[15] 郭宝震, 左万利, 王英. 采用词向量注意力机制的双路卷积神经网络句子分类模型[J]. 浙江大学学报(工学版), 2018, 52(9): 1729-1737.