Obstacle recognition of unmanned rail electric locomotive in underground coal mine
Tun YANG1,3(),Yongcun GUO1,2,3,*(),Shuang WANG1,2,3,Xin MA1,3
1. State Key Laboratory of Mining Response and Disaster Prevention and Control in Deep Coal Mines, Anhui University of Science and Technology, Huainan 232001, China 2. Anhui Key Laboratory of Mine Intelligent Equipment and Technology, Anhui University of Science and Technology, Huainan 232001, China 3. School of Mechanical Engineering, Anhui University of Science and Technology, Huainan 232001, China
The PDM-YOLO model for accurate real-time obstacle detection in unmanned electric locomotives was proposed in order to address the problem of low accuracy of obstacle recognition in existing coal mine underground unmanned electric locomotives due to poor roadway environments. The ordinary convolution in the C3 module of the conventional YOLOv5 model was replaced with partial convolution to construct the C3_P feature extraction module, which effectively reduced the floating-point operations (FLOPs) and computational delay of the model. The improved decoupled head was used to decouple the prediction head of the conventional YOLOv5 model in order to improve the convergence speed of the model and the accuracy of obstacle recognition. The Mosaic data augmentation method was optimized to enrich the feature information of the training images and enhance the generalizability and robustness of the model. The experimental results showed that the mean average precision (mAP) of the PDM-YOLO model reached 96.3% and the average detection speed reached 109.2 frames per second on the self-built dataset. The detection accuracy of the PDM-YOLO model on the PASCAL VOC public dataset is higher than that of the existing mainstream YOLO series models.
Tun YANG,Yongcun GUO,Shuang WANG,Xin MA. Obstacle recognition of unmanned rail electric locomotive in underground coal mine. Journal of ZheJiang University (Engineering Science), 2024, 58(1): 29-39.
Tab.1Comparison of total floating-point operations and detection speed for two decoupled heads added to model
Fig.4Comparison of Mosaic data enhancement effects before and after optimization
Fig.5Structure of PDM-YOLO model
Fig.6Two types of C3_ P and Bottleneck
Fig.7Detection dataset of partial obstacle
超参数
参数值
输入图像尺寸(image size)
640×640
学习率(learning rate)
0.01
动量(momentum)
0.937
权重衰减系数(weight decay)
0.0005
色调(hue augmentation)
0.015
饱和度(saturation augmentation)
0.7
曝光度(value augmentation)
0.4
Mosaic
1.0
Mixup
1.0
Tab.2Model training hyper-parameter setting
模型
AP0.5/%
mAP0.5/%
FLOPs/109
Miner
Locomotive
Mine_car
Flat_car
Gangue
YOLOv5
93.1
97.8
92.3
85.3
97.2
93.1
15.9
YOLOv5-C3_P
96.9
97.5
94.5
97.6
92.3
95.7
13.0
YOLOv5-Decoupled
96.4
97.8
93.1
99.5
93.8
96.1
36.5
YOLOv5-Mosaic9
96.3
98.5
95.2
78.3
97.6
93.2
15.9
PDM-YOLO
96.3
97.7
94.6
99.1
93.7
96.3
33.4
Tab.3Comparative analysis of ablation
Fig.8Comparison of training loss
模型
AP0.5/%
mAP0.5/%
FLOPs /109
v/(帧·s?1)
内存/MB
Miner
Locomotive
Mine_car
Flat_car
Gangue
YOLOv3
94.1
96.6
91.2
92.3
91.6
93.2
154.6
27.0
246.4
YOLOv4
94.2
97.5
93.0
90.1
90.8
93.1
141.1
31.8
256.1
YOLOv3-Tiny
92.3
93.8
91.8
80.6
86.4
88.9
12.9
117.8
34.7
YOLOv4-Tiny
91.3
93.4
91.9
80.8
86.1
88.7
16.1
126.5
23.6
YOLOv5
93.1
97.8
92.3
85.3
97.2
93.1
15.9
118.9
14.4
YOLOv6
93.8
96.2
93.2
87.2
97.2
93.5
44.1
109.8
38.0
YOLOv7
95.1
97.1
92.3
90.8
93.1
93.7
26.1
105.2
19.0
YOLOv8
94.2
97.8
91.7
92.9
92.8
93.9
28.4
111.3
22.5
PDM-YOLO
96.3
97.7
94.6
99.1
93.7
96.3
33.4
109.2
26.6
Tab.4Comparative analysis of different models
Fig.9Comparison of PDM-YOLO and YOLOv5 detection results
场景
图像编号
PDM-YOLO检测结果
YOLOv5检测结果
障碍物部分遮挡
A1，B1
无漏检
漏检矿工
A2，B2
无漏检
漏检矿工
A3，B3
无漏检
漏检矿工
光照不足
A4，B4
无漏检
漏检矿工
A5，B5
无漏检
漏检矿工
A6，B6
无漏检
漏检矿工
小目标障碍物
A7，B7
无漏检
漏检矸石
A8，B8
无漏检
漏检矸石
A9，B9
无漏检
漏检矸石
远距离障碍物
A10，B10
无漏检
漏检矿工
A11，B11
无漏检
漏检矿工
A12，B12
无漏检
漏检矿工及煤矿运输小车
Tab.5Comparative analysis of test results between PDM-YOLO and YOLOv5
模型
Nc
Nf
Nm
Nz
YOLOv3
2 618
48
191
2 857
YOLOv4
2 656
42
159
2 857
YOLOv3-Tiny
2 515
59
283
2 857
YOLOv4-Tiny
2 554
51
252
2 857
YOLOv5
2 675
31
151
2 857
YOLOv6
2 652
47
158
2 857
YOLOv7
2 686
36
135
2 857
YOLOv8
2 692
42
123
2 857
PDM-YOLO
2 786
19
52
2 857
Tab.6Comparison of obstacle detection numbers
模型
PASCAL VOC 2007
PASCAL VOC 2012
mAP0.5 /%
mAP0.5:0.95 /%
mAP0.5 /%
mAP0.5:0.95 /%
YOLOv5
68.7
48.8
70.6
48.4
YOLOv6
61.4
43.3
61.9
43.2
YOLOv7
67.6
47.8
69.9
49.8
YOLOv8
66.1
48.9
66.3
49.7
PDM-YOLO
70.9
49.0
71.1
49.6
Tab.7Comparison of detection accuracy of different models on public datasets
[1]
王国法 煤矿智能化最新技术进展与问题探讨[J]. 煤炭科学技术, 2022, 50 (1): 1- 27 WANG Guofa New technological progress of coal mine intelligence and its problems[J]. Coal Science and Technology, 2022, 50 (1): 1- 27
doi: 10.3969/j.issn.0253-2336.2022.1.mtkxjs202201001
[2]
葛世荣, 胡而已, 李允旺 煤矿机器人技术新进展及新方向[J]. 煤炭学报, 2023, 48 (1): 54- 73 GE Shirong, HU Eryi, LI Yunwang New progress and direction of robot technology in coal mine[J]. Journal of China Coal Society, 2023, 48 (1): 54- 73
[3]
韩江洪, 卫星, 陆阳, 等 煤矿井下机车无人驾驶系统关键技术[J]. 煤炭学报, 2020, 45 (6): 2104- 2115 HAN Jianghong, WEI Xing, LU Yang, et al Driverless technology of underground locomotive in coal mine[J]. Journal of China Coal Society, 2020, 45 (6): 2104- 2115
[4]
FARSHAD G, ESMAEEL K, MOHAMMAD R Real-time obstacle detection by stereo vision and ultrasonic data fusion[J]. Measurement, 2022, 190: 110718
doi: 10.1016/j.measurement.2022.110718
[5]
靳舒凯, 魏冠楠, 王春明, 等 煤矿副井矿车装载物智能识别方法[J]. 工矿自动化, 2022, 48 (4): 14- 19 JIN Shukai, WEI Guannan, WANG Chunming, et al Intelligent identification method for mine car load in coal mine auxiliary shaft[J]. Journal of Mine Automation, 2022, 48 (4): 14- 19
[6]
卢万杰, 付华, 赵洪瑞 基于深度学习算法的矿用巡检机器人设备识别[J]. 工程设计学报, 2019, 26 (5): 527- 533 LU Wanjie, FU Hua, ZHAO Hongrui Equipment recognition of mining patrol robot based on deep learning algorithm[J]. Chinese Journal of Engineering Design, 2019, 26 (5): 527- 533
[7]
李伟山, 卫晨, 王琳 改进的Faster RCNN煤矿井下行人检测算法[J]. 计算机工程与应用, 2019, 55 (4): 200- 207 LI Weishan, WEI Chen, WANG Lin Improved faster RCNN approach for pedestrian detection in underground coal mine[J]. Computer Engineering and Applications, 2019, 55 (4): 200- 207
[8]
PAN H, SHI Y, LEI X, et al Fast identification model for coal and gangue based on the improved tiny YOLO v3[J]. Journal of Real-Time Image Processing, 2022, 19 (3): 687- 701
doi: 10.1007/s11554-022-01215-1
[9]
张庆贺, 陈晨, 袁亮, 等 基于DIC和YOLO算法的复杂裂隙岩石破坏过程动态裂隙早期智能识别[J]. 煤炭学报, 2022, 47 (3): 1208- 1219 ZHANG Qinghe, CHEN Chen, YUAN Liang, et al Early and intelligent recognition of dynamic cracks during damage of complex fractured rock masses based on DIC and YOLO algorithms[J]. Journal of China Coal Society, 2022, 47 (3): 1208- 1219
[10]
李飞, 胡坤, 张勇, 等 基于混合域注意力 YOLOv4 的输送带纵向撕裂多维度检测[J]. 浙江大学学报: 工学版, 2022, 56 (11): 2156- 2167 LI Fei, HU Kun, ZHANG Yong, et al Multi-dimensional detection of longitudinal tearing of conveyor belt based on YOLOv4 of hybrid domain attention[J]. Journal of Zhejiang University: Engineering Science, 2022, 56 (11): 2156- 2167
[11]
HOWARD A, ZHU M, CHEN B, et al. MobileNets: efficient convolutional neural networks for mobile vision applications. [EB/OL]. [2023-05-15]. https://doi.org/10.48550/arXiv.1704.04861.
[12]
ZHANG X, ZHOU X, LIN M, et al. ShuffleNet: an extremely efficient convolutional neural network for mobile devices [C]// Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition. Salt Lake City: IEEE, 2018: 6848–6856.
[13]
HAN K, WANG Y, TIAN Q, et al. GhostNet: more features from cheap operations [C]// Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition. Seattle: IEEE, 2020: 1577–1586.
[14]
CHEN J, KAO S, HE H, et al. Run, don’t walk: chasing higher FLOPS for faster neural networks [EB/OL]. [2023-05-15]. https://doi.org/10.48550/arXiv.2303.03667.
[15]
GE Z, LIU S, WANG F, et al. YOLOX: exceeding YOLO series in 2021 [EB/OL]. [2023-05-15]. https://doi.org/10.48550/arXiv.2107.08430.
[16]
BOCHKOVSKIY A, WANG C Y, LIAO H Y M. YOLOv4: optimal speed and accuracy of object detection [EB/OL]. [2023-05-15]. https://doi.org/10.48550/arXiv.2004.10934.
[17]
LIN T Y, DOLLÁR P, GIRSHICK R, et al. Feature pyramid networks for object detection [C]// Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition. Honolulu: IEEE, 2017: 936-944.
[18]
LIU S, QI L, QIN H, et al. Path aggregation network for instance segmentation [C]// Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition. Salt Lake City: IEEE, 2018: 8759-8768.
[19]
曾耀, 高法钦 基于改进YOLOv5的电子元件表面缺陷检测算法[J]. 浙江大学学报: 工学版, 2023, 57 (3): 455- 465 ZENG Yao, GAO Faqin Surface defect detection algorithm of electronic components based on improved YOLOv5[J]. Journal of Zhejiang University: Engineering Science, 2023, 57 (3): 455- 465
[20]
HE Y, LIU Z A feature fusion method to improve the driving obstacle detection under foggy weather[J]. IEEE Transactions on Transportation Electrification, 2021, 7 (4): 2505- 2515
doi: 10.1109/TTE.2021.3080690
[21]
GIRSHICK R, DONAHUE J, DARRELL T, et al. Rich feature hierarchies for accurate object detection and semantic segmentation [C]// Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition. Columbus: IEEE, 2014: 580-587.
[22]
REN S, HE K, GIRSHICK R, et al Faster R-CNN: towards real-time object detection with region proposal networks[J]. IEEE Transactions on Pattern Analysis and Machine Intelligence, 2017, 39 (6): 1137- 1149
doi: 10.1109/TPAMI.2016.2577031
[23]
HE K, GKIOXARI G, DOLLÁR P, et al. Mask R-CNN [C]// IEEE International Conference on Computer Vision. Venice: IEEE, 2017: 2980-2988.
[24]
REDMON J, DIVVALA S, GIRSHICK R, et al. You only look once: unified, real-time object detection [C]// Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition. Seattle: IEEE, 2016: 779-788.
[25]
REDMON J, FARHADI A. YOLO9000: better, faster, stronger [C]// Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition. Honolulu: IEEE, 2017: 6517-6525.
[26]
REDMON J, FARHADI A. YOLOv3: an incremental improvement [EB/OL]. [2023-05-15]. https://doi.org/10.48550/arXiv.1804.02767
[27]
徐印赟, 江明, 李云飞, 等 基于改进YOLO及NMS的水果目标检测[J]. 电子测量与仪器学报, 2022, 36 (4): 114- 123 XU Yinyun, JIANG Ming, LI Yunfei, et al Fruit target detection based on improved YOLO and NMS[J]. Journal of Electronic Measurement and Instrumentation, 2022, 36 (4): 114- 123
