Please wait a minute...
浙江大学学报(工学版)
浙江大学学报(工学版)  2024, Vol. 58 Issue (1): 29-39    DOI: 10.3785/j.issn.1008-973X.2024.01.004
计算机技术     
煤矿井下无人驾驶轨道电机车障碍物识别
杨豚1,3(),郭永存1,2,3,*(),王爽1,2,3,马鑫1,3
1. 安徽理工大学 深部煤矿采动响应与灾害防控国家重点实验室，安徽 淮南 232001
2. 安徽理工大学 矿山智能装备与技术安徽省重点实验室，安徽 淮南 232001
3. 安徽理工大学 机械工程学院，安徽 淮南 232001
Obstacle recognition of unmanned rail electric locomotive in underground coal mine
Tun YANG1,3(),Yongcun GUO1,2,3,*(),Shuang WANG1,2,3,Xin MA1,3
1. State Key Laboratory of Mining Response and Disaster Prevention and Control in Deep Coal Mines, Anhui University of Science and Technology, Huainan 232001, China
2. Anhui Key Laboratory of Mine Intelligent Equipment and Technology, Anhui University of Science and Technology, Huainan 232001, China
3. School of Mechanical Engineering, Anhui University of Science and Technology, Huainan 232001, China
 全文: PDF(2463 KB)   HTML
摘要：

针对现有煤矿井下无人驾驶轨道电机车因巷道环境恶劣导致障碍物识别精度低的问题，提出用于无人驾驶电机车障碍物精准实时检测的PDM-YOLO模型. 基于YOLOv5，将模型C3模块中的传统卷积替换为部分卷积，构建C3_P特征提取模块，有效减少模型的浮点运算量（FLOPs）与计算延迟. 采用改进后的解耦头，对传统YOLOv5的预测头进行解耦，提高模型的收敛速度及对障碍物的识别精度. 优化Mosaic数据增强方法，丰富训练图像的特征信息，提高模型的普适性和鲁棒性. 实验结果表明，PDM-YOLO模型在自制数据集上的平均检测精度（mAP）达到96.3%，平均检测速度达到109.2 帧/s，PDM-YOLO模型在PASCAL VOC公共数据集上的检测精度高于现有主流的YOLO系列模型.
关键词： 无人驾驶电机车;  YOLO;  障碍物识别;  部分卷积;  解耦头;  Mosaic数据增强    
Abstract:

The PDM-YOLO model for accurate real-time obstacle detection in unmanned electric locomotives was proposed in order to address the problem of low accuracy of obstacle recognition in existing coal mine underground unmanned electric locomotives due to poor roadway environments. The ordinary convolution in the C3 module of the conventional YOLOv5 model was replaced with partial convolution to construct the C3_P feature extraction module, which effectively reduced the floating-point operations (FLOPs) and computational delay of the model. The improved decoupled head was used to decouple the prediction head of the conventional YOLOv5 model in order to improve the convergence speed of the model and the accuracy of obstacle recognition. The Mosaic data augmentation method was optimized to enrich the feature information of the training images and enhance the generalizability and robustness of the model. The experimental results showed that the mean average precision (mAP) of the PDM-YOLO model reached 96.3% and the average detection speed reached 109.2 frames per second on the self-built dataset. The detection accuracy of the PDM-YOLO model on the PASCAL VOC public dataset is higher than that of the existing mainstream YOLO series models.
Key words: unmanned electric locomotive    YOLO    obstacle recognition    partial convolution    decoupled head    Mosaic data augmentation
收稿日期: 2023-05-30 出版日期: 2023-11-07
CLC:  TP 391  
基金资助: 安徽省高校杰出青年科研资助项目（2022AH020056）；国家自然科学基金资助项目（52274152）；中国科协青年托举人才工程资助项目（YESS20220337）
通讯作者: 郭永存     E-mail: yangtun0324@126.com;guoyc1965@126.com
作者简介: 杨豚（1998—），男，博士生，从事煤矿辅助运输装备视觉检测的研究. orcid.org/0009-0008-4414-9268.E-mail： yangtun0324@126.com
服务　　
把本文推荐给朋友
加入引用管理器
E-mail Alert
作者相关文章　　
杨豚
郭永存
王爽
马鑫

引用本文:

杨豚,郭永存,王爽,马鑫. 煤矿井下无人驾驶轨道电机车障碍物识别[J]. 浙江大学学报(工学版), 2024, 58(1): 29-39.

Tun YANG,Yongcun GUO,Shuang WANG,Xin MA. Obstacle recognition of unmanned rail electric locomotive in underground coal mine. Journal of ZheJiang University (Engineering Science), 2024, 58(1): 29-39.

链接本文:

https://www.zjujournals.com/eng/CN/10.3785/j.issn.1008-973X.2024.01.004    或    https://www.zjujournals.com/eng/CN/Y2024/V58/I1/29

图 1  部分卷积过程
图 2  C3与C3_P的结构
图 3  几种预测头的结构
模型 FLOPs/109 v/（帧·s?1）
YOLOv5-YOLOX解耦头 56.3 89.5
YOLOv5-改进的解耦头 36.5 104.2
表 1  模型添加2种解耦头的浮点运算总量及检测速度的对比
图 4  优化前、后的Mosaic数据增强效果对比
图 5  PDM-YOLO模型的结构
图 6  2种C3_P及2种Bottleneck
图 7  部分障碍物的检测数据集
超参数 参数值
输入图像尺寸(image size) 640×640
学习率(learning rate) 0.01
动量(momentum) 0.937
权重衰减系数(weight decay) 0.0005
色调(hue augmentation) 0.015
饱和度(saturation augmentation) 0.7
曝光度(value augmentation) 0.4
Mosaic 1.0
Mixup 1.0
表 2  模型训练超参数设定
模型 AP0.5/% mAP0.5/% FLOPs/109
Miner Locomotive Mine_car Flat_car Gangue
YOLOv5 93.1 97.8 92.3 85.3 97.2 93.1 15.9
YOLOv5-C3_P 96.9 97.5 94.5 97.6 92.3 95.7 13.0
YOLOv5-Decoupled 96.4 97.8 93.1 99.5 93.8 96.1 36.5
YOLOv5-Mosaic9 96.3 98.5 95.2 78.3 97.6 93.2 15.9
PDM-YOLO 96.3 97.7 94.6 99.1 93.7 96.3 33.4
表 3  消融对比分析
图 8  训练损失的对比
模型 AP0.5/% mAP0.5/% FLOPs /109 v/(帧·s?1) 内存/MB
Miner Locomotive Mine_car Flat_car Gangue
YOLOv3 94.1 96.6 91.2 92.3 91.6 93.2 154.6 27.0 246.4
YOLOv4 94.2 97.5 93.0 90.1 90.8 93.1 141.1 31.8 256.1
YOLOv3-Tiny 92.3 93.8 91.8 80.6 86.4 88.9 12.9 117.8 34.7
YOLOv4-Tiny 91.3 93.4 91.9 80.8 86.1 88.7 16.1 126.5 23.6
YOLOv5 93.1 97.8 92.3 85.3 97.2 93.1 15.9 118.9 14.4
YOLOv6 93.8 96.2 93.2 87.2 97.2 93.5 44.1 109.8 38.0
YOLOv7 95.1 97.1 92.3 90.8 93.1 93.7 26.1 105.2 19.0
YOLOv8 94.2 97.8 91.7 92.9 92.8 93.9 28.4 111.3 22.5
PDM-YOLO 96.3 97.7 94.6 99.1 93.7 96.3 33.4 109.2 26.6
表 4  不同模型的对比分析
图 9  PDM-YOLO与YOLOv5检测结果的对比
场景 图像编号 PDM-YOLO检测结果 YOLOv5检测结果
障碍物部分遮挡 A1，B1 无漏检 漏检矿工
A2，B2 无漏检 漏检矿工
A3，B3 无漏检 漏检矿工
光照不足 A4，B4 无漏检 漏检矿工
A5，B5 无漏检 漏检矿工
A6，B6 无漏检 漏检矿工
小目标障碍物 A7，B7 无漏检 漏检矸石
A8，B8 无漏检 漏检矸石
A9，B9 无漏检 漏检矸石
远距离障碍物 A10，B10 无漏检 漏检矿工
A11，B11 无漏检 漏检矿工
A12，B12 无漏检 漏检矿工及煤矿运输小车
表 5  PDM-YOLO与YOLOv5检测结果的对比分析
模型 Nc Nf Nm Nz
YOLOv3 2 618 48 191 2 857
YOLOv4 2 656 42 159 2 857
YOLOv3-Tiny 2 515 59 283 2 857
YOLOv4-Tiny 2 554 51 252 2 857
YOLOv5 2 675 31 151 2 857
YOLOv6 2 652 47 158 2 857
YOLOv7 2 686 36 135 2 857
YOLOv8 2 692 42 123 2 857
PDM-YOLO 2 786 19 52 2 857
表 6  障碍物检测数量的对比
模型 PASCAL VOC 2007 PASCAL VOC 2012
mAP0.5 /% mAP0.5:0.95 /% mAP0.5 /% mAP0.5:0.95 /%
YOLOv5 68.7 48.8 70.6 48.4
YOLOv6 61.4 43.3 61.9 43.2
YOLOv7 67.6 47.8 69.9 49.8
YOLOv8 66.1 48.9 66.3 49.7
PDM-YOLO 70.9 49.0 71.1 49.6
表 7  不同模型在公共数据集上的检测精度比较
1 王国法 煤矿智能化最新技术进展与问题探讨[J]. 煤炭科学技术, 2022, 50 (1): 1- 27
WANG Guofa New technological progress of coal mine intelligence and its problems[J]. Coal Science and Technology, 2022, 50 (1): 1- 27
doi: 10.3969/j.issn.0253-2336.2022.1.mtkxjs202201001
2 葛世荣, 胡而已, 李允旺 煤矿机器人技术新进展及新方向[J]. 煤炭学报, 2023, 48 (1): 54- 73
GE Shirong, HU Eryi, LI Yunwang New progress and direction of robot technology in coal mine[J]. Journal of China Coal Society, 2023, 48 (1): 54- 73
3 韩江洪, 卫星, 陆阳, 等 煤矿井下机车无人驾驶系统关键技术[J]. 煤炭学报, 2020, 45 (6): 2104- 2115
HAN Jianghong, WEI Xing, LU Yang, et al Driverless technology of underground locomotive in coal mine[J]. Journal of China Coal Society, 2020, 45 (6): 2104- 2115
4 FARSHAD G, ESMAEEL K, MOHAMMAD R Real-time obstacle detection by stereo vision and ultrasonic data fusion[J]. Measurement, 2022, 190: 110718
doi: 10.1016/j.measurement.2022.110718
5 靳舒凯, 魏冠楠, 王春明, 等 煤矿副井矿车装载物智能识别方法[J]. 工矿自动化, 2022, 48 (4): 14- 19
JIN Shukai, WEI Guannan, WANG Chunming, et al Intelligent identification method for mine car load in coal mine auxiliary shaft[J]. Journal of Mine Automation, 2022, 48 (4): 14- 19
6 卢万杰, 付华, 赵洪瑞 基于深度学习算法的矿用巡检机器人设备识别[J]. 工程设计学报, 2019, 26 (5): 527- 533
LU Wanjie, FU Hua, ZHAO Hongrui Equipment recognition of mining patrol robot based on deep learning algorithm[J]. Chinese Journal of Engineering Design, 2019, 26 (5): 527- 533
7 李伟山, 卫晨, 王琳 改进的Faster RCNN煤矿井下行人检测算法[J]. 计算机工程与应用, 2019, 55 (4): 200- 207
LI Weishan, WEI Chen, WANG Lin Improved faster RCNN approach for pedestrian detection in underground coal mine[J]. Computer Engineering and Applications, 2019, 55 (4): 200- 207
8 PAN H, SHI Y, LEI X, et al Fast identification model for coal and gangue based on the improved tiny YOLO v3[J]. Journal of Real-Time Image Processing, 2022, 19 (3): 687- 701
doi: 10.1007/s11554-022-01215-1
9 张庆贺, 陈晨, 袁亮, 等 基于DIC和YOLO算法的复杂裂隙岩石破坏过程动态裂隙早期智能识别[J]. 煤炭学报, 2022, 47 (3): 1208- 1219
ZHANG Qinghe, CHEN Chen, YUAN Liang, et al Early and intelligent recognition of dynamic cracks during damage of complex fractured rock masses based on DIC and YOLO algorithms[J]. Journal of China Coal Society, 2022, 47 (3): 1208- 1219
10 李飞, 胡坤, 张勇, 等 基于混合域注意力 YOLOv4 的输送带纵向撕裂多维度检测[J]. 浙江大学学报: 工学版, 2022, 56 (11): 2156- 2167
LI Fei, HU Kun, ZHANG Yong, et al Multi-dimensional detection of longitudinal tearing of conveyor belt based on YOLOv4 of hybrid domain attention[J]. Journal of Zhejiang University: Engineering Science, 2022, 56 (11): 2156- 2167
11 HOWARD A, ZHU M, CHEN B, et al. MobileNets: efficient convolutional neural networks for mobile vision applications. [EB/OL]. [2023-05-15]. https://doi.org/10.48550/arXiv.1704.04861.
12 ZHANG X, ZHOU X, LIN M, et al. ShuffleNet: an extremely efficient convolutional neural network for mobile devices [C]// Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition. Salt Lake City: IEEE, 2018: 6848–6856.
13 HAN K, WANG Y, TIAN Q, et al. GhostNet: more features from cheap operations [C]// Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition. Seattle: IEEE, 2020: 1577–1586.
14 CHEN J, KAO S, HE H, et al. Run, don’t walk: chasing higher FLOPS for faster neural networks [EB/OL]. [2023-05-15]. https://doi.org/10.48550/arXiv.2303.03667.
15 GE Z, LIU S, WANG F, et al. YOLOX: exceeding YOLO series in 2021 [EB/OL]. [2023-05-15]. https://doi.org/10.48550/arXiv.2107.08430.
16 BOCHKOVSKIY A, WANG C Y, LIAO H Y M. YOLOv4: optimal speed and accuracy of object detection [EB/OL]. [2023-05-15]. https://doi.org/10.48550/arXiv.2004.10934.
17 LIN T Y, DOLLÁR P, GIRSHICK R, et al. Feature pyramid networks for object detection [C]// Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition. Honolulu: IEEE, 2017: 936-944.
18 LIU S, QI L, QIN H, et al. Path aggregation network for instance segmentation [C]// Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition. Salt Lake City: IEEE, 2018: 8759-8768.
19 曾耀, 高法钦 基于改进YOLOv5的电子元件表面缺陷检测算法[J]. 浙江大学学报: 工学版, 2023, 57 (3): 455- 465
ZENG Yao, GAO Faqin Surface defect detection algorithm of electronic components based on improved YOLOv5[J]. Journal of Zhejiang University: Engineering Science, 2023, 57 (3): 455- 465
20 HE Y, LIU Z A feature fusion method to improve the driving obstacle detection under foggy weather[J]. IEEE Transactions on Transportation Electrification, 2021, 7 (4): 2505- 2515
doi: 10.1109/TTE.2021.3080690
21 GIRSHICK R, DONAHUE J, DARRELL T, et al. Rich feature hierarchies for accurate object detection and semantic segmentation [C]// Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition. Columbus: IEEE, 2014: 580-587.
22 REN S, HE K, GIRSHICK R, et al Faster R-CNN: towards real-time object detection with region proposal networks[J]. IEEE Transactions on Pattern Analysis and Machine Intelligence, 2017, 39 (6): 1137- 1149
doi: 10.1109/TPAMI.2016.2577031
23 HE K, GKIOXARI G, DOLLÁR P, et al. Mask R-CNN [C]// IEEE International Conference on Computer Vision. Venice: IEEE, 2017: 2980-2988.
24 REDMON J, DIVVALA S, GIRSHICK R, et al. You only look once: unified, real-time object detection [C]// Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition. Seattle: IEEE, 2016: 779-788.
25 REDMON J, FARHADI A. YOLO9000: better, faster, stronger [C]// Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition. Honolulu: IEEE, 2017: 6517-6525.
26 REDMON J, FARHADI A. YOLOv3: an incremental improvement [EB/OL]. [2023-05-15]. https://doi.org/10.48550/arXiv.1804.02767
27 徐印赟, 江明, 李云飞, 等 基于改进YOLO及NMS的水果目标检测[J]. 电子测量与仪器学报, 2022, 36 (4): 114- 123
XU Yinyun, JIANG Ming, LI Yunfei, et al Fruit target detection based on improved YOLO and NMS[J]. Journal of Electronic Measurement and Instrumentation, 2022, 36 (4): 114- 123
[1] 秦思怡,盖绍彦,达飞鹏. 混合采样下多级特征聚合的视频目标检测算法[J]. 浙江大学学报(工学版), 2024, 58(1): 10-19.
[2] 金鑫,庄建军,徐子恒. 轻量化YOLOv5s网络车底危险物识别算法[J]. 浙江大学学报(工学版), 2023, 57(8): 1516-1526.
[3] 方浩杰,董红召,林少轩,罗建宇,方勇. 多特征融合的驾驶员疲劳状态检测方法[J]. 浙江大学学报(工学版), 2023, 57(7): 1287-1296.
[4] 张艳,孙晶雪,孙叶美,刘树东,王传启. 基于分割注意力与线性变换的轻量化目标检测[J]. 浙江大学学报(工学版), 2023, 57(6): 1195-1204.
[5] 卞佰成,陈田,吴入军,刘军. 基于改进YOLOv3的印刷电路板缺陷检测算法[J]. 浙江大学学报(工学版), 2023, 57(4): 735-743.
[6] 马庆禄,鲁佳萍,唐小垚,段学锋. 改进YOLOv5s的公路隧道烟火检测方法[J]. 浙江大学学报(工学版), 2023, 57(4): 784-794.
[7] 曾耀,高法钦. 基于改进YOLOv5的电子元件表面缺陷检测算法[J]. 浙江大学学报(工学版), 2023, 57(3): 455-465.
[8] 袁天乐,袁巨龙,朱勇建,郑翰辰. 基于改进YOLOv5的推力球轴承表面缺陷检测算法[J]. 浙江大学学报(工学版), 2022, 56(12): 2349-2357.
[9] 李飞,胡坤,张勇,王文善,蒋浩. 基于混合域注意力YOLOv4的输送带纵向撕裂多维度检测[J]. 浙江大学学报(工学版), 2022, 56(11): 2156-2167.
[10] 张云佐,郭威,蔡昭权,李文博. 联合多尺度与注意力机制的遥感图像目标检测[J]. 浙江大学学报(工学版), 2022, 56(11): 2215-2223.
[11] 董红召,方浩杰,张楠. 旋转框定位的多尺度再生物品目标检测算法[J]. 浙江大学学报(工学版), 2022, 56(1): 16-25.
[12] 辛文斌,郝惠敏,卜明龙,兰媛,黄家海,熊晓燕. 基于ShuffleNetv2-YOLOv3模型的静态手势实时识别方法[J]. 浙江大学学报(工学版), 2021, 55(10): 1815-1824.