An improved algorithm MR2-YOLOV5 based on YOLOv5 was proposed aiming at the problem that the traditional target detection algorithm did not consider the diversity of the target shape scale in the actual sorting scene and could not obtain the rotation angle information. Precise rotation angle detection was completed by adding angle prediction branches and introducing angle classification method of ring smooth label (CSL). The target detection layer was added to improve the detection ability of different scales of the model. Transformer attention mechanism was used at the end of the backbone network to give different weights to each channel and strengthen feature extraction. The feature graphs of different levels extracted from the backbone network were input into the BiFPN network structure to conduct multi-scale feature fusion. The experimental results showed that the mean average precision (mAP) of MR2-YOLOV5 on the self-made data set was 90.56%, which was 5.36% higher than that of YOLOv5s with only angle prediction branch. Categories and rotation angles can be recognized for objects such as occlusion, transparent and deformation. The detection time of single frame is 0.02-0.03 s, which meets the performance requirements of target detection algorithm for sorting scenes.
Hong-zhao DONG,Hao-jie FANG,Nan ZHANG. Multi-scale object detection algorithm for recycled objects based on rotating block positioning. Journal of ZheJiang University (Engineering Science), 2022, 56(1): 16-25.
Fig.5Rotation frame annotation schematic diagram (dot is starting point of annotation)
Fig.6Data set label statistics and distribution
Fig.7Input image after Mosaic data enhancement method
模型
mAP/%
r = 0
r = 2
r = 4
r = 6
r = 8
YOLOv5s
68.79
82.47
84.20
81.20
79.94
Tab.1Comparison of detection performance under different window radii
多尺度融合网络
mAP/%
M/106
FPN
81.05
6.52
PANet
84.20
7.55
BiFPN
85.36
7.83
Tab.2Performance comparison of multi-scale feature fusion network
角度预测分支（CSL）
检测层
Transformer
融合网络
M/106
FLOPs/109
AP
mAP/%
Cs
Gb
Pb
?
3
?
PANet
7.28
17.1
91.7
90.1
85.4
89.05
?
4
?
PANet
8.17
28.2
94.5
92.4
87.6
91.34
?
4
√
PANet
8.17
28.0
95.4
94.1
90.2
93.23
?
4
√
BiFPN
9.25
29.6
95.2
94.7
93.4
94.46
√
3
?
PANet
7.55
18.0
90.5
82.5
79.1
85.20
√
4
?
PANet
8.73
33.4
92.1
85.3
82.2
86.53
√
4
√
PANet
9.91
32.6
93.8
87.5
83.9
88.44
√
4
√
BiFPN
10.99
34.2
96.5
91.2
84.2
90.56
Tab.3Ablation experiment based on YOLOv5s model
Fig.8Training curves of each model（without angle prediction branch）
Fig.9Training curves of each model（including angle prediction branch）
Fig.10Angle loss curves of each model validation set
Fig.11Detection effects of different scales
Fig.12Detection effect under occlusion of multiple objects
Fig.13Detection effect under high target transparency
Fig.14Detection effect in case of deformation of target
[1]
康庄, 杨杰, 郭濠奇 基于机器视觉的垃圾自动分类系统设计[J]. 浙江大学学报:工学版, 2020, 54 (7): 1272- 1280 KANG Zhuang, YANG Jie, GUO Hao-qi Automatic garbage classification system based on machine vision[J]. Journal of Zhejiang University: Engineering Science, 2020, 54 (7): 1272- 1280
[2]
谢先武, 熊禾根, 陶永, 等 一种面向机器人分拣的杂乱工件视觉检测识别方法[J]. 高技术通讯, 2018, 28 (4): 344- 353 XIE Xian-wu, XIONG He-gen, TAO Yong, et al A method for visual detection and recognition of clutter workpieces for robot sorting[J]. Chinese High Technology Letters, 2018, 28 (4): 344- 353
doi: 10.3772/j.issn.1002-0470.2018.04.008
[3]
YANG Z, LI D Wasnet: a neural network-based garbage collection management system[J]. IEEE Access, 2020, 8: 103984- 103993
doi: 10.1109/ACCESS.2020.2999678
[4]
陈智超, 焦海宁, 杨杰, 等 基于改进 MobileNet v2 的垃圾图像分类算法[J]. 浙江大学学报:工学版, 2021, 55 (8): 1490- 1499 CHEN Zhi-chao, JIAO Hai-ning, YANG Jie, et al Garbage image classification algorithm based on improved MobileNet v2[J]. Journal of Zhejiang University: Engineering Science, 2021, 55 (8): 1490- 1499
[5]
袁建野, 南新元, 蔡鑫, 等 基于轻量级残差网路的垃圾图片分类方法[J]. 环境工程, 2021, 39 (2): 6 YUAN Jian-ye, NAN Xin-yuan, CAI Xin, et al Garbage image classification by lightweight residual network[J]. Environmental Engineering, 2021, 39 (2): 6
[6]
NIE Z, DUAN W, LI X Domestic garbage recognition and detection based on Faster R-CNN[J]. Journal of Physics: Conference Series, 2021, 1738 (1): 012089
doi: 10.1088/1742-6596/1738/1/012089
[7]
LIANG B, WANG Y, WANG Y, et al Garbage sorting system based on composite layer cnn and multi-robots[J]. Journal of Physics: Conference Series, 2020, 1634 (1): 012083
doi: 10.1088/1742-6596/1634/1/012083
[8]
周滢慜. 基于机器视觉的生活垃圾智能分拣系统的设计与实现 [D]. 哈尔滨: 哈尔滨工业大学, 2018. ZHOU Ying-min. Design and implementation of visionbased Sorting system for solid waste [D]. Harbin: Harbin Institute of Technology, 2018.
[9]
REDMON J, DIVVALA S, GIRSHICK R, et al. You only look once: unified, real-time object detection [C]// 2016 IEEE Conference on Computer Vision and Pattern Recognition. Las Vegas: IEEE, 2016: 779-788.
[10]
LIU K, TANG H, HE S, et al. Performance validation of Yolo variants for object detection [C]// Proceedings of the 2021 International Conference on Bioinformatics and Intelligent Computing. Vancouver: [s. n. ], 2021: 239-243.
[11]
朱煜, 方观寿, 郑兵兵, 等. 基于旋转框精细定位的遥感目标检测方法研究 [EB/OL]. [2021-10-01]. http://www.aas.net.cn/cn/article/doi/10.16383/j.aas.C200261. ZHU Yu, FANG Guan-shou, ZHENG Bing-bing, et al. Research on detection method of refined rotated boxes in remote sensing [EB/OL]. [2021-10-01]. http://www.aas.net.cn/cn/article/doi/10.16383/j.aas.C200261.
[12]
DING J, XUE N, LONG Y, et al. Learning roi transformer for oriented object detection in aerial images [C]// 2019 IEEE Conference on Computer Vision and Pattern Recognition. Long Bench: IEEE, 2019: 2849-2858.
[13]
XU Y, FU M, WANG Q, et al Gliding vertex on the horizontal bounding box for multi-oriented object detection[J]. IEEE Transactions on Pattern Analysis and Machine Intelligence, 2020, 43 (4): 1452- 1459
[14]
CHEN Y , DING W , LI H , et al. Arbitrary-oriented dense object detection in remote sensing imagery [C]// 2018 IEEE 9th International Conference on Software Engineering and Service Science. Beijing: IEEE, 2019: 436-440.
[15]
YANG X, YANG J, YAN J, et al. Scrdet: towards more robust detection for small, cluttered and rotated objects [C]// 2019 IEEE/CVF International Conference on Computer Vision. South Korea: IEEE, 2019: 8232-8241.
[16]
YANG X, LIU Q, YAN J, et al. R3det: refined single-stage detector with feature refinement for rotating object [EB/OL]. [2021-10-01]. https://arxiv.org/abs/1908.05612.
[17]
YANG X, YAN J, HE T. On the arbitrary-oriented object detection: classification based approaches revisited [EB/OL]. [2021-10-01]. https://arxiv.org/abs/2003.05597v3.
[18]
ZHU X, LIU S, WANG X, et al. TPH-YOLOv5: Improved YOLOv5 based on transformer prediction head for object detection on drone-captured scenarios [C]// Proceedings of the IEEE/CVF International Conference on Computer Vision. Montreal: IEEE, 2021: 2778-2788.
[19]
FU L, GU W, LI W, et al Bidirectional parallel multi-branch convolution feature pyramid network for target detection in aerial images of swarm UAVs[J]. Defence Technology, 2021, 17 (4): 1531- 1541
doi: 10.1016/j.dt.2020.09.018
[20]
VASWANI A, SHAZEER N, PARMAR N, et al. Attention is all you need [C]// Advances in Neural Information Processing Systems. Long Beach: [s. n], 2017: 5998-6008.
[21]
鲁博, 瞿绍军 融合BiFPN和改进Yolov3-tiny网络的航拍图像车辆检测方法[J]. 小型微型计算机系统, 2021, 42 (8): 1694- 1698 LU bo, QU Shao-jun Vehicle detection method in aerial images based on BiFPN and improved Yolov3-tiny Network[J]. Journal of Chinese Computer Systems, 2021, 42 (8): 1694- 1698
doi: 10.3969/j.issn.1000-1220.2021.08.020
[22]
LIN T Y, DOLLAR P, GIRSHICK R, et al. Feature pyramid networks for object detection [C]// 2017 IEEE Conference on Computer Vision and Pattern Recognition. Hawaii: IEEE, 2017: 2117-2125.
[23]
LIU S, QI L, QIN H, et al. Path aggregation network for instance segmentation [C]// 2018 IEEE/CVF Conference on Computer Vision and Pattern Recognition. Salt Lake City: IEEE, 2018: 8759-8768.
[24]
GHIASI G, LIN T Y, LE Q V. Nas-fpn: Learning scalable feature pyramid architecture for object detection [C]// 2019 IEEE/CVF Conference on Computer Vision and Pattern Recognition. Long Beach: IEEE, 2019: 7036-7045.
