Please wait a minute...
Journal of ZheJiang University (Engineering Science)  2022, Vol. 56 Issue (1): 16-25    DOI: 10.3785/j.issn.1008-973X.2022.01.002
    
Multi-scale object detection algorithm for recycled objects based on rotating block positioning
Hong-zhao DONG(),Hao-jie FANG,Nan ZHANG
ITS Joint Research Institute, Zhejiang University of Technology, Hangzhou 310014, China
Download: HTML     PDF(1360KB) HTML
Export: BibTeX | EndNote (RIS)      

Abstract  

An improved algorithm MR2-YOLOV5 based on YOLOv5 was proposed aiming at the problem that the traditional target detection algorithm did not consider the diversity of the target shape scale in the actual sorting scene and could not obtain the rotation angle information. Precise rotation angle detection was completed by adding angle prediction branches and introducing angle classification method of ring smooth label (CSL). The target detection layer was added to improve the detection ability of different scales of the model. Transformer attention mechanism was used at the end of the backbone network to give different weights to each channel and strengthen feature extraction. The feature graphs of different levels extracted from the backbone network were input into the BiFPN network structure to conduct multi-scale feature fusion. The experimental results showed that the mean average precision (mAP) of MR2-YOLOV5 on the self-made data set was 90.56%, which was 5.36% higher than that of YOLOv5s with only angle prediction branch. Categories and rotation angles can be recognized for objects such as occlusion, transparent and deformation. The detection time of single frame is 0.02-0.03 s, which meets the performance requirements of target detection algorithm for sorting scenes.



Key wordsdetection of recycled goods      YOLOv5      rotating frame detection      circular smooth label      feature pyramid      attentional mechanism     
Received: 25 November 2021      Published: 05 January 2022
CLC:  TP 391  
  X 705  
Fund:  国家自然科学基金资助项目(61773347);浙江公益技术研究项目(LGF19F030001)
Cite this article:

Hong-zhao DONG,Hao-jie FANG,Nan ZHANG. Multi-scale object detection algorithm for recycled objects based on rotating block positioning. Journal of ZheJiang University (Engineering Science), 2022, 56(1): 16-25.

URL:

https://www.zjujournals.com/eng/10.3785/j.issn.1008-973X.2022.01.002     OR     https://www.zjujournals.com/eng/Y2022/V56/I1/16


旋转框定位的多尺度再生物品目标检测算法

针对传统目标检测算法未考虑实际分拣场景目标物形态尺度的多样性,无法获取旋转角度信息的问题,提出基于YOLOv5的改进算法MR2-YOLOv5. 通过添加角度预测分支,引入环形平滑标签(CSL)角度分类方法,完成旋转角度精准检测. 增加目标检测层用于提升模型不同尺度检测能力,在主干网络末端利用Transformer注意力机制对各通道赋予不同的权重,强化特征提取. 利用主干网络提取到的不同层次特征图输入BiFPN网络结构中,开展多尺度特征融合. 实验结果表明,MR2-YOLOv5在自制数据集上的均值平均精度(mAP)为90.56%,较仅添加角度预测分支的YOLOv5s基础网络提升5.36%;对于遮挡、透明、变形等目标物,均可以识别类别和旋转角度,图像单帧检测时间为0.02~0.03 s,满足分拣场景对目标检测算法的性能需求.


关键词: 再生物品检测,  YOLOv5,  旋转框检测,  环形平滑标签,  特征金字塔,  注意力机制 
Fig.1 MR2-YOLOv5 network structure
Fig.2 MR2-YOLOv5 structure of each module
Fig.3 CSL schematic diagram
Fig.4 Angle definition diagram
Fig.5 Rotation frame annotation schematic diagram (dot is starting point of annotation)
Fig.6 Data set label statistics and distribution
Fig.7 Input image after Mosaic data enhancement method
模型 mAP/%
r = 0 r = 2 r = 4 r = 6 r = 8
YOLOv5s 68.79 82.47 84.20 81.20 79.94
Tab.1 Comparison of detection performance under different window radii
多尺度融合网络 mAP/% M/106
FPN 81.05 6.52
PANet 84.20 7.55
BiFPN 85.36 7.83
Tab.2 Performance comparison of multi-scale feature fusion network
角度预测分支(CSL) 检测层 Transformer 融合网络 M/106 FLOPs/109 AP mAP/%
Cs Gb Pb
? 3 ? PANet 7.28 17.1 91.7 90.1 85.4 89.05
? 4 ? PANet 8.17 28.2 94.5 92.4 87.6 91.34
? 4 PANet 8.17 28.0 95.4 94.1 90.2 93.23
? 4 BiFPN 9.25 29.6 95.2 94.7 93.4 94.46
3 ? PANet 7.55 18.0 90.5 82.5 79.1 85.20
4 ? PANet 8.73 33.4 92.1 85.3 82.2 86.53
4 PANet 9.91 32.6 93.8 87.5 83.9 88.44
4 BiFPN 10.99 34.2 96.5 91.2 84.2 90.56
Tab.3 Ablation experiment based on YOLOv5s model
Fig.8 Training curves of each model(without angle prediction branch)
Fig.9 Training curves of each model(including angle prediction branch)
Fig.10 Angle loss curves of each model validation set
Fig.11 Detection effects of different scales
Fig.12 Detection effect under occlusion of multiple objects
Fig.13 Detection effect under high target transparency
Fig.14 Detection effect in case of deformation of target
[1]   康庄, 杨杰, 郭濠奇 基于机器视觉的垃圾自动分类系统设计[J]. 浙江大学学报:工学版, 2020, 54 (7): 1272- 1280
KANG Zhuang, YANG Jie, GUO Hao-qi Automatic garbage classification system based on machine vision[J]. Journal of Zhejiang University: Engineering Science, 2020, 54 (7): 1272- 1280
[2]   谢先武, 熊禾根, 陶永, 等 一种面向机器人分拣的杂乱工件视觉检测识别方法[J]. 高技术通讯, 2018, 28 (4): 344- 353
XIE Xian-wu, XIONG He-gen, TAO Yong, et al A method for visual detection and recognition of clutter workpieces for robot sorting[J]. Chinese High Technology Letters, 2018, 28 (4): 344- 353
doi: 10.3772/j.issn.1002-0470.2018.04.008
[3]   YANG Z, LI D Wasnet: a neural network-based garbage collection management system[J]. IEEE Access, 2020, 8: 103984- 103993
doi: 10.1109/ACCESS.2020.2999678
[4]   陈智超, 焦海宁, 杨杰, 等 基于改进 MobileNet v2 的垃圾图像分类算法[J]. 浙江大学学报:工学版, 2021, 55 (8): 1490- 1499
CHEN Zhi-chao, JIAO Hai-ning, YANG Jie, et al Garbage image classification algorithm based on improved MobileNet v2[J]. Journal of Zhejiang University: Engineering Science, 2021, 55 (8): 1490- 1499
[5]   袁建野, 南新元, 蔡鑫, 等 基于轻量级残差网路的垃圾图片分类方法[J]. 环境工程, 2021, 39 (2): 6
YUAN Jian-ye, NAN Xin-yuan, CAI Xin, et al Garbage image classification by lightweight residual network[J]. Environmental Engineering, 2021, 39 (2): 6
[6]   NIE Z, DUAN W, LI X Domestic garbage recognition and detection based on Faster R-CNN[J]. Journal of Physics: Conference Series, 2021, 1738 (1): 012089
doi: 10.1088/1742-6596/1738/1/012089
[7]   LIANG B, WANG Y, WANG Y, et al Garbage sorting system based on composite layer cnn and multi-robots[J]. Journal of Physics: Conference Series, 2020, 1634 (1): 012083
doi: 10.1088/1742-6596/1634/1/012083
[8]   周滢慜. 基于机器视觉的生活垃圾智能分拣系统的设计与实现 [D]. 哈尔滨: 哈尔滨工业大学, 2018.
ZHOU Ying-min. Design and implementation of visionbased Sorting system for solid waste [D]. Harbin: Harbin Institute of Technology, 2018.
[9]   REDMON J, DIVVALA S, GIRSHICK R, et al. You only look once: unified, real-time object detection [C]// 2016 IEEE Conference on Computer Vision and Pattern Recognition. Las Vegas: IEEE, 2016: 779-788.
[10]   LIU K, TANG H, HE S, et al. Performance validation of Yolo variants for object detection [C]// Proceedings of the 2021 International Conference on Bioinformatics and Intelligent Computing. Vancouver: [s. n. ], 2021: 239-243.
[11]   朱煜, 方观寿, 郑兵兵, 等. 基于旋转框精细定位的遥感目标检测方法研究 [EB/OL]. [2021-10-01]. http://www.aas.net.cn/cn/article/doi/10.16383/j.aas.C200261.
ZHU Yu, FANG Guan-shou, ZHENG Bing-bing, et al. Research on detection method of refined rotated boxes in remote sensing [EB/OL]. [2021-10-01]. http://www.aas.net.cn/cn/article/doi/10.16383/j.aas.C200261.
[12]   DING J, XUE N, LONG Y, et al. Learning roi transformer for oriented object detection in aerial images [C]// 2019 IEEE Conference on Computer Vision and Pattern Recognition. Long Bench: IEEE, 2019: 2849-2858.
[13]   XU Y, FU M, WANG Q, et al Gliding vertex on the horizontal bounding box for multi-oriented object detection[J]. IEEE Transactions on Pattern Analysis and Machine Intelligence, 2020, 43 (4): 1452- 1459
[14]   CHEN Y , DING W , LI H , et al. Arbitrary-oriented dense object detection in remote sensing imagery [C]// 2018 IEEE 9th International Conference on Software Engineering and Service Science. Beijing: IEEE, 2019: 436-440.
[15]   YANG X, YANG J, YAN J, et al. Scrdet: towards more robust detection for small, cluttered and rotated objects [C]// 2019 IEEE/CVF International Conference on Computer Vision. South Korea: IEEE, 2019: 8232-8241.
[16]   YANG X, LIU Q, YAN J, et al. R3det: refined single-stage detector with feature refinement for rotating object [EB/OL]. [2021-10-01]. https://arxiv.org/abs/1908.05612.
[17]   YANG X, YAN J, HE T. On the arbitrary-oriented object detection: classification based approaches revisited [EB/OL]. [2021-10-01]. https://arxiv.org/abs/2003.05597v3.
[18]   ZHU X, LIU S, WANG X, et al. TPH-YOLOv5: Improved YOLOv5 based on transformer prediction head for object detection on drone-captured scenarios [C]// Proceedings of the IEEE/CVF International Conference on Computer Vision. Montreal: IEEE, 2021: 2778-2788.
[19]   FU L, GU W, LI W, et al Bidirectional parallel multi-branch convolution feature pyramid network for target detection in aerial images of swarm UAVs[J]. Defence Technology, 2021, 17 (4): 1531- 1541
doi: 10.1016/j.dt.2020.09.018
[20]   VASWANI A, SHAZEER N, PARMAR N, et al. Attention is all you need [C]// Advances in Neural Information Processing Systems. Long Beach: [s. n], 2017: 5998-6008.
[21]   鲁博, 瞿绍军 融合BiFPN和改进Yolov3-tiny网络的航拍图像车辆检测方法[J]. 小型微型计算机系统, 2021, 42 (8): 1694- 1698
LU bo, QU Shao-jun Vehicle detection method in aerial images based on BiFPN and improved Yolov3-tiny Network[J]. Journal of Chinese Computer Systems, 2021, 42 (8): 1694- 1698
doi: 10.3969/j.issn.1000-1220.2021.08.020
[22]   LIN T Y, DOLLAR P, GIRSHICK R, et al. Feature pyramid networks for object detection [C]// 2017 IEEE Conference on Computer Vision and Pattern Recognition. Hawaii: IEEE, 2017: 2117-2125.
[23]   LIU S, QI L, QIN H, et al. Path aggregation network for instance segmentation [C]// 2018 IEEE/CVF Conference on Computer Vision and Pattern Recognition. Salt Lake City: IEEE, 2018: 8759-8768.
[24]   GHIASI G, LIN T Y, LE Q V. Nas-fpn: Learning scalable feature pyramid architecture for object detection [C]// 2019 IEEE/CVF Conference on Computer Vision and Pattern Recognition. Long Beach: IEEE, 2019: 7036-7045.
[1] Shu-qin YANG,Yu-hao MA,Ming-yu FANG,Wei-xing QIAN,Jie-xuan CAI,Tong LIU. Lane detection method in complex environments based on instance segmentation[J]. Journal of ZheJiang University (Engineering Science), 2022, 56(4): 809-815, 832.
[2] Li-feng XU,Hai-fan HUANG,Wei-long DING,Yu-lei FAN. Detection of small fruit target based on improved DenseNet[J]. Journal of ZheJiang University (Engineering Science), 2021, 55(2): 377-385.
[3] Zhi-jie LIN,Zhuang LUO,Lei ZHAO,Dong-ming LU. Multi-scale convolution target detection algorithm with feature pyramid[J]. Journal of ZheJiang University (Engineering Science), 2019, 53(3): 533-540.