Please wait a minute...
浙江大学学报(工学版)
浙江大学学报(工学版)  2019, Vol. 53 Issue (5): 940-949    DOI: 10.3785/j.issn.1008-973X.2019.05.015
交通工程     
基于残差单发多框检测器模型的交通标志检测与识别
张淑芳(),朱彤
天津大学 电气自动化与信息工程学院，天津 300072
Traffic sign detection and recognition based on residual single shot multibox detector model
Shu-fang ZHANG(),Tong ZHU
School of Electronical and Information Engineering, Tianjin University, Tianjin 300072, China
 全文: PDF(1907 KB)   HTML
摘要：

针对现有目标检测方法仅适用于大尺寸、少量特定种类交通标志的检测，且对复杂交通场景图像检测效果不佳的问题，以抗退化性能较强的ResNet101为基础网络，增加若干卷积层构建残差单发多框检测器（SSD）模型，对高分辨率的交通图像进行多尺度分块检测。为了加快检测速度，采取由粗到精的策略，省略对纯背景图像块的预测. 利用中等尺度图像块的初检结果缩小目标范围；对目标范围内的其他图像块进行检测；将所有图像块结果映射回原图像，并结合非极大值抑制实现精准识别。实验结果表明，该模型在公开的交通标志数据集Tsinghua-Tencent 100K上取得了94%的总体准确率和95%的总体召回率，对多分辨率图像中不同大小和形态的交通标志都具有良好的检测能力，鲁棒性较强。
关键词： 交通标志;  残差单发多框检测器(SSD)模型;  多尺度分块;  检测;  由粗到精    
Abstract:

The existing target detection methods were only suitable for large size and few specific types of traffic signs, and showed poor performance on complex traffic scene images. The ResNet101 with strong anti-degradation performance was used as basic network, and then a residual single shot multibox detector (SSD) model added with a number of convolution layers was proposed, in order to conduct multi-scale block detection on high resolution traffic images. A strategy Coarse-to-Fine was adopted to omit the prediction of pure background image blocks, in order to speed up. The target range was narrowed by the initial detection results of the medium scale image block. The other blocks within the target range were detected. All the block results were mapped back to the original image and non-maximum suppression was used to realize accurate recognition. Experiment results showed that the proposed method achieved 94% overall accuracy and 95% overall recall on the public traffic sign dataset Tsinghua-Tencent 100K. The detection ability on traffic sign with different sizes and shapes in multi-resolution images was strong and the proposed model was robust.
Key words: traffic sign    residual single shot multibox detector (SSD) model    multi-scale block    detection    Coarse-to-Fine
收稿日期: 2018-04-11 出版日期: 2019-05-17
CLC:  TP 391  
作者简介: 张淑芳(1979—)，女，副教授，从事图像视频质量评价、图像识别等研究. orcid.org/0000-0002-9888-2587. E-mail： shufangzhang@tju.edu.cn
服务　　
把本文推荐给朋友
加入引用管理器
E-mail Alert
作者相关文章　　
张淑芳
朱彤

引用本文:

张淑芳,朱彤. 基于残差单发多框检测器模型的交通标志检测与识别[J]. 浙江大学学报(工学版), 2019, 53(5): 940-949.

Shu-fang ZHANG,Tong ZHU. Traffic sign detection and recognition based on residual single shot multibox detector model. Journal of ZheJiang University (Engineering Science), 2019, 53(5): 940-949.

链接本文:

http://www.zjujournals.com/eng/CN/10.3785/j.issn.1008-973X.2019.05.015    或    http://www.zjujournals.com/eng/CN/Y2019/V53/I5/940

图 1  残差单发多框检测器模型图
图 2  多尺度分块示意图
图 3  由粗到精的检测流程图
图 4  45类交通标志图像以及相应的类别名称
图 5  所提方法与其他方法对不同尺寸标志的检测性能比较
方法 at rt
文献[17]方法 0.88 0.91
文献[18]方法 0.90 0.93
本研究方法 0.94 0.95
表 1  本研究方法和其他方法总体精确度和召回率对比
网络
类型 		图像块类型 at rt t/s
基础SSD网络 尺度1 0.77 0.81 59.99
尺度2 0.82 0.84 25.18
尺度3
（中等尺度） 		0.84 0.84 13.75
尺度4 0.84 0.86 8.40
尺度5 0.78 0.78 3.37
全部多尺度分块 0.88 0.89 111.61
由粗到精+
多尺度分块 		0.88 0.88 27.74
残差SSD网络 尺度1 0.79 0.83 45.75
尺度2 0.84 0.86 13.80
尺度3
（中等尺度） 		0.87 0.87 7.50
尺度4 0.86 0.87 4.64
尺度5 0.79 0.80 1.90
全部多尺度分块 0.94 0.95 73.60
由粗到精+
多尺度分块		 0.94 0.95 13.60
表 2  不同方法横向性能和耗时对比
图 6  无遮挡光线良好环境下不同方法的检测结果对比
图 7  有遮挡光线较差的环境下不同方法的检测结果对比图
1 RUTA A, LI Y M, LIU X H. Detection, tracking and recognition of traffic signs from video input [C]// Proceedings of the 11th International IEEE Conference on Intelligent Transportation Systems. Beijing: IEEE, 2008: 55–60.
2 ABUKHAIT J, ABDEL-QADER I, OH J S, et al Road sign detection and shape recognition invariant to sign defects[J]. Social Science Electronic Publishing, 2012, 61 (1): 1- 6
3 刘芳, 邹琪 基于视觉注意机制的交通标志检测[J]. 计算机工程, 2013, 39 (2): 192- 196
LIU Fang, ZOU Qi Traffic sign detection based on visual attention mechanism[J]. Computer Engineering, 2013, 39 (2): 192- 196
4 HUANG Z Y, YU Y L, GU J, et al An efficient method for traffic sign recognition based on extreme learning machine[J]. IEEE Transactions on Cybernetics, 2017, 47 (4): 920- 933
5 KRIZHEVSKY A, SUTSKEVER I, HINTON G E. ImageNet classification with deep convolutional neural networks [C]// Advances in Neural Information Processing Systems 25. Nevada: NIPS, 2012: 1097–1105.
6 GIRSHICK R, DONAHUE J, DARRELL T, et al. Rich feature hierarchies for accurate object detection and semantic segmentation [C]// Proceedings of the 2014 IEEE Conference on Computer Vision and Pattern Recognition. Ohio: IEEE, 2014: 580–587.
7 GIRSHICK R. Fast R-CNN [C]// Proceedings of the 2015 IEEE International Conference on Computer Vision. Santiago: IEEE, 2015: 1440–1448.
8 REN S Q, HE K M, GIRSHICK R, et al. Faster R-CNN: towards real-time object detection with region proposal networks [C]// Advances in Neural Information Processing Systems 28. Montreal: NIPS, 2015: 91–99.
9 DAI J F, LI Y, HE K M, et al. R-FCN: object detection via region-based fully convolutional networks [C]// Advances in Neural Information Processing Systems 29. Barcelona: NIPS, 2016: 379–387.
10 REDMON J, DIVVALA S, GIRSHICK R, et al. You only look once: unified, real-time object detection [C]// Proceedings of the 2016 IEEE Conference on Computer Vision And Pattern. Nevada: IEEE, 2016: 779–788.
11 LIU W, ANGUELOV D, ERHAN D, et al. SSD: single shot multibox detector [C]// Proceedings of the 2016 European Conference on Computer Vision. Amsterdam: ECCV, 2016: 21–37.
12 RUSSAKOVSKY O, DENG J, SU H, et al ImageNet large scale visual recognition challenge[J]. International Journal of Computer Vision, 2015, 115 (3): 211- 252
13 EVERINGHAM M, GOOL L, WILLIAMS C K, et al The pascal visual object classes (voc) challenge[J]. International Journal of Computer Vision, 2010, 88 (2): 202- 228
14 STALLKAMP J, SCHLIPSING M, SALMEN J, et al. The German traffic sign detection benchmark[EB/OL]. [2013-11-05]. http://benchmark.ini.rub.de/?section=gtsdb&subsection=news.
15 STALLKAMP J, SCHLIPSING M, SALMEN J, et al. The German traffic sign recognition benchmark[EB/OL]. [2012-03-16]. http://benchmark.ini.rub.de/?section=gtsrb&subsection=news.
16 TIMOFTE R. BelgiumTS dataset [EB/OL]. [2014-02-18]. http://btsd.ethz.ch/shareddata/.
17 ZHU Z, LIANG D, ZHANG S H, et al. Traffic-sign detection and classification in the wild [C]// Proceedings of the 2016 IEEE Conference on Computer Vision And Pattern Recognition. Nevada: IEEE, 2016: 2110–2118.
18 MENG Z B, FAN X C, CHEN X, et al. Detecting small signs from large images [C]// Proceedings of the 2017 IEEE International Conference on Information Reuse and Integration. California: IEEE, 2017: 217–224.
19 HE K M, ZHANG X Y, REN S Q, et al. Deep residual learning for image recognition [C] // Proceedings of the 2016 IEEE Conference on Computer Vision and Pattern Recognition. Nevada: IEEE, 2016: 770–778.
20 SIMON-YAN K, ZISSERMAN A. Very deep convolutional networks for large-scale image recognition [C]// International Conference on Learning Representations. San Diego: ICLR, 2015: 1-14.
21 SHELHAMER E, LONG J, DARRELL T, et al Fully convolutional networks for semantic segmentation[J]. IEEE Transactions on Pattern Analysis and Machine Intelligence, 2017, 39 (4): 640- 651
22 HARIHARAN B, ARBELAEZ P, GIRSHICK R, et al. Hypercolumns for object segmentation and fine-grained localization [C]// Proceedings of the 2015 IEEE Conference on Computer Vision and Pattern Recognition. Massachusetts: IEEE, 2015: 447–456.
23 LIU W, RABINOVICH A, BERG A C. ParseNet: looking wider to see better [C]// Proceedings of the 2015 IEEE Conference on Computer Vision and Pattern Recognition. Massachusetts: IEEE, 2015: 1–11.
24 JIA Y, SHELHAMER E, DONAHUE J, et al. Caffe: convolutional architecture for fast feature embedding [C]// ACM International Conference on Multimedia. Florida: ACM, 2014: 675–678.
[1] 于勇,薛静远,戴晟,鲍强伟,赵罡. 机加零件质量预测与工艺参数优化方法[J]. 浙江大学学报(工学版), 2021, 55(3): 441-447.
[2] 白大鹏,张斌,洪昊岑,李洋,季清华,杨华勇. 生物3D打印装置及打印模型形貌检测[J]. 浙江大学学报(工学版), 2021, 55(2): 289-298.
[3] 王友卫,凤丽洲. 基于合群度-隶属度噪声检测及动态特征选择的改进AdaBoost算法[J]. 浙江大学学报(工学版), 2021, 55(2): 367-376.
[4] 徐利锋,黄海帆,丁维龙,范玉雷. 基于改进DenseNet的水果小目标检测[J]. 浙江大学学报(工学版), 2021, 55(2): 377-385.
[5] 陈杨波,伊国栋,张树有. 基于点云特征对比的曲面翘曲变形检测方法[J]. 浙江大学学报(工学版), 2021, 55(1): 81-88.
[6] 董大钊,徐冠华,高继良,徐月同,傅建中. 基于机器视觉的机器人装配位姿在线校正算法[J]. 浙江大学学报(工学版), 2021, 55(1): 145-152.
[7] 郑浦,白宏阳,李伟,郭宏伟. 复杂背景下的小目标检测算法[J]. 浙江大学学报(工学版), 2020, 54(9): 1777-1784.
[8] 吴宾,黄添添,叶凌云,宋开臣. 惯性元件再平衡回路噪声整形机理研究[J]. 浙江大学学报(工学版), 2020, 54(9): 1819-1826.
[9] 张剑锋,赵朋,周宏伟,傅建中,陈子辰. 注射成形中聚合物熔体黏度的在线测量装置[J]. 浙江大学学报(工学版), 2020, 54(8): 1474-1480.
[10] 沈麒,赵琰,周晓炜,袁晓冉. 结合结构与梯度的图像哈希算法[J]. 浙江大学学报(工学版), 2020, 54(8): 1525-1533.
[11] 武秋韵,丁伟. 基于动态暗网的互联网扫描行为分析[J]. 浙江大学学报(工学版), 2020, 54(8): 1550-1556.
[12] 夏莹杰,欧阳聪宇. 面向高速公路抛洒物检测的动态背景建模方法[J]. 浙江大学学报(工学版), 2020, 54(7): 1249-1255.
[13] 刘葛辉,陈绍宽,金华,刘爽,彭宏勤. 基于延迟时间模型的不完全检修计划优化模型[J]. 浙江大学学报(工学版), 2020, 54(7): 1298-1307.
[14] 张峻宁,苏群星,刘鹏远,王正军,谷宏强. 基于空间约束的自适应单目3D物体检测算法[J]. 浙江大学学报(工学版), 2020, 54(6): 1138-1146.
[15] 汪周飞,袁伟娜. 基于深度学习的多载波系统信道估计与检测[J]. 浙江大学学报(工学版), 2020, 54(4): 732-738.