Please wait a minute...
浙江大学学报(工学版)  2022, Vol. 56 Issue (4): 809-815, 832    DOI: 10.3785/j.issn.1008-973X.2022.04.021
计算机技术、信息工程     
基于实例分割的复杂环境车道线检测方法
杨淑琴(),马玉浩,方铭宇,钱伟行*(),蔡洁萱,刘童
南京师范大学 电气与自动化工程学院,江苏 南京 210023
Lane detection method in complex environments based on instance segmentation
Shu-qin YANG(),Yu-hao MA,Ming-yu FANG,Wei-xing QIAN*(),Jie-xuan CAI,Tong LIU
School of Electrical and Automation Engineering, Nanjing Normal University, Nanjing 210023, China
 全文: PDF(1696 KB)   HTML
摘要:

针对基于语义分割的车道线检测方法存在的特征表述模糊、语义信息利用率较低的问题,采用实例分割算法,提出基于改进混合任务级联(HTC)网络的车道线检测方法. 基于HTC网络模型,在主干网络中引入可变形卷积,提升主干网络对复杂环境中车道线特征的提取能力. 改进特征金字塔网络结构,在特征金字塔网络的基础上添加自底向上的低层特征传递路径,引入空洞卷积,在不损失车道线特征信息的情况下增加特征图感受野,利用低层特征中所包含的车道线的精确定位信息,提高车道线的检测精度. 实验结果表明,改进HTC网络模型可以实现车道线特征的鲁棒提取,在复杂道路环境中可以获得较好的检测性能,有效提高了车道线检测精度.

关键词: 无人驾驶技术车道线检测实例分割可变形卷积特征金字塔网络    
Abstract:

An instance segmentation algorithm was adopted and a lane detection method based on improved hybrid task cascade (HTC) network was proposed aiming at the problems of fuzzy feature representation and low semantic information utilization in the lane detection method based on semantic segmentation. Deformable convolution was introduced into the backbone network based on HTC network model in order to improve the ability of the backbone network to extract lane features in complex environments. The structure of feature pyramid network was improved by adding a bottom-up low-level feature transmission path based on feature pyramid network, and dilated convolution was introduced to increase the receptive field of feature map without loss of lane feature information. The accurate location information of lane lines contained in the low-level features was used to improve the accuracy of lane detection. The experimental results show that the improved HTC network model can realize the robust extraction of lane features, obtain better detection performance in complex road environments, and effectively improve the accuracy of lane detection.

Key words: driverless technology    lane detection    instance segmentation    deformable convolution    feature pyramid network
收稿日期: 2021-05-01 出版日期: 2022-04-24
CLC:  TP 311  
基金资助: 国家自然科学基金资助项目(61803208);江苏省自然科学基金资助项目(BK20180726, BK20191371).
通讯作者: 钱伟行     E-mail: 15651785167@163.com;61192@njnu.edu.cn
作者简介: 杨淑琴 (1997—),女,硕士生,从事图像处理算法研究. orcid.org/0000-0002-5762-6026. E-mail: 15651785167@163.com
服务  
把本文推荐给朋友
加入引用管理器
E-mail Alert
作者相关文章  
杨淑琴
马玉浩
方铭宇
钱伟行
蔡洁萱
刘童

引用本文:

杨淑琴,马玉浩,方铭宇,钱伟行,蔡洁萱,刘童. 基于实例分割的复杂环境车道线检测方法[J]. 浙江大学学报(工学版), 2022, 56(4): 809-815, 832.

Shu-qin YANG,Yu-hao MA,Ming-yu FANG,Wei-xing QIAN,Jie-xuan CAI,Tong LIU. Lane detection method in complex environments based on instance segmentation. Journal of ZheJiang University (Engineering Science), 2022, 56(4): 809-815, 832.

链接本文:

https://www.zjujournals.com/eng/CN/10.3785/j.issn.1008-973X.2022.04.021        https://www.zjujournals.com/eng/CN/Y2022/V56/I4/809

图 1  混合任务级联网络结构图
图 2  改进混合任务级联的网络结构图
图 3  主干网络提取的车道线特征图
图 4  车道线数据集的构建流程图
主干网络 ResNet mAP AP0.5 AP0.75
C3 C4 C5
Conv Conv DCN 57.5 94.2 62.2
Conv DCN DCN 58.2 94.3 62.8
DCN DCN DCN 58.8 94.5 64.4
表 1  基于主干网络不同特征提取方式的车道线检测结果
图 5  训练过程的学习率变化曲线
图 6  训练过程的损失值变化曲线
增强路径特征
提取方式
M/106 mAP AP0.5 AP0.75
常规卷积[18] 100.76 58.6 94.3 63.5
空洞卷积 100.76 58.8 94.5 64.4
表 2  基于不同增强路径特征提取方式的车道线检测结果
模型 结构 M/106 mAP AP0.5 AP0.75
Mask R-CNN[20] 两阶段 62.75 54.6 90.8 58.7
Cascade Mask R-CNN[12] 多阶段 95.80 57.7 92.0 62.8
HTC[12] 多阶段 95.93 57.8 92.4 62.8
本文方法 多阶段 100.76 58.8 94.5 64.4
表 3  基于不同深度学习模型的车道线检测结果
图 7  不同环境下的车道线检测结果
图 8  不同场景下的车道线检测结果
1 罗杨. 复杂环境下的车道线检测 [D]. 成都: 电子科技大学, 2020.
LUO Yang. Lane detection under complicated environment [D]. Chengdu: University of Electronic Science and Technology of China, 2020.
2 NGUYEN T N A, PHUNG S L, BOUZERDOUM A Hybrid deep learning-gaussian process network for pedestrian lane detection in unstructured scenes[J]. IEEE Transactions on Neural Networks and Learning Systems, 2020, 31 (12): 5324- 5338
doi: 10.1109/TNNLS.2020.2966246
3 吴一全, 刘莉 基于视觉的车道线检测方法研究进展[J]. 仪器仪表学报, 2019, 40 (12): 92- 109
WU Yi-quan, LIU Li Research and development of the vision-based lane detection methods[J]. Chinese Journal of Scientific Instrument, 2019, 40 (12): 92- 109
4 胡忠闯. 基于深度学习的车道线检测算法研究 [D]. 杭州: 浙江大学, 2018.
HU Zhong-chuang. Deep learning based lane markings detection algorithm [D]. Hangzhou: Zhejiang University, 2018.
5 YOO H, YANG U, SOHN K Gradient-enhancing conversion for illumination-robust lane detection[J]. IEEE Transactions on Intelligent Transportation Systems, 2013, 14 (3): 1083- 1094
doi: 10.1109/TITS.2013.2252427
6 吴骅跃, 赵祥模 基于IPM和边缘图像过滤的多干扰车道线检测[J]. 中国公路学报, 2020, 33 (5): 153- 164
WU Hua-yue, ZHAO Xiang-mo Multi-interference lane recognition based on IPM and edge image filtering[J]. China Journal of Highway and Transport, 2020, 33 (5): 153- 164
doi: 10.3969/j.issn.1001-7372.2020.05.014
7 SHEN Y H, BI Y R, YANG Z, et al Lane line detection and recognition based on dynamic ROI and modified firefly algorithm[J]. International Journal of Intelligent Robotics and Applications, 2021, 5 (2): 143- 155
doi: 10.1007/s41315-021-00175-2
8 庞彦伟, 修宇璇 基于边缘特征融合和跨连接的车道线语义分割神经网络[J]. 天津大学学报: 自然科学与工程技术版, 2019, 52 (8): 779- 787
PANG Yan-wei, XIU Yu-xuan Lane semantic segmentation neural network based on edge feature merging and skip connections[J]. Journal of Tianjin University: Science and Technology, 2019, 52 (8): 779- 787
9 PAN X G, SHI J P, LUO P, et al. Spatial as deep: spatial CNN for traffic scene understanding [C]// 32nd AAAI Conference on Artificial Intelligence. New Orleans: AAAI, 2018: 7276-7283.
10 NEVEN D, BRABANDERE B D, GEORGOULIS S, et al. Towards end-to-end lane detection: an instance segmentation approach [C]// 2018 IEEE Intelligent Vehicles Symposium. Suzhou: IEEE, 2018: 286-291.
11 CAI Z W, VASCONCELOS N. Cascade R-CNN: delving into high quality object detection [C]// IEEE Conference on Computer Vision and Pattern Recognition. Salt Lake City: IEEE, 2018: 6154-6162.
12 CHEN K, PANG J M, WANG J Q, et al. Hybrid task cascade for instance segmentation [C]// IEEE Conference on Computer Vision and Pattern Recognition. Long Beach: IEEE, 2019: 4969-4978.
13 HE K M, ZHANG X Y, REN S Q, et al. Deep residual learning for image recognition [C]// IEEE Conference on Computer Vision and Pattern Recognition. Las Vegas: IEEE, 2016: 770-778.
14 YU F, KOLTUN V. Multi-scale context aggregation by dilated convolutions [C]// International Conference on Learning Representations. San Juan: [s.n.], 2016: 1-13.
15 DAI J F, QI H Z, XIONG Y W, et al. Deformable convolutional networks [C]// IEEE International Conference on Computer Vision. Venice: IEEE, 2017: 764-773.
16 LIU Z W, SHEN C, FAN X, et al Scale-aware limited deformable convolutional neural networks for traffic sign detection and classification[J]. IET Intelligent Transport Systems, 2020, 14 (12): 1712- 1722
doi: 10.1049/iet-its.2020.0217
17 CHEN S Y, ZHANG Z X, ZHONG R F, et al A dense feature pyramid network-based deep learning model for road marking instance segmentation using MLS point clouds[J]. IEEE Transactions on Geoscience and Remote Sensing, 2021, 59 (1): 784- 800
doi: 10.1109/TGRS.2020.2996617
18 LIU S, QI L, QIN H F, et al. Path aggregation network for instance segmentation [C]// IEEE Conference on Computer Vision and Pattern Recognition. Salt Lake City: IEEE, 2018: 8759-8768.
19 周志华. 机器学习[M]. 1版. 北京: 清华大学出版社, 2016: 24-26.
[1] 牛英杰,苏燕辰,程敦诚,廖家,赵海波,高永强. 高铁接触网U型抱箍螺母故障检测算法[J]. 浙江大学学报(工学版), 2021, 55(10): 1912-1921.