Please wait a minute...
Journal of ZheJiang University (Engineering Science)  2022, Vol. 56 Issue (4): 809-815, 832    DOI: 10.3785/j.issn.1008-973X.2022.04.021
    
Lane detection method in complex environments based on instance segmentation
Shu-qin YANG(),Yu-hao MA,Ming-yu FANG,Wei-xing QIAN*(),Jie-xuan CAI,Tong LIU
School of Electrical and Automation Engineering, Nanjing Normal University, Nanjing 210023, China
Download: HTML     PDF(1696KB) HTML
Export: BibTeX | EndNote (RIS)      

Abstract  

An instance segmentation algorithm was adopted and a lane detection method based on improved hybrid task cascade (HTC) network was proposed aiming at the problems of fuzzy feature representation and low semantic information utilization in the lane detection method based on semantic segmentation. Deformable convolution was introduced into the backbone network based on HTC network model in order to improve the ability of the backbone network to extract lane features in complex environments. The structure of feature pyramid network was improved by adding a bottom-up low-level feature transmission path based on feature pyramid network, and dilated convolution was introduced to increase the receptive field of feature map without loss of lane feature information. The accurate location information of lane lines contained in the low-level features was used to improve the accuracy of lane detection. The experimental results show that the improved HTC network model can realize the robust extraction of lane features, obtain better detection performance in complex road environments, and effectively improve the accuracy of lane detection.



Key wordsdriverless technology      lane detection      instance segmentation      deformable convolution      feature pyramid network     
Received: 01 May 2021      Published: 24 April 2022
CLC:  TP 311  
Fund:  国家自然科学基金资助项目(61803208);江苏省自然科学基金资助项目(BK20180726, BK20191371).
Corresponding Authors: Wei-xing QIAN     E-mail: 15651785167@163.com;61192@njnu.edu.cn
Cite this article:

Shu-qin YANG,Yu-hao MA,Ming-yu FANG,Wei-xing QIAN,Jie-xuan CAI,Tong LIU. Lane detection method in complex environments based on instance segmentation. Journal of ZheJiang University (Engineering Science), 2022, 56(4): 809-815, 832.

URL:

https://www.zjujournals.com/eng/10.3785/j.issn.1008-973X.2022.04.021     OR     https://www.zjujournals.com/eng/Y2022/V56/I4/809


基于实例分割的复杂环境车道线检测方法

针对基于语义分割的车道线检测方法存在的特征表述模糊、语义信息利用率较低的问题,采用实例分割算法,提出基于改进混合任务级联(HTC)网络的车道线检测方法. 基于HTC网络模型,在主干网络中引入可变形卷积,提升主干网络对复杂环境中车道线特征的提取能力. 改进特征金字塔网络结构,在特征金字塔网络的基础上添加自底向上的低层特征传递路径,引入空洞卷积,在不损失车道线特征信息的情况下增加特征图感受野,利用低层特征中所包含的车道线的精确定位信息,提高车道线的检测精度. 实验结果表明,改进HTC网络模型可以实现车道线特征的鲁棒提取,在复杂道路环境中可以获得较好的检测性能,有效提高了车道线检测精度.


关键词: 无人驾驶技术,  车道线检测,  实例分割,  可变形卷积,  特征金字塔网络 
Fig.1 Network structure of hybrid task cascade
Fig.2 Network structure of improved hybrid task cascade
Fig.3 Lane feature maps extracted through backbone network
Fig.4 Construction process of lane dataset
主干网络 ResNet mAP AP0.5 AP0.75
C3 C4 C5
Conv Conv DCN 57.5 94.2 62.2
Conv DCN DCN 58.2 94.3 62.8
DCN DCN DCN 58.8 94.5 64.4
Tab.1 Lane detection results based on backbone networks with different feature extraction methods
Fig.5 Change curve of learning rate in training process
Fig.6 Change curve of loss value in training process
增强路径特征
提取方式
M/106 mAP AP0.5 AP0.75
常规卷积[18] 100.76 58.6 94.3 63.5
空洞卷积 100.76 58.8 94.5 64.4
Tab.2 Lane detection results based on different feature extraction methods in augmented path
模型 结构 M/106 mAP AP0.5 AP0.75
Mask R-CNN[20] 两阶段 62.75 54.6 90.8 58.7
Cascade Mask R-CNN[12] 多阶段 95.80 57.7 92.0 62.8
HTC[12] 多阶段 95.93 57.8 92.4 62.8
本文方法 多阶段 100.76 58.8 94.5 64.4
Tab.3 Lane detection results based on different deep learning models
Fig.7 Lane detection results in different environments
Fig.8 Lane detection results in different scenes
[1]   罗杨. 复杂环境下的车道线检测 [D]. 成都: 电子科技大学, 2020.
LUO Yang. Lane detection under complicated environment [D]. Chengdu: University of Electronic Science and Technology of China, 2020.
[2]   NGUYEN T N A, PHUNG S L, BOUZERDOUM A Hybrid deep learning-gaussian process network for pedestrian lane detection in unstructured scenes[J]. IEEE Transactions on Neural Networks and Learning Systems, 2020, 31 (12): 5324- 5338
doi: 10.1109/TNNLS.2020.2966246
[3]   吴一全, 刘莉 基于视觉的车道线检测方法研究进展[J]. 仪器仪表学报, 2019, 40 (12): 92- 109
WU Yi-quan, LIU Li Research and development of the vision-based lane detection methods[J]. Chinese Journal of Scientific Instrument, 2019, 40 (12): 92- 109
[4]   胡忠闯. 基于深度学习的车道线检测算法研究 [D]. 杭州: 浙江大学, 2018.
HU Zhong-chuang. Deep learning based lane markings detection algorithm [D]. Hangzhou: Zhejiang University, 2018.
[5]   YOO H, YANG U, SOHN K Gradient-enhancing conversion for illumination-robust lane detection[J]. IEEE Transactions on Intelligent Transportation Systems, 2013, 14 (3): 1083- 1094
doi: 10.1109/TITS.2013.2252427
[6]   吴骅跃, 赵祥模 基于IPM和边缘图像过滤的多干扰车道线检测[J]. 中国公路学报, 2020, 33 (5): 153- 164
WU Hua-yue, ZHAO Xiang-mo Multi-interference lane recognition based on IPM and edge image filtering[J]. China Journal of Highway and Transport, 2020, 33 (5): 153- 164
doi: 10.3969/j.issn.1001-7372.2020.05.014
[7]   SHEN Y H, BI Y R, YANG Z, et al Lane line detection and recognition based on dynamic ROI and modified firefly algorithm[J]. International Journal of Intelligent Robotics and Applications, 2021, 5 (2): 143- 155
doi: 10.1007/s41315-021-00175-2
[8]   庞彦伟, 修宇璇 基于边缘特征融合和跨连接的车道线语义分割神经网络[J]. 天津大学学报: 自然科学与工程技术版, 2019, 52 (8): 779- 787
PANG Yan-wei, XIU Yu-xuan Lane semantic segmentation neural network based on edge feature merging and skip connections[J]. Journal of Tianjin University: Science and Technology, 2019, 52 (8): 779- 787
[9]   PAN X G, SHI J P, LUO P, et al. Spatial as deep: spatial CNN for traffic scene understanding [C]// 32nd AAAI Conference on Artificial Intelligence. New Orleans: AAAI, 2018: 7276-7283.
[10]   NEVEN D, BRABANDERE B D, GEORGOULIS S, et al. Towards end-to-end lane detection: an instance segmentation approach [C]// 2018 IEEE Intelligent Vehicles Symposium. Suzhou: IEEE, 2018: 286-291.
[11]   CAI Z W, VASCONCELOS N. Cascade R-CNN: delving into high quality object detection [C]// IEEE Conference on Computer Vision and Pattern Recognition. Salt Lake City: IEEE, 2018: 6154-6162.
[12]   CHEN K, PANG J M, WANG J Q, et al. Hybrid task cascade for instance segmentation [C]// IEEE Conference on Computer Vision and Pattern Recognition. Long Beach: IEEE, 2019: 4969-4978.
[13]   HE K M, ZHANG X Y, REN S Q, et al. Deep residual learning for image recognition [C]// IEEE Conference on Computer Vision and Pattern Recognition. Las Vegas: IEEE, 2016: 770-778.
[14]   YU F, KOLTUN V. Multi-scale context aggregation by dilated convolutions [C]// International Conference on Learning Representations. San Juan: [s.n.], 2016: 1-13.
[15]   DAI J F, QI H Z, XIONG Y W, et al. Deformable convolutional networks [C]// IEEE International Conference on Computer Vision. Venice: IEEE, 2017: 764-773.
[16]   LIU Z W, SHEN C, FAN X, et al Scale-aware limited deformable convolutional neural networks for traffic sign detection and classification[J]. IET Intelligent Transport Systems, 2020, 14 (12): 1712- 1722
doi: 10.1049/iet-its.2020.0217
[17]   CHEN S Y, ZHANG Z X, ZHONG R F, et al A dense feature pyramid network-based deep learning model for road marking instance segmentation using MLS point clouds[J]. IEEE Transactions on Geoscience and Remote Sensing, 2021, 59 (1): 784- 800
doi: 10.1109/TGRS.2020.2996617
[18]   LIU S, QI L, QIN H F, et al. Path aggregation network for instance segmentation [C]// IEEE Conference on Computer Vision and Pattern Recognition. Salt Lake City: IEEE, 2018: 8759-8768.
[19]   周志华. 机器学习[M]. 1版. 北京: 清华大学出版社, 2016: 24-26.
[1] Li-feng XU,Hai-fan HUANG,Wei-long DING,Yu-lei FAN. Detection of small fruit target based on improved DenseNet[J]. Journal of ZheJiang University (Engineering Science), 2021, 55(2): 377-385.
[2] Ying-jie NIU,Yan-chen SU,Dun-cheng CHENG,Jia LIAO,Hai-bo ZHAO,Yong-qiang GAO. High-speed rail contact network U-holding nut fault detection algorithm[J]. Journal of ZheJiang University (Engineering Science), 2021, 55(10): 1912-1921.
[3] Yao JIN,Wei ZHANG. Real-time fire detection algorithm with Anchor-Free network architecture[J]. Journal of ZheJiang University (Engineering Science), 2020, 54(12): 2430-2436.