Please wait a minute...
浙江大学学报(工学版)  2023, Vol. 57 Issue (8): 1636-1643    DOI: 10.3785/j.issn.1008-973X.2023.08.016
土木工程、交通工程     
基于时空融合的多头注意力车辆轨迹预测
宋秀兰1(),董兆航1,单杭冠2,陆炜杰1
1. 浙江工业大学 信息工程学院,浙江 杭州 310023
2. 浙江大学 信息与电子工程学院,浙江 杭州 310027
Vehicle trajectory prediction based on temporal-spatial multi-head attention mechanism
Xiu-lan SONG1(),Zhao-hang DONG1,Hang-guan SHAN2,Wei-jie LU1
1. College of Information Engineering, Zhejiang University of Technology, Hangzhou 310023
2. College of Information and Electronic Engineering, Zhejiang University, Hangzhou 310027
 全文: PDF(1310 KB)   HTML
摘要:

针对时空维度特征影响自动驾驶车辆轨迹精度的问题,提出基于时空融合的多头注意力(TSMHA)车辆轨迹预测模型,对于空间与时间2个维度的特征信息,分别使用多头注意力机制提取车辆空间交互感知与时间运动模式. 为了获得互补特征,并除去特征数据中的冗余,将处理后的时空特征信息传输至门控特征融合模型进行特征融合. 使用基于长短期记忆(LSTM)的编解码器结构,考虑编码与解码2个过程中轨迹之间潜在的相互作用,循环生成目标车辆未来预测轨迹. 在训练过程中使用L2损失函数,以此降低预测轨迹与真实轨迹的差值. 实验表明,与对比算法模型相比,在直线高速公路、城市十字路口、环岛场景下,本研究所提出的模型的精度分别提高了3.95%、 15.64%、31.40%.

关键词: 车辆智能决策轨迹预测时空融合注意力机制多目标车辆神经网络    
Abstract:

Aiming at the problem that temporal-spatial features affect the trajectory prediction accuracy of autonomous vehicle, a temporal-spatial multi-head attention (TSMHA) vehicle trajectory prediction model was proposed. For the feature information of spatial and temporal dimensions, the multi-head attention mechanism was used to extract the spatial interaction perception and temporal motion pattern of the vehicle. The processed temporal-spatial feature information was transmitted to the gate fusion model for feature fusion, in order to obtain complementary features and remove redundancy. Using the encoder-decoder structure based on long short-term memory (LSTM), future trajectories were recurrently generated considering the potential interaction between trajectories during encoding and decoding. In the training process, the L2 loss function was used to reduce the difference between the predicted trajectory and the ground-truth trajectory. Experimental results show that, compared with the comparison models, the accuracy of the proposed model was improved by 3.95% in the highway, 15.64% in the urban roads, and 31.40% in the roundabout scenario.

Key words: vehicle intelligent decision-making    trajectory prediction    temporal-spatial attention mechanism    multi-target vehicle    neural network
收稿日期: 2022-10-11 出版日期: 2023-08-31
CLC:  U 491  
基金资助: 国家自然科学基金资助项目(62273307);浙江省公益性技术应用研究资助项目(LGF22F030013);浙江省重点研发计划资助项目(2021C11096)
作者简介: 宋秀兰(1982—),女,副教授,从事智能网联车辆和深度学习应用研究. orcid.org/0000-0001-8802-7010. E-mail: songxl2008@zjut.edu.cn
服务  
把本文推荐给朋友
加入引用管理器
E-mail Alert
作者相关文章  
宋秀兰
董兆航
单杭冠
陆炜杰

引用本文:

宋秀兰,董兆航,单杭冠,陆炜杰. 基于时空融合的多头注意力车辆轨迹预测[J]. 浙江大学学报(工学版), 2023, 57(8): 1636-1643.

Xiu-lan SONG,Zhao-hang DONG,Hang-guan SHAN,Wei-jie LU. Vehicle trajectory prediction based on temporal-spatial multi-head attention mechanism. Journal of ZheJiang University (Engineering Science), 2023, 57(8): 1636-1643.

链接本文:

https://www.zjujournals.com/eng/CN/10.3785/j.issn.1008-973X.2023.08.016        https://www.zjujournals.com/eng/CN/Y2023/V57/I8/1636

图 1  轨迹预测模型总架构
图 2  门控特征融合模型架构
图 3  不同场景下的空间范围示意图
L /m RMSE
te=1 s te=2 s te=3 s te=4 s te=5 s
50 0.66 1.30 1.95 3.01 4.14
55 0.63 1.27 1.92 2.94 4.05
60 0.56 1.19 1.87 2.84 3.93
65 0.61 1.23 1.92 2.88 4.01
70 0.64 1.26 1.91 2.97 4.07
表 1  不同长度L对模型预测性能的影响
模型 RMSE
te=1 s te=2 s te=3 s te=4 s te=5 s
1 0.75 1.41 2.26 3.21 4.37
2 0.68 1.52 2.31 2.93 4.25
3 0.61 1.38 2.24 3.04 4.18
4 0.56 1.19 1.87 2.84 3.93
表 2  门控特征融合消融实验
模型 RMSE
te=1 s te=2 s te=3 s te=4 s te=5 s
V-LSTM 0.70 1.79 3.22 4.96 7.04
CS-LSTM 0.61 1.27 2.09 3.10 4.37
S-GAN 0.57 1.32 2.22 3.26 4.40
MATF 0.66 1.34 2.08 2.97 4.13
STA 0.56 1.32 2.03 3.08 4.24
SIT 0.58 1.23 1.99 2.96 4.05
PIP 0.55 1.18 1.94 2.88 4.04
TSMHA 0.56 1.19 1.87 2.84 3.93
表 3  高速公路场景不同模型的预测性能对比
图 4  高速公路场景下的轨迹可视化
模型 RMSE
te=1 s te=2 s te=3 s te=4 s te=5 s
V-LSTM 0.81 1.93 4.48 6.20 8.13
CS-LSTM 0.78 1.61 2.67 3.44 4.56
S-GAN 0.75 1.56 3.03 3.58 4.92
MATF 0.82 1.51 2.88 3.15 4.58
STA 0.68 1.53 2.71 3.24 4.66
SIT 0.73 1.45 2.42 3.16 4.35
PIP 0.74 1.32 2.41 3.03 4.27
TSMHA 0.67 1.24 2.29 2.87 4.13
表 4  城市道路场景下不同模型的预测性能对比
图 5  城市道路场景下的轨迹可视化
模型 RMSE
te=1 s te=2 s te=3 s
V-LSTM 0.89 2.15 6.42
CS-LSTM 0.83 1.86 3.59
S-GAN 0.78 1.84 3.83
MATF 0.85 2.03 3.85
STA 0.77 1.73 3.24
SIT 0.79 1.76 3.06
PIP 0.83 1.63 2.85
TSMHA 0.75 1.34 2.46
表 5  环岛道路场景不同模型的预测性能对比
图 6  环岛场景下的轨迹可视化
图 7  时间和空间注意力权重示意图
1 LI X L, XIA J, CHEN X Y, et al SIT: a spatial interaction-aware transformer-based model for freeway trajectory prediction[J]. ISPRS International Journal of Geo-Information, 2022, 11 (2): 79- 94
doi: 10.3390/ijgi11020079
2 ELNAGAR A. Prediction of moving objects in dynamic environments using kalman filters [C]// IEEE International Symposium on Computational Intelligence in Robotics and Automation. New York: IEEE, 2001: 414-419.
3 季学武, 费 聪, 何祥坤, 等 基于 LSTM 网络的驾驶意图识别及车辆轨迹预测[J]. 中国公路学报, 2019, 32 (6): 34- 42
JI Xue-wu, FEI Cong, HE Xiang-kun, et al Intention recognition and trajectory prediction for vehicles using LSTM network[J]. China Journal of Highway and Transport, 2019, 32 (6): 34- 42
4 韩皓, 谢天 基于注意力 Seq2Seq 网络的高速公路交织区车辆变道轨迹预测[J]. 中国公路学报, 2020, 33 (6): 106- 118
HAN Hao, XIE Tian Lane change trajectory prediction of vehicles in highway interweaving area using Seq2Seq-attention network[J]. China Journal of Highway and Transport, 2020, 33 (6): 106- 118
5 FEI C, HE X K, JI X Multi-modal vehicle trajectory prediction based on mutual information[J]. IET Intelligent Transport Systems, 2020, 14 (3): 148- 153
doi: 10.1049/iet-its.2019.0299
6 ALAHI A, GOEL K, RAMANATHAN V, et al. Social LSTM: human trajectory prediction in crowded spaces [C]// IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR). New York: IEEE, 2016: 961-971.
7 XIN L, WANG P, CHAN C, et al. Intention-aware long horizon trajectory prediction of surrounding vehicles using dual LSTM networks [C]// 21st International Conference on Intelligent Transportation Systems. New York: IEEE, 2018: 1441-1446.
8 HOU L, XIN L, LE S E, et al Interactive trajectory prediction of surrounding road users for autonomous driving using structural-LSTM network[J]. IEEE Transactions on Intelligent Transportation Systems, 2020, 21 (11): 4615- 4625
doi: 10.1109/TITS.2019.2942089
9 KIM B, KANG C M, KIM J, et al. Probabilistic vehicle trajectory prediction over occupancy grid map via recurrent neural network [C]// IEEE 20th International Conference on Intelligent Transportation Systems. New York: IEEE, 2017: 399-404.
10 WU Y, CHEN G, LI Z, et al HSTA: a hierarchical spatio-temporal attention model for trajectory prediction[J]. IEEE Transactions on Vehicular Technology, 2021, 70 (11): 11295- 11307
doi: 10.1109/TVT.2021.3115018
11 DEO N, TRIVEDI M M. Convolutional social pooling for vehicle trajectory prediction [C]// IEEE/CVF Conference on Computer Vision and Pattern Recognition Workshops. New York: IEEE, 2018: 1468-1476.
12 MO X Y, XING Y, LV C. Interaction-aware trajectory prediction of connected vehicles using CNN-LSTM networks [C]// 46th Annual Conference of the IEEE-Industrial- Electronics-Society (IECON). New York: IEEE, 2020: 5057-5062.
13 MUKHERJEE S, WANG S, WEALLACE A. Interacting vehicle trajectory prediction with convolutional recurrent neural networks [C]// IEEE International Conference on Robotics and Automation. New York: IEEE, 2020: 4336-4342.
14 GUPTA A, JOHNSON J, LI F F, et al. Social GAN: socially acceptable trajectories with generative Adversarial networks [C]// IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR). New York: IEEE, 2018: 2255-2264.
15 MNIH V, HEESS N, GRAVES A, et al. Recurrent models of visual attention [C]// 28th Conference on Neural Information Processing Systems. California: NeurIPS, 2014: 2204-2212.
16 LIN L, LI W Z, BI H K, et al Vehicle trajectory prediction using LSTMs with spatial-temporal attention mechanisms[J]. IEEE Intelligent Transportation Systems Magazine, 2021, 14 (2): 197- 208
17 YANG J, SUN X, WANG R G, et al PTPGC: pedestrian trajectory prediction by graph attention network with ConvLSTM[J]. Robotics and Autonomous Systems, 2022, 148: 103931- 103942
doi: 10.1016/j.robot.2021.103931
18 YAN J, PENG Z F, YIN H L, et al Trajectory prediction for intelligent vehicles using spatial-attention mechanism[J]. IET Intelligent Transport Systems, 2020, 14 (13): 1855- 1863
doi: 10.1049/iet-its.2020.0274
19 CHEN X B, ZHANG H J, ZHAO F, et al Intention-aware vehicle trajectory prediction based on spatial-temporal dynamic attention network for internet of vehicles[J]. IEEE Transactions on Intelligent Transportation Systems, 2022, 23 (10): 19471- 19483
doi: 10.1109/TITS.2022.3170551
20 MOHAMED A, QIAN K, ELHOSEINY M, et al. Social-STGCNN: a social spatio-temporal graph convolutional neural network for human trajectory prediction [C]// IEEE/CVF Conference on Computer Vision and Pattern Recognition Workshops. New York: IEEE, 2020: 14412-14420.
21 YAN S, XIONG Y, LIN D. Spatial temporal graph convolutional networks for skeleton-based action recognition [C]// AAAI Conference on Artificial Intelligence. California: AAAI, 2018, 32(1): 7444-7452.
22 XUE H, HUYNH D Q, REYNOLDS M A location-velocity-temporal attention LSTM model for pedestrian trajectory prediction[J]. IEEE ACCESS, 2020, 8: 44576- 44589
doi: 10.1109/ACCESS.2020.2977747
23 MESSAOUD K, YAHIAOUI I, VERROUST-BLONDET A, et al Attention based vehicle trajectory prediction[J]. IEEE Transactions on Intelligent Vehicles, 2021, 6 (1): 175- 185
doi: 10.1109/TIV.2020.2991952
24 VASWANI A, SHAZEER N, PARMAR N, et al. Attention is all you need [C]// 31st Conference on Neural Information Processing Systems. California: NeurIPS, 2017: 5998-6008.
25 ZHAO T, XU Y, MONFORT M, et al. Multi-agent tensor fusion for contextual trajectory prediction [C]// IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR). New York: IEEE, 2019: 12118-12126.
[1] 王殿海,谢瑞,蔡正义. 基于最优汇集时间间隔的城市间断交通流预测[J]. 浙江大学学报(工学版), 2023, 57(8): 1607-1617.
[2] 权巍,蔡永青,王超,宋佳,孙鸿凯,李林轩. 基于3D-ResNet双流网络的VR病评估模型[J]. 浙江大学学报(工学版), 2023, 57(7): 1345-1353.
[3] 周欣磊,顾海挺,刘晶,许月萍,耿芳,王冲. 基于集成学习与深度学习的日供水量预测方法[J]. 浙江大学学报(工学版), 2023, 57(6): 1120-1127.
[4] 张海波,蔡磊,任俊平,王汝言,刘富. 基于Transformer的高效自适应语义分割网络[J]. 浙江大学学报(工学版), 2023, 57(6): 1205-1214.
[5] 覃承进,蒋俊正. 基于深度神经网络的雷达距离超分辨方法[J]. 浙江大学学报(工学版), 2023, 57(6): 1215-1223.
[6] 周传华,操礼春,周家亿,詹凤. 图卷积融合计算时效网络节点重要性评估分析[J]. 浙江大学学报(工学版), 2023, 57(5): 930-938.
[7] 熊帆,陈田,卞佰成,刘军. 基于卷积循环神经网络的芯片表面字符识别[J]. 浙江大学学报(工学版), 2023, 57(5): 948-956.
[8] 王誉翔,钟智伟,夏鹏程,黄亦翔,刘成良. 基于改进Transformer的复合故障解耦诊断方法[J]. 浙江大学学报(工学版), 2023, 57(5): 855-864.
[9] 吕鑫栋,李娇,邓真楠,冯浩,崔欣桐,邓红霞. 基于改进Transformer的结构化图像超分辨网络[J]. 浙江大学学报(工学版), 2023, 57(5): 865-874.
[10] 张剑钊,郭继昌,汪昱东. 基于融合逆透射率图的水下图像增强算法[J]. 浙江大学学报(工学版), 2023, 57(5): 921-929.
[11] 张超凡,乔一铭,曹露,王志刚,崔少伟,王硕. 基于神经形态的触觉滑动感知方法[J]. 浙江大学学报(工学版), 2023, 57(4): 683-692.
[12] 徐少铭,李钰,袁晴龙. 基于强化学习和3σ准则的组合剪枝方法[J]. 浙江大学学报(工学版), 2023, 57(3): 486-494.
[13] 周青松,蔡晓东,刘家良. 结合社交影响和长短期偏好的个性化推荐算法[J]. 浙江大学学报(工学版), 2023, 57(3): 495-502.
[14] 曾菊香,王平辉,丁益东,兰林,蔡林熹,管晓宏. 面向节点分类的图神经网络节点嵌入增强模型[J]. 浙江大学学报(工学版), 2023, 57(2): 219-225.
[15] 杨长春,叶赞挺,刘半藤,王柯,崔海东. 基于多源信息融合的医学图像分割方法[J]. 浙江大学学报(工学版), 2023, 57(2): 226-234.