Please wait a minute...
浙江大学学报(工学版)  2025, Vol. 59 Issue (4): 661-668    DOI: 10.3785/j.issn.1008-973X.2025.04.001
交通工程     
基于交通事件短视频资源的多模态情绪特征分析
董镇滔1,5(),徐暟敏1,万清颖2,刘晓菲3,申昊1,李书涵3,奇格奇1,4,5,*()
1. 北京交通大学 交通运输学院,北京 100044
2. 北京交通大学 电气工程学院,北京 100044
3. 北京交通大学 计算机科学与技术学院,北京 100044
4. 北京交通大学 综合交通运输大数据应用技术交通运输行业重点实验室,北京 100044
5. 上海外国语大学 脑机协同信息行为教育部重点实验室,上海 201620
Multimodal emotional feature analysis based on short video resources of traffic incidents
Zhentao DONG1,5(),Kaimin XU1,Qingying WAN2,Xiaofei LIU3,Hao SHEN1,Shuhan LI3,Geqi QI1,4,5,*()
1. School of Traffic and Transportation, Beijing Jiaotong University, Beijing 100044, China
2. School of Electrical Engineering, Beijing Jiaotong University, Beijing 100044, China
3. School of Computer Science and Technology, Beijing Jiaotong University, Beijing 100044, China
4. Key Laboratory of Transport Industry of Big Data Application Technologies for Comprehensive Transport, Beijing Jiaotong University, Beijing 100044, China
5. Key Laboratory of Brain-Machine Intelligence for Information Behavior-Ministry of Education, Shanghai International Studies University, Shanghai 201620, China
 全文: PDF(3695 KB)   HTML
摘要:

为了刻画以短视频形式传播的交通事件舆情对公众情绪的导向,通过文本情感分析和多模态生理信号特征提取,构建生理特征图谱. 爬取抖音平台136个高赞视频及38 805条评论,以所有视频为文档集,单个视频为文档,评论为单词,采用隐狄利克雷分布主题模型进行主题挖掘,获得不同主题的评论单词分布和不同视频的主题分布. 使用基于朴素贝叶斯的SnowNLP计算评论单词的情感分数,分析不同舆情主题表达的情感倾向. 开展神经科学实验,采集脑电、眼动、心电和呼吸等多模态生理信号及情绪评分. 统计检验结果表明,不同情感倾向的视频会诱发不同情绪,不同情绪下脑电的相对谱功率、眨眼频率、呼吸标准差和心电极低频功率等多模态生理特征具有特异性,评论文本中蕴含的情感语义会在视频诱发情绪的基础上对公众情绪造成不同方式的影响.

关键词: 文本信息挖掘情绪特征主题模型短视频舆情脑电图(EEG)    
Abstract:

In order to portray the public emotion orientation caused by the public opinion on traffic incidents disseminated in short videos, a physiological feature graph was constructed by the text sentiment analysis and the multimodal physiological signal feature extraction. This work collected 136 highly-liked videos with 38 805 comments on TikTok. Considering all videos as a document set, with each video treated as a document and comments as words, the latent Dirichlet allocation topic model was adopted to obtain the distribution of comments under different topics and the distribution of topics under different videos. Naive Bayes-based SnowNLP was utilized to calculate the sentiment scores of comments and analyze the sentiment tendencies expressed by different opinion topics. Neuroscience experiments were carried out to collect multimodal physiological signals such as EEG, eye movement, ECG, and respiration as well as emotion ratings. Statistical test results show that videos with different sentiment tendencies induce different emotions, and the multimodal physiological features such as the relative spectral power of EEG, blinking frequency, respiration standard deviation, and the very low-frequency power of ECG are specific under different emotions. The emotional semantics embedded in the comments influence public emotion in various ways beyond that evoked by videos.

Key words: text mining    emotional characteristics    topic model    short video public opinion    electroencephalogram (EEG)
收稿日期: 2024-02-26 出版日期: 2025-04-25
CLC:  U 491  
基金资助: 国家自然科学基金资助项目(72101014);脑机协同信息行为教育部重点实验室开放课题项目(2023JYBKFKT009).
通讯作者: 奇格奇     E-mail: ztdong@bjtu.edu.cn;gqqi@bjtu.edu.cn
作者简介: 董镇滔(2001—),男,博士生,从事脑机交互和人机共驾研究. orcid.org/0009-0004-5957-943X. E-mail:ztdong@bjtu.edu.cn
服务  
把本文推荐给朋友
加入引用管理器
E-mail Alert
作者相关文章  
董镇滔
徐暟敏
万清颖
刘晓菲
申昊
李书涵
奇格奇

引用本文:

董镇滔,徐暟敏,万清颖,刘晓菲,申昊,李书涵,奇格奇. 基于交通事件短视频资源的多模态情绪特征分析[J]. 浙江大学学报(工学版), 2025, 59(4): 661-668.

Zhentao DONG,Kaimin XU,Qingying WAN,Xiaofei LIU,Hao SHEN,Shuhan LI,Geqi QI. Multimodal emotional feature analysis based on short video resources of traffic incidents. Journal of ZheJiang University (Engineering Science), 2025, 59(4): 661-668.

链接本文:

https://www.zjujournals.com/eng/CN/10.3785/j.issn.1008-973X.2025.04.001        https://www.zjujournals.com/eng/CN/Y2025/V59/I4/661

图 1  交通主题词云
图 2  多模态神经科学实验流程
图 3  ErgoLAB多模态生理信号采集平台
图 4  困惑度和一致性随主题数的变化
图 5  交通事件舆情主题分布
图 6  每个主题下视频的情感分数分布
视频编号视频摘要$ \overline S_{\mathrm{E}} $主题编号
42客机起降禁开遮光板,空乘多次提醒,男子拒不配合0.3291
120北京,市区到机场,无障碍车0.7603
80三轮车逆行被撞得支离破碎,交通事故猛于虎,一次侥幸可能会造成终生不幸0.2942
64如何在北京刷手掌乘地铁?今天我来教你0.3102
639月1日起北京取消二环主路公交专用道0.3314
123疑似为给小蓝车挪地儿,俩男子将小黄车扔到路中,自行车堆成小山0.3285
131北京首批“整车无人”自动驾驶车辆上路测试0.4704
117国庆返程高峰,假期最后一天,北京北四环堵车现状;网友:我是自愿上班的0.3496
56暴雨后的北京丰台站大量乘客滞留,有乘客准备在车站过夜,还有学生无奈退改签4次0.2791
4最高礼仪迎接!C919顺利飞抵北京,商业首航成功,点赞祝贺!0.6921
表 1  情绪诱发实验选取视频的详细信息
图 7  情绪诱发实验问卷汇总结果
图 8  情绪诱发实验问卷统计分析结果
图 9  脑电-愉悦度评分相关性分析结果
图 10  短视频诱发情绪的生理特征差异
图 11  观看评论引起的脑电特征差异
1 MA X, LIU W, ZHOU X, et al Evolution of online public opinion during meteorological disasters[J]. Environmental Hazards, 2020, 19 (4): 375- 397
doi: 10.1080/17477891.2019.1685932
2 JIA F, CHEN C C Emotional characteristics and time series analysis of Internet public opinion participants based on emotional feature words[J]. International Journal of Advanced Robotic Systems, 2020, 17 (1): 1- 11
3 于群, 英启昊, 曹娜, 等. 基于属性数学理论的电网停电事故网络舆情风险评估[E/OL]. (2024−01−12)[2024−02−20]. https://doi.org/10.13335/j.1000-3673.pst.2023.1923.
4 REN S, GONG C, ZHANG C, et al Public opinion communication mechanism of public health emergencies in Weibo: take the COVID-19 epidemic as an example[J]. Frontiers in Public Health, 2023, 11: 1276083
doi: 10.3389/fpubh.2023.1276083
5 ALI F, KWAK D, KHAN P, et al Transportation sentiment analysis using word embedding and ontology-based topic modeling[J]. Knowledge-Based Systems, 2019, 174: 27- 42
doi: 10.1016/j.knosys.2019.02.033
6 CHEN Y, CHEN H, LIU L, et al. A review on social media data for social transportation [C]// Proceedings of the CICTP 2021 . Xi’an: American Society of Civil Engineers, 2021: 126–136.
7 QI B, COSTIN A, JIA M A framework with efficient extraction and analysis of Twitter data for evaluating public opinions on transportation services[J]. Travel Behaviour and Society, 2020, 21: 10- 23
doi: 10.1016/j.tbs.2020.05.005
8 汤文蕴, 丁子羿, 马健霄 不同类型重大公共事件下交通管控舆情分析[J]. 浙江大学学报: 工学版, 2022, 56 (11): 2271- 2279
TANG Wenyun, DING Ziyi, MA Jianxiao Public opinion analysis on traffic control under different major public events[J]. Journal of Zhejiang University: Engineering Science, 2022, 56 (11): 2271- 2279
9 BAI Z, MA S, LI G A WeChat official account reading quantity prediction model based on text and image feature extraction[J]. IEEE Access, 2022, 10: 28348- 28360
doi: 10.1109/ACCESS.2022.3157715
10 LI J, LI Z, MA X, et al Sentiment analysis on online videos by time-sync comments[J]. Entropy, 2023, 25 (7): 1016
doi: 10.3390/e25071016
11 LIU J, HU X, SHEN X, et al Electrophysiological representations of multivariate human emotion experience[J]. Cognition and Emotion, 2024, 38 (3): 378- 388
doi: 10.1080/02699931.2023.2297272
12 PFEIFFER C, HOLLENSTEIN N, ZHANG C, et al Neural dynamics of sentiment processing during naturalistic sentence reading[J]. NeuroImage, 2020, 218: 116934
doi: 10.1016/j.neuroimage.2020.116934
13 赵卿, 张雪英, 陈桂军, 等 基于模态注意力图卷积特征融合的EEG和fNIRS情感识别[J]. 浙江大学学报: 工学版, 2023, 57 (10): 1987- 1997
ZHAO Qing, ZHANG Xueying, CHEN Guijun, et al EEG and fNIRS emotion recognition based on modality attention graph convolution feature fusion[J]. Journal of Zhejiang University: Engineering Science, 2023, 57 (10): 1987- 1997
14 郑伟龙, 石振锋, 吕宝粮 用异质迁移学习构建跨被试脑电情感模型[J]. 计算机学报, 2020, 43 (2): 177- 189
ZHENG Weilong, SHI Zhenfeng, LU Baoliang Building cross-subject EEG-based affective models using heterogeneous transfer learning[J]. Chinese Journal of Computers, 2020, 43 (2): 177- 189
doi: 10.11897/SP.J.1016.2020.00177
15 BLEI D M, NG A Y, JORDAN M I Latent Dirichlet allocation[J]. The Journal of Machine Learning Research, 2003, 3: 993- 1022
16 奇格奇, 张子贤, 卫振林, 等 基于语义挖掘的快递运输货品风险评价研究[J]. 交通运输系统工程与信息, 2021, 21 (4): 248- 255
QI Geqi, ZHANG Zixian, WEI Zhenlin, et al Risk evaluation of express delivery goods based on semantic mining[J]. Journal of Transportation Systems Engineering and Information Technology, 2021, 21 (4): 248- 255
17 SYED S, SPRUIT M. Full-text or abstract? Examining topic coherence scores using latent dirichlet allocation [C]// Proceedings of the IEEE International Conference on Data Science and Advanced Analytics . Tokyo: IEEE, 2017: 165–174.
[1] 李宜轩,李颖,肖倩,王灵月,尹宁,杨硕. 不同情绪错误记忆的脑电微状态功能网络分析[J]. 浙江大学学报(工学版), 2025, 59(1): 49-61.
[2] 陆佳炜,郑嘉弘,李端倪,徐俊,肖刚. 面向服务聚类的短文本优化主题模型[J]. 浙江大学学报(工学版), 2022, 56(12): 2416-2425.
[3] 涂鼎, 陈岭, 陈根才, 吴勇, 王敬昌. 基于在线层次化非负矩阵分解的文本流主题检测[J]. 浙江大学学报(工学版), 2016, 50(8): 1618-1626.
[4] 苏劲松, 董槐林, 陈毅东, 史晓东, 吴清强. 引入基于主题复述知识的统计机器翻译模型[J]. 浙江大学学报(工学版), 2014, 48(10): 1843-1849.