|
|
Multimodal emotional feature analysis based on short video resources of traffic incidents |
Zhentao DONG1,5( ),Kaimin XU1,Qingying WAN2,Xiaofei LIU3,Hao SHEN1,Shuhan LI3,Geqi QI1,4,5,*( ) |
1. School of Traffic and Transportation, Beijing Jiaotong University, Beijing 100044, China 2. School of Electrical Engineering, Beijing Jiaotong University, Beijing 100044, China 3. School of Computer Science and Technology, Beijing Jiaotong University, Beijing 100044, China 4. Key Laboratory of Transport Industry of Big Data Application Technologies for Comprehensive Transport, Beijing Jiaotong University, Beijing 100044, China 5. Key Laboratory of Brain-Machine Intelligence for Information Behavior-Ministry of Education, Shanghai International Studies University, Shanghai 201620, China |
|
|
Abstract In order to portray the public emotion orientation caused by the public opinion on traffic incidents disseminated in short videos, a physiological feature graph was constructed by the text sentiment analysis and the multimodal physiological signal feature extraction. This work collected 136 highly-liked videos with 38 805 comments on TikTok. Considering all videos as a document set, with each video treated as a document and comments as words, the latent Dirichlet allocation topic model was adopted to obtain the distribution of comments under different topics and the distribution of topics under different videos. Naive Bayes-based SnowNLP was utilized to calculate the sentiment scores of comments and analyze the sentiment tendencies expressed by different opinion topics. Neuroscience experiments were carried out to collect multimodal physiological signals such as EEG, eye movement, ECG, and respiration as well as emotion ratings. Statistical test results show that videos with different sentiment tendencies induce different emotions, and the multimodal physiological features such as the relative spectral power of EEG, blinking frequency, respiration standard deviation, and the very low-frequency power of ECG are specific under different emotions. The emotional semantics embedded in the comments influence public emotion in various ways beyond that evoked by videos.
|
Received: 26 February 2024
Published: 25 April 2025
|
|
Fund: 国家自然科学基金资助项目(72101014);脑机协同信息行为教育部重点实验室开放课题项目(2023JYBKFKT009). |
Corresponding Authors:
Geqi QI
E-mail: ztdong@bjtu.edu.cn;gqqi@bjtu.edu.cn
|
基于交通事件短视频资源的多模态情绪特征分析
为了刻画以短视频形式传播的交通事件舆情对公众情绪的导向,通过文本情感分析和多模态生理信号特征提取,构建生理特征图谱. 爬取抖音平台136个高赞视频及38 805条评论,以所有视频为文档集,单个视频为文档,评论为单词,采用隐狄利克雷分布主题模型进行主题挖掘,获得不同主题的评论单词分布和不同视频的主题分布. 使用基于朴素贝叶斯的SnowNLP计算评论单词的情感分数,分析不同舆情主题表达的情感倾向. 开展神经科学实验,采集脑电、眼动、心电和呼吸等多模态生理信号及情绪评分. 统计检验结果表明,不同情感倾向的视频会诱发不同情绪,不同情绪下脑电的相对谱功率、眨眼频率、呼吸标准差和心电极低频功率等多模态生理特征具有特异性,评论文本中蕴含的情感语义会在视频诱发情绪的基础上对公众情绪造成不同方式的影响.
关键词:
文本信息挖掘,
情绪特征,
主题模型,
短视频舆情,
脑电图(EEG)
|
|
[1] |
MA X, LIU W, ZHOU X, et al Evolution of online public opinion during meteorological disasters[J]. Environmental Hazards, 2020, 19 (4): 375- 397
doi: 10.1080/17477891.2019.1685932
|
|
|
[2] |
JIA F, CHEN C C Emotional characteristics and time series analysis of Internet public opinion participants based on emotional feature words[J]. International Journal of Advanced Robotic Systems, 2020, 17 (1): 1- 11
|
|
|
[3] |
于群, 英启昊, 曹娜, 等. 基于属性数学理论的电网停电事故网络舆情风险评估[E/OL]. (2024−01−12)[2024−02−20]. https://doi.org/10.13335/j.1000-3673.pst.2023.1923.
|
|
|
[4] |
REN S, GONG C, ZHANG C, et al Public opinion communication mechanism of public health emergencies in Weibo: take the COVID-19 epidemic as an example[J]. Frontiers in Public Health, 2023, 11: 1276083
doi: 10.3389/fpubh.2023.1276083
|
|
|
[5] |
ALI F, KWAK D, KHAN P, et al Transportation sentiment analysis using word embedding and ontology-based topic modeling[J]. Knowledge-Based Systems, 2019, 174: 27- 42
doi: 10.1016/j.knosys.2019.02.033
|
|
|
[6] |
CHEN Y, CHEN H, LIU L, et al. A review on social media data for social transportation [C]// Proceedings of the CICTP 2021 . Xi’an: American Society of Civil Engineers, 2021: 126–136.
|
|
|
[7] |
QI B, COSTIN A, JIA M A framework with efficient extraction and analysis of Twitter data for evaluating public opinions on transportation services[J]. Travel Behaviour and Society, 2020, 21: 10- 23
doi: 10.1016/j.tbs.2020.05.005
|
|
|
[8] |
汤文蕴, 丁子羿, 马健霄 不同类型重大公共事件下交通管控舆情分析[J]. 浙江大学学报: 工学版, 2022, 56 (11): 2271- 2279 TANG Wenyun, DING Ziyi, MA Jianxiao Public opinion analysis on traffic control under different major public events[J]. Journal of Zhejiang University: Engineering Science, 2022, 56 (11): 2271- 2279
|
|
|
[9] |
BAI Z, MA S, LI G A WeChat official account reading quantity prediction model based on text and image feature extraction[J]. IEEE Access, 2022, 10: 28348- 28360
doi: 10.1109/ACCESS.2022.3157715
|
|
|
[10] |
LI J, LI Z, MA X, et al Sentiment analysis on online videos by time-sync comments[J]. Entropy, 2023, 25 (7): 1016
doi: 10.3390/e25071016
|
|
|
[11] |
LIU J, HU X, SHEN X, et al Electrophysiological representations of multivariate human emotion experience[J]. Cognition and Emotion, 2024, 38 (3): 378- 388
doi: 10.1080/02699931.2023.2297272
|
|
|
[12] |
PFEIFFER C, HOLLENSTEIN N, ZHANG C, et al Neural dynamics of sentiment processing during naturalistic sentence reading[J]. NeuroImage, 2020, 218: 116934
doi: 10.1016/j.neuroimage.2020.116934
|
|
|
[13] |
赵卿, 张雪英, 陈桂军, 等 基于模态注意力图卷积特征融合的EEG和fNIRS情感识别[J]. 浙江大学学报: 工学版, 2023, 57 (10): 1987- 1997 ZHAO Qing, ZHANG Xueying, CHEN Guijun, et al EEG and fNIRS emotion recognition based on modality attention graph convolution feature fusion[J]. Journal of Zhejiang University: Engineering Science, 2023, 57 (10): 1987- 1997
|
|
|
[14] |
郑伟龙, 石振锋, 吕宝粮 用异质迁移学习构建跨被试脑电情感模型[J]. 计算机学报, 2020, 43 (2): 177- 189 ZHENG Weilong, SHI Zhenfeng, LU Baoliang Building cross-subject EEG-based affective models using heterogeneous transfer learning[J]. Chinese Journal of Computers, 2020, 43 (2): 177- 189
doi: 10.11897/SP.J.1016.2020.00177
|
|
|
[15] |
BLEI D M, NG A Y, JORDAN M I Latent Dirichlet allocation[J]. The Journal of Machine Learning Research, 2003, 3: 993- 1022
|
|
|
[16] |
奇格奇, 张子贤, 卫振林, 等 基于语义挖掘的快递运输货品风险评价研究[J]. 交通运输系统工程与信息, 2021, 21 (4): 248- 255 QI Geqi, ZHANG Zixian, WEI Zhenlin, et al Risk evaluation of express delivery goods based on semantic mining[J]. Journal of Transportation Systems Engineering and Information Technology, 2021, 21 (4): 248- 255
|
|
|
[17] |
SYED S, SPRUIT M. Full-text or abstract? Examining topic coherence scores using latent dirichlet allocation [C]// Proceedings of the IEEE International Conference on Data Science and Advanced Analytics . Tokyo: IEEE, 2017: 165–174.
|
|
|
|
Viewed |
|
|
|
Full text
|
|
|
|
|
Abstract
|
|
|
|
|
Cited |
|
|
|
|
|
Shared |
|
|
|
|
|
Discussed |
|
|
|
|