Please wait a minute...
浙江大学学报(工学版)  2024, Vol. 58 Issue (9): 1822-1831    DOI: 10.3785/j.issn.1008-973X.2024.09.007
土木与建筑工程     
基于YOLOv5和Mask-RCNN组合模型的社交媒体内涝灾害分析
张凌嘉(),周欣磊,许月萍,江衍铭*()
浙江大学 建筑工程学院,浙江 杭州 310058
Analysis of inundation from social media based on integrated YOLOv5 and Mask-RCNN model
Lingjia ZHANG(),Xinlei ZHOU,Yueping XU,Yenming CHIANG*()
College of Civil Engineering and Architecture, Zhejiang University, Hangzhou 310058, China
 全文: PDF(2093 KB)   HTML
摘要:

由于缺少淹没实测数据,针对城市市区淹没深度测量数据需求,基于社交媒体与深度学习技术,探索根据社交媒体用户上传信息提取洪水淹没信息的新方法. 研发基于YOLOv5与Mask-RCNN组合模型的实例分割算法,制作轿车众多关键部位的识别数据集. 根据模型训练结果实现全新的市区淹没事件的淹没高度提取方法. 通过输入淹没图像对城市内涝中淹没点位与淹没深度进行预测,与淹没重演模型得到的数据进行比较. 基于模拟淹没实验来制作验证数据集,验证该方法的可行性. 结果表明,所研发的YOLOv5 与 Mask-RCNN组合模型的纳什效率系数为0.98. 使用郑州市“7·20”城市内涝的实际社交媒体图像进行可靠性验证. 结果表明,所提方法能够为城市市区内涝淹没过程提供有效数据来源.

关键词: 城市内涝社交媒体YOLOv5Mask-RCNN水深提取图像识别    
Abstract:

Due to the lack of empirical inundation data, a novel method was explored for extracting flood inundation information from social media uploads using social media and deep learning technologies, addressing the need for urban area inundation depth measurement data. Firstly, the image segmentation model based on the YOLOv5 and Mask-RCNN was constructed and the identification dataset of various key parts of vehicles was constructed. A novel method for extracting the inundation height during urban flooding events was proposed according to the model training results. By inputting inundation images, the prediction of submerged locations and depths in urban inundation was conducted. These predictions were then compared with the data obtained from the inundation recurrence model. A validation dataset was formed by conducting the simulated inundation experiments, to verify the feasibility of the proposed model. Results showed that the Nash efficiency coefficient of the proposed model was 0.98. Moreover, the social media images from the actual urban flooding during the "7·20" event in Zhengzhou City were used to verify the reliability of the proposed model. Results showed that the proposed method can provide an effective data source for the process of urban inundation.

Key words: urban inundation    social media    YOLOv5    Mask-RCNN    water depth extraction    image recognition
收稿日期: 2023-12-03 出版日期: 2024-08-30
CLC:  TV 122  
基金资助: 浙江省重点研发计划资助项目(2021C03017) ;浙江省自然基金重点研究资助项目(LZ20E090001).
通讯作者: 江衍铭     E-mail: 22312102@zju.edu.cn;chiangym@zju.edu.cn
作者简介: 张凌嘉(2001—),男,硕士生,从事内涝模拟和人工智能研究. orcid.org/0009-0003-4274-063X. E-mail:22312102@zju.edu.cn
服务  
把本文推荐给朋友
加入引用管理器
E-mail Alert
作者相关文章  
张凌嘉
周欣磊
许月萍
江衍铭

引用本文:

张凌嘉,周欣磊,许月萍,江衍铭. 基于YOLOv5和Mask-RCNN组合模型的社交媒体内涝灾害分析[J]. 浙江大学学报(工学版), 2024, 58(9): 1822-1831.

Lingjia ZHANG,Xinlei ZHOU,Yueping XU,Yenming CHIANG. Analysis of inundation from social media based on integrated YOLOv5 and Mask-RCNN model. Journal of ZheJiang University (Engineering Science), 2024, 58(9): 1822-1831.

链接本文:

https://www.zjujournals.com/eng/CN/10.3785/j.issn.1008-973X.2024.09.007        https://www.zjujournals.com/eng/CN/Y2024/V58/I9/1822

图 1  利用社交媒体图像提取市区淹没深度方法的流程图
图 2  YOLOv5神经网络模型架构
图 3  Mask-RCNN神经网络模型架构
位置h1/mm
车轮胎车把手车前灯后视镜前车牌后车牌
顶部6609147371067483622
底部100813640940343480
表 1  轿车的关键部位高度转换标准
图 4  关键物体部位识别以及水深测量方法示意图
淹没等级h2/mm
轿车SUV公交货车
17580150235
2225240450705
33754007501175
452556010501645
567572013502115
682588016502585
7975104019503055
81125120022503525
91275136025503995
101425152028504465
表 2  关键物体淹没等级高度转换标准
用户名内容发布时间关键词地点
用户A#郑州大暴雨#淋得真棒 高抬腿走路 水淹大腿了2021?07?21 21∶11:03#郑州大暴雨#郑州·花园路
用户B#郑州大暴雨# 希望排水系统给力一些!2021?07?21 20:22:03#郑州大暴雨#郑州·祝福红城
用户C#郑州特大暴雨为千年一遇##郑州暴雨# 看图2021?07?21 00:22:03#郑州特大暴雨为千年一遇#郑州·河南省体育馆
表 3  社交媒体文字信息获取示例
图 5  YOLOv5图像识别模拟实验示例
图 6  Mask-RCNN图像识别模拟实验示例
模型$ \mathrm{M}\mathrm{A}\mathrm{E} $$ \mathrm{M}\mathrm{A}\mathrm{P}\mathrm{E} $$ \mathrm{R}\mathrm{M}\mathrm{S}\mathrm{E} $$ \mathrm{N}\mathrm{S}\mathrm{E} $
YOLOv596.690.1497146.300.9486
Mask-RCNN137.400.5818227.600.6949
YOLOv5
&Mask-RCNN
54.420.106791.670.9820
表 4  3种水深提取模型的模拟实验误差分析
图 7  3种水深提取模型的模拟图像实验结果对比
图 8  两模型的社交媒体图像识别实验结果示例
地点$ H $/mm
图像1图像2图像3图像4图像5
郑州·祝福红城75.075.0562.0718.0737.0
郑州·富田太阳城75.0562.0562.0197.0
郑州·火车站56275.075.075.0
郑州·商都嘉园穆庄小区东院75.075.075.075.0737.0
表 5  城市市区地点内涝水深示例
20 章卫军, 廖青桃, 杨森, 等 从郑州“2021.7. 20”水灾模型推演看城市洪涝风险管理[J]. 中国防汛抗旱, 2021, 31 (9): 1- 4
ZHANG Weijun, LIAO Qingtao, YANG Sen, et al Thoughts and inspirations: urban flood risk management inferred from Zhengzhou flood model[J]. China Flood and Drought Management, 2021, 31 (9): 1- 4
1 李莹, 赵珊珊 2001—2020年中国洪涝灾害损失与致灾危险性研究[J]. 气候变化研究进展, 2022, 18 (2): 154- 165
LI Ying, ZHAO Shanshan Floods losses and hazards in China from 2001 to 2020[J]. Climate Change Research, 2022, 18 (2): 154- 165
2 新华社. 河南郑州 “7·20” 特大暴雨灾害调查报告公布[EB/OL]. [2022-11-26]. https://www.gov.cn/xinwen/2022-01/21/content_5669723.htm, 2022.
3 刘家宏, 裴羽佳, 梅超, 等 郑州“7·20”特大暴雨内涝成因及灾害防控[J]. 郑州大学学报: 工学版, 2023, 44 (2): 38- 45
LIU Jiahong, PEI Yujia, MEI Chao, et al Waterlogging cause and disaster prevention and control of “7·20” torrential rain in Zhengzhou[J]. Journal of Zhengzhou University: Engineering Science, 2023, 44 (2): 38- 45
4 DE ALBUQUERQUE J P, HERFORT B, BRENNING A, et al A geographic approach for combining social media and authoritative data towards identifying useful information for disaster management[J]. International Journal of Geographical Information Science, 2015, 29 (4): 667- 689
doi: 10.1080/13658816.2014.996567
5 ROSSER J F, LEIBOVICI D G, JACKSON M J Rapid flood inundation mapping using social media, remote sensing and topographic data[J]. Natural Hazards, 2017, 87: 103- 120
doi: 10.1007/s11069-017-2755-0
6 SATHIANARAYANAN M, HSU P H, CHANG C C. Extracting disaster location identification from social media images using deep learning[J]. International Journal of Disaster Risk Reduction , 2024: 104352.
7 KIM J, HASTAK M Social network analysis: characteristics of online social networks after a disaster[J]. International Journal of Information Management, 2018, 38 (1): 86- 96
doi: 10.1016/j.ijinfomgt.2017.08.003
8 周利敏 大数据时代的社交媒体与自然灾害治理[J]. 人文杂志, 2021, (1): 102- 109
ZHOU Limin Social media and natural disaster management in the era of big data[J]. The Journal of Humanities, 2021, (1): 102- 109
9 金城, 吴文渊, 陈柏儒, 等 面向不同用户群体的社交媒体台风舆情演化分析及对比研究[J]. 地球信息科学学报, 2021, 23 (12): 2174- 2186
JIN Cheng, WU Wenyuan, CHEN Bairu, et al Analysis and comparative study of the evolution of public opinion on social media during typhoon for different user groups[J]. Journal of Geo-information Science, 2021, 23 (12): 2174- 2186
10 JING M, SCOTNEY B W, COLEMAN S A, et al. Integration of text and image analysis for flood event image recognition [C]// 27th Irish Signals and Systems Conference . Londonderry: IEEE, 2016: 1−6.
11 GUPTA A, LAMBA H, KUMARAGURU P, et al. Faking sandy: characterizing and identifying fake images on twitter during hurricane sandy [C]// Proceedings of the 22nd International Conference on World Wide Web . Rio de Janeiro: ACM, 2013: 729−736.
12 ALLCOTT H, GENTZKOW M, YU C, et al. Trends in the diffusion of misinformation on social media [J]. Research and Politics , 2019, 6(2): 2053168019848554.
13 TALWAR S, DHIR A, KAURP, et al Why do people share fake news? Associations between the dark side of social media use and fake news sharing behavior[J]. Journal of Retailing and Consumer Services, 2019, 1 (51): 72- 82
14 JIM Z, CAO J, GUO H, et al. Multimodal fusion with recurrent neural networks for rumor detection on microblogs [C]// Proceedings of the 25th ACM International Conference on Multimedia . New York: ACM, 2017: 795−816.
15 LIN F, CHANG W Y, LEE L C, et al. Applications of image recognition for real-time water level and surface velocity [C]// IEEE International Symposium on Multimedia . Anaheim: IEEE, 2013: 259−262.
16 许小华, 李亚琳, 吕姚, 等 基于两阶段深度学习的水位智能识别方法[J]. 中国防汛抗旱, 2023, 33 (10): 13- 20
XU Xiaohua, LI Yalin, LYU Yao, et al Intelligent recognition algorithm for water level measurement based on improved YOLOX-S algorithm[J]. China Flood and Drought Management, 2023, 33 (10): 13- 20
17 李小宁, 俞悦, 王船海, 等 南方平原河网区城市内涝的全过程监测与特征分析[J]. 水利学报, 2022, 53 (7): 845- 853
LI Xiaoning, YU Yue, WANG Chuanhai, et al Monitoring and characterization of the entire urban flooding process in the typical southern plain river network area[J]. Journal of Hydraulic Engineering, 2022, 53 (7): 845- 853
18 JAFARI NH, LI X, CHEN Q, et al Real-time water level monitoring using live cameras and computer vision techniques[J]. Computers and Geosciences, 2021, 147: 104642
doi: 10.1016/j.cageo.2020.104642
19 柳进元, 张明锋 基于YOLOv5算法的图像水深自动提取[J]. 福建师范大学学报: 自然科学版, 2023, 39 (1): 86- 92
LIU Jinyuan, ZHANG Mingfeng Automatic image water depth extraction based on YOLOv5 algorithm[J]. Journal of Fujian Normal University: Natural Science Edition, 2023, 39 (1): 86- 92
[1] 冉庆东,郑力新. 基于改进YOLOv5的锂电池极片缺陷检测方法[J]. 浙江大学学报(工学版), 2024, 58(9): 1811-1821.
[2] 陈健松,蔡艺军. 面向垃圾分类场景的轻量化目标检测方案[J]. 浙江大学学报(工学版), 2024, 58(1): 71-77.
[3] 金鑫,庄建军,徐子恒. 轻量化YOLOv5s网络车底危险物识别算法[J]. 浙江大学学报(工学版), 2023, 57(8): 1516-1526.
[4] 方浩杰,董红召,林少轩,罗建宇,方勇. 多特征融合的驾驶员疲劳状态检测方法[J]. 浙江大学学报(工学版), 2023, 57(7): 1287-1296.
[5] 马庆禄,鲁佳萍,唐小垚,段学锋. 改进YOLOv5s的公路隧道烟火检测方法[J]. 浙江大学学报(工学版), 2023, 57(4): 784-794.
[6] 曾耀,高法钦. 基于改进YOLOv5的电子元件表面缺陷检测算法[J]. 浙江大学学报(工学版), 2023, 57(3): 455-465.
[7] 鞠晓臣,赵欣欣,钱胜胜. 基于自注意力机制的桥梁螺栓检测算法[J]. 浙江大学学报(工学版), 2022, 56(5): 901-908.
[8] 袁天乐,袁巨龙,朱勇建,郑翰辰. 基于改进YOLOv5的推力球轴承表面缺陷检测算法[J]. 浙江大学学报(工学版), 2022, 56(12): 2349-2357.
[9] 张云佐,郭威,蔡昭权,李文博. 联合多尺度与注意力机制的遥感图像目标检测[J]. 浙江大学学报(工学版), 2022, 56(11): 2215-2223.
[10] 孙浩,唐坤林,金爱兵,刘美辰,陈帅军,李木芽. 单一粗颗粒对矿岩颗粒体系剪切特性影响试验[J]. 浙江大学学报(工学版), 2022, 56(10): 2037-2048.
[11] 董红召,方浩杰,张楠. 旋转框定位的多尺度再生物品目标检测算法[J]. 浙江大学学报(工学版), 2022, 56(1): 16-25.
[12] 郑英杰,吴松荣,韦若禹,涂振威,廖进,刘东. 基于目标图像FCM算法的地铁定位点匹配及误报排除方法[J]. 浙江大学学报(工学版), 2021, 55(3): 586-593.
[13] 陈功,郑春华,翁献明,HAMEEDBaustani,胡鸿昊,马晓宇,柳景青. 基于多水文数据关联分析的雨水口异常诊断[J]. 浙江大学学报(工学版), 2021, 55(1): 55-61.
[14] 黄毅鹏,胡冀苏,钱旭升,周志勇,赵文露,马麒,沈钧康,戴亚康. SE-Mask-RCNN:多参数MRI前列腺癌分割方法[J]. 浙江大学学报(工学版), 2021, 55(1): 203-212.
[15] 陈巧红,陈翊,李文书,贾宇波. 多尺度SE-Xception服装图像分类[J]. 浙江大学学报(工学版), 2020, 54(9): 1727-1735.