Dynamic 3D reconstruction method using binocular vision and improved YOLOv8

doi:10.3785/j.issn.1008-973X.2025.07.012

Journal of ZheJiang University (Engineering Science)

2025, Vol. 59

Issue (7): 1443-1450 DOI: 10.3785/j.issn.1008-973X.2025.07.012

Dynamic 3D reconstruction method using binocular vision and improved YOLOv8

Jingyao HE1(

),Pengfei LI1,*(

),Chengzhi WANG1,Zhenming LV2,Ping MU1

1. College of River and Ocean Engineering, Chongqing Jiaotong University, Chongqing 400074, China
2. School of Mechatronics and Vehicle Engineering, Chongqing Jiaotong University, Chongqing 400074, China

Download:

HTML

PDF(7266KB) HTML
Export: BibTeX | EndNote (RIS)

Abstract

A dynamic 3D reconstruction technology for construction sites was proposed to ensure safety and efficiency in the construction process. A Binocular camera was deployed to scan the reconstruction site in 3D to obtain the model base and target activity trajectory. The YOLOv8 model was enhanced with an attentional scale sequence fusion (ASF) module to form the YOLOv8-ASF framework, which improved the accuracy and performance of the model, to solve the pain points such as target occlusion and target loss. The improved semi-global block matching (SGBM) algorithm was fused, and the YOLOv8-ASF-SGBM algorithm was integrated with the YOLOv8-ASF to achieve near-real-time target recognition and localization based on 2D images. The obtained depth information was used to 3D project the behavior trajectories of dynamic elements into the substrate, to realize the near-real-time and full-view monitoring of the real construction site. Experimental results show that the proposed technology reproduces the movement trajectory of construction dynamic elements in high-precision three-dimensional, and the relative error with the real motion trajectory of dynamic elements is less than 5%, which can realize high-precision full-view three-dimensional monitoring based on two-dimensional image and video information, and has good application scenarios and engineering value.

Key words： dynamic 3D reconstruction YOLOv8-attentional scale sequence fusion (ASF) semi-global block matching (SGBM) algorithm target occlusion binocular vision

Received: 31 May 2024 Published: 25 July 2025

CLC:

TU 17

Fund: 国家自然科学基金资助项目（52379115）；重庆市自然科学基金资助项目（CSTB2022NSCQ-MSX0509）.

Corresponding Authors: Pengfei LI E-mail: 622220960055@mails.cqjtu.edu.cn;lipengfei@cqjtu.edu.cn

	Service
	E-mail this article
	Add to my bookshelf
	Add to citation manager
	E-mail Alert
	RSS
	Articles by authors
	Jingyao HE
	Pengfei LI
	Chengzhi WANG
	Zhenming LV
	Ping MU

Cite this article:

Jingyao HE,Pengfei LI,Chengzhi WANG,Zhenming LV,Ping MU. Dynamic 3D reconstruction method using binocular vision and improved YOLOv8. Journal of ZheJiang University (Engineering Science), 2025, 59(7): 1443-1450.

URL:

https://www.zjujournals.com/eng/10.3785/j.issn.1008-973X.2025.07.012 OR https://www.zjujournals.com/eng/Y2025/V59/I7/1443

基于双目视觉和改进YOLOv8的动态三维重建方法

为了确保施工过程中的安全和效率，提出施工现场动态三维重建技术. 部署双目摄像头对重建现场进行三维扫描获取模型基底和目标活动轨迹，基于YOLOv8模型引入注意力量表序列融合(ASF)模块形成YOLOv8-ASF框架，提高预测模型精度和性能，解决如目标遮挡、目标丢失的痛点. 融合改进的半全局立体匹配 (SGBM)算法，与YOLOv8-ASF集成YOLOv8-ASF-SGBM算法，实现基于二维图像的目标近实时识别和定位. 利用获取的深度信息，将动态要素行为轨迹三维动态投影至模型基底中，实现对真实施工现场的近实时、全视角监控. 实验结果表明：所提技术高精度三维复现了施工动态要素的运动轨迹，且与动态要素真实运动轨迹的相对误差小于5%，实现了基于二维图像视频信息的高精度全视角三维立体化监控，具有良好的应用场景和工程价值.

关键词： 动态三维重建, YOLOv8-注意力量表序列融合(ASF), 半全局立体匹配(SGBM)算法, 目标遮挡, 双目视觉

Fig.1 Technical framework for dynamic 3D reconstruction

Fig.2 Simulation scenario for layout of dynamic and static elements at construction site

Fig.3 Principle of binocular camera calibration

Fig.4 Reconstruction results under simulation scenario for dynamic and static element layout at construction site

Tab.1 Ratio error between true sizes of dynamic and static elements and point-cloud sizes

Fig.5 Framework of YOLOv8-attentional scale sequence fusion model

Fig.6 Visualization of model target recognition results before and after attenetional scale sequence fusion to improve YOLOv8

Fig.7 Comparison of object detection performance of different YOLOv8 models

Fig.8 Binocular ranging principle

Fig.9 Detection and localization of excavator by improved model in simulation scenarios

Fig.10 Feature point matching accuracy of different algorithms in four simulation scenarios

Fig.11 Dynamic reconstruction of excavator behavior trajectory in simulation scenario

Fig.12 Accuracy of dynamic reconstruction of excavator behavior trajectory in simulation scenario


[1]	CAO J B. Research on construction quality management of construction project [C]// Proceedings of the 2018 8th International Conference on Management, Education and Information (MEICI 2018). [S.l.]: Atlantis Press, 2018: 1275–1278.

[2]	DÍAZ CASELLES L M, GUEVARA J Sustainability performance in on-site construction processes: a systematic literature review[J]. Sustainability, 2024, 16 (3): 1047 doi: 10.3390/su16031047

[3]	CARVAJAL-ARANGO D, BAHAMÓN-JARAMILLO S, ARISTIZÁBAL-MONSALVE P, et al Relationships between lean and sustainable construction: positive impacts of lean practices over sustainability during construction phase[J]. Journal of Cleaner Production, 2019, 234: 1322- 1337 doi: 10.1016/j.jclepro.2019.05.216

[4]	LIU T, WANG N, FU Q, et al. Research on 3D reconstruction method based on laser rotation scanning [C]// Proceedings of the IEEE International Conference on Mechatronics and Automation. Tianjin: IEEE, 2019: 1600–1604.

[5]	YU Q, HELMHOLZ P, BELTON D Semantically enhanced 3D building model reconstruction from terrestrial laser-scanning data[J]. Journal of Surveying Engineering, 2017, 143 (4): 04017015 doi: 10.1061/(ASCE)SU.1943-5428.0000232

[6]	ABDULWAHAB S, RASHWAN H A, GARCIA M A, et al Monocular depth map estimation based on a multi-scale deep architecture and curvilinear saliency feature boosting[J]. Neural Computing and Applications, 2022, 34 (19): 16423- 16440 doi: 10.1007/s00521-022-07663-x

[7]	JIN S K, OU Y S. Feature-based monocular dynamic 3D object reconstruction [C]// Social Robotics. [S.l.]: Springer, 2018: 380–389.

[8]	ZHANG Y, GU J, RAO T, et al A shape reconstruction and measurement method for spherical hedges using binocular vision[J]. Frontiers in Plant Science, 2022, 13: 849821 doi: 10.3389/fpls.2022.849821

[9]	GAI Q Optimization of stereo matching in 3D reconstruction based on binocular vision[J]. Journal of Physics: Conference Series, 2018, 960 (1): 012029

[10]	XIANG R, JIANG H, YING Y Recognition of clustered tomatoes based on binocular stereo vision[J]. Computers and Electronics in Agriculture, 2014, 106: 75- 90 doi: 10.1016/j.compag.2014.05.006

[11]	KANG M, TING C M, TING F F, et al ASF-YOLO: a novel YOLO model with attentional scale sequence fusion for cell instance segmentation[J]. Image and Vision Computing, 2024, 147: 105057 doi: 10.1016/j.imavis.2024.105057

[12]	DUAN R, DENG H, TIAN M, et al SODA: a large-scale open site object detection dataset for deep learning in construction[J]. Automation in Construction, 2022, 142: 104499 doi: 10.1016/j.autcon.2022.104499

[13]	AN X, ZHOU L, LIU Z, et al Dataset and benchmark for detecting moving objects in construction sites[J]. Automation in Construction, 2021, 122: 103482 doi: 10.1016/j.autcon.2020.103482

[14]	HUANG H, BRENNER C, SESTER M A generative statistical approach to automatic 3D building roof reconstruction from laser scanning data[J]. ISPRS Journal of Photogrammetry and Remote Sensing, 2013, 79: 29- 43 doi: 10.1016/j.isprsjprs.2013.02.004

[15]	ZHANG Y, HAO Y A survey of SAR image target detection based on convolutional neural networks[J]. Remote Sensing, 2022, 14 (24): 6240 doi: 10.3390/rs14246240

[16]	TEIXEIRA E, ARAUJO B, COSTA V, et al Literature review on ship localization, classification, and detection methods based on optical sensors and neural networks[J]. Sensors, 2022, 22 (18): 6879 doi: 10.3390/s22186879

[17]	白博建筑防火技术在民用建筑设计中的应用研究[J]. 消防界(电子版), 2025, 11 (2): 64- 66 BAI Bo Research on the application of building fire protection technology in civil building design[J]. Fire Protection Circle (Electronic Edition), 2025, 11 (2): 64- 66

[18]	林绿开, 钮倩倩, 李毅基于棋盘标定板的优化相机参数标定方法[J]. 计算机技术与发展, 2023, 33 (12): 101- 105 LIN Lyukai, NIU Qianqian, LI Yi Optimized camera parameter calibration method based on checkerboard[J]. Computer Technology and Development, 2023, 33 (12): 101- 105 doi: 10.3969/j.issn.1673-629X.2023.12.014

[19]	冯晓硕, 沈樾, 王冬琦基于图像的数据增强方法发展现状综述[J]. 计算机科学与应用, 2021, (2): 370- 382 FENG Xiaoshuo, SHEN Yue, WANG Dongqi A survey on the development of image data augmentation[J]. Computer Science and Application, 2021, (2): 370- 382

[20]	LIANG H, LIU M, HUI M, et al. 3D reconstruction of typical entities based on multi-perspective images [C]// Proceedings of the Optical Metrology and Inspection for Industrial Applications IX. [S.l.]: SPIE, 2022: 56.

[21]	GUPTA S K, SHUKLA D P Application of drone for landslide mapping, dimension estimation and its 3D reconstruction[J]. Journal of the Indian Society of Remote Sensing, 2018, 46 (6): 903- 914 doi: 10.1007/s12524-017-0727-1

[22]	ZHAI X, HUANG Z, LI T, et al YOLO-drone: an optimized YOLOv8 network for tiny UAV object detection[J]. Electronics, 2023, 12 (17): 3664

[23]	凌雄娟, 周云郊, 彭建喜基于双目立体视觉的乘员运动姿态测量方法研究[J]. 机械设计与制造工程, 2024, 53 (1): 126- 130 LING Xiongjuan, ZHOU Yunjiao, PENG Jianxi Research on test method of occupant motion attitude measurement based on binocular stereo vision[J]. Machine Design and Manufacturing Engineering, 2024, 53 (1): 126- 130 doi: 10.3969/j.issn.2095-509X.2024.01.026

[24]	XIAO H, TENG X, LIU C, et al A review of deep learning-based three-dimensional medical image registration methods[J]. Quantitative Imaging in Medicine and Surgery, 2021, 11 (12): 4895- 4916 doi: 10.21037/qims-21-175

[25]	刘志勇, 王淑贤基于鼠标事件的虚拟仿真实验中学习状态评价模型研究[J]. 软件工程, 2022, 25 (10): 37- 40 LIU Zhiyong, WANG Shuxian Research on learning state evaluation model in virtual simulation experiment based on mouse events[J]. Software Engineering, 2022, 25 (10): 37- 40

[1]	Hao-ran MA,Ya-bin DING. Calibration method of laser displacement sensor based on binocular vision[J]. Journal of ZheJiang University (Engineering Science), 2021, 55(9): 1634-1642.

[2]	KE Xian-xin, ZHANG Wen-zhen, YANG Yang, WEN Lei. Multi-sensor positioning system for humanoid robot[J]. Journal of ZheJiang University (Engineering Science), 2018, 52(7): 1247-1252.

Viewed

Full text

Abstract

Cited

Shared

Discussed