|
|
Intelligent rebar inspection based on improved Mask R-CNN and stereo vision |
Cuiting WEI1( ),Weijian ZHAO1,2,Bochao SUN1,2,*( ),Yunyi LIU1 |
1. College of Civil Engineering and Architecture, Zhejiang University, Hangzhou 310058, China 2. Center for Balance Architecture, Zhejiang University, Hangzhou 310028, China |
|
|
Abstract A rebar inspection method based on improved mask region with convolutional neural network (Mask R-CNN) model and stereo vision technology was proposed in order to promote the transformation of reinforcement inspection to intelligence. The improved model Mask R-CNN with channel attention and spatial attention (Mask R-CNN+CA-SA) was formed by adding a bottom-up path with attention mechanism in Mask R-CNN. The diameter and spacing of rebar can be obtained by combining stereo vision technology for coordinate transformation, thereby achieving intelligent rebar inspection. The training was conducted on a self-built dataset containing 3450 rebar pictures. Results showed that the Mask R-CNN+CA-SA model increased the F1 score and mean average precision (mAP) by 2.54% and 2.47% compared with the basic network of Mask R-CNN, respectively. The rebar mesh verification test and complex background test showed that the absolute error and relative error of rebar diameter were basically controlled within 1.7 mm and 10%, and the absolute error and relative error of rebar spacing were controlled within 4 mm and 3.2% respectively. The proposed method is highly operable in practical applications. The intelligent rebar inspection technology can greatly improve work efficiency and reduce labor costs while ensuring sufficient inspection accuracy.
|
Received: 09 August 2023
Published: 26 April 2024
|
|
Fund: 国家自然科学基金资助项目(52108254, 52208215). |
Corresponding Authors:
Bochao SUN
E-mail: 22112262@zju.edu.cn;sunbochao@zju.edu.cn
|
基于改进Mask R-CNN与双目视觉的智能配筋检测
为了提高配筋检测的智能化水平,提出基于改进掩膜区域卷积神经网络(Mask R-CNN)模型与双目视觉技术的配筋检测方法. 通过在Mask R-CNN中加入自下而上的注意力机制路径,形成了带通道注意力和空间注意力的掩膜区域卷积神经网络(Mask R-CNN+CA-SA)改进模型. 结合双目视觉技术进行坐标转换,获取钢筋直径与间距,实现智能配筋检测. 在自建的包含3 450 张钢筋图片的数据集上进行训练,结果表明,改进模型的F1分数和全类平均精确率(mAP)相较于Mask R-CNN基础网络分别提高了2.54%和2.47%. 通过钢筋网验证试验和复杂背景测试,钢筋直径的绝对误差和相对误差基本小于1.7 mm和10%,钢筋间距的绝对误差和相对误差分别小于4 mm和3.2%,所提方法在实际应用中具有较强的可操作性. 智能配筋检测技术在保证足够的检测精度的同时,能够大大提升工效,降低人工成本.
关键词:
配筋质量检测,
Mask R-CNN,
注意力机制,
深度学习,
双目视觉技术
|
|
[1] |
孙昊. 基于数字图像的钢筋工程快速检查方法研究[D]. 镇江: 江苏大学, 2019. SUN Hao. Research on fast inspection method of reinforcement based on digital image [D]. Zhenjiang: Jiangsu University, 2019.
|
|
|
[2] |
KIM M K, THEDJA J P P, WANG Q Automated dimensional quality assessment for formwork and rebar of reinforced concrete components using 3D point cloud data[J]. Automation in Construction, 2020, 112: 103077
doi: 10.1016/j.autcon.2020.103077
|
|
|
[3] |
CHI H L, KIM M K, LIU K Z, et al Rebar inspection integrating augmented reality and laser scanning[J]. Automation in Construction, 2022, 136: 104183
doi: 10.1016/j.autcon.2022.104183
|
|
|
[4] |
YANG X, LI X, YE Y, et al Road detection and centerline extraction via deep recurrent convolutional neural network U-Net[J]. IEEE Transactions on Geoscience and Remote Sensing, 2019, 57 (9): 7209- 7220
doi: 10.1109/TGRS.2019.2912301
|
|
|
[5] |
PAN Y, ZHANG L Dual attention deep learning network for automatic steel surface defect segmentation[J]. Computer ‐Aided Civil and Infrastructure Engineering, 2022, 37 (11): 1468- 1487
doi: 10.1111/mice.12792
|
|
|
[6] |
LI S, ZHAO X A performance improvement strategy for concrete damage detection using stacking ensemble learning of multiple semantic segmentation networks[J]. Sensors, 2022, 22 (9): 3341
doi: 10.3390/s22093341
|
|
|
[7] |
姚刚, 廖港, 杨阳, 等. 基于卷积神经网络的预制叠合板多目标智能化检测方法[J/OL]. 土木与环境工程学报(中英文) , 2024, 46(1): 93-101 [2023-06-30]. http://kns.cnki.net/kcms/detail/50.1218.TU.20220423.2103.002.html. YAO Gang, LIAO Gang, YANG Yang, et al. Multi-target intelligent detection method of prefabricated laminated board based on convolutional neural network [J/OL]. Journal of Civil and Environmental Engineering , 2024, 46(1): 93-101[2023-06-30]. http://kns.cnki.net/kcms/detail/50.1218.TU.20220423.2103.002.html.
|
|
|
[8] |
董国梁, 张雷, 辛山 基于深度学习的钢筋绑扎机器人目标识别定位[J]. 电子测量技术, 2022, 45 (11): 35- 44 DONG Guoliang, ZHANG Lei, XIN Shan Target recognition and location of steel bar binding robot based on deep learning[J]. Electronic Measurement Technology, 2022, 45 (11): 35- 44
|
|
|
[9] |
JIN J, ZHANG W, LI F, et al Robotic binding of rebar based on active perception and planning[J]. Automation in Construction, 2021, 132: 103939
doi: 10.1016/j.autcon.2021.103939
|
|
|
[10] |
LI Y, LU Y, CHEN J A deep learning approach for real-time rebar counting on the construction site based on YOLOv3 detector[J]. Automation in Construction, 2021, 124: 103602
doi: 10.1016/j.autcon.2021.103602
|
|
|
[11] |
SHIN Y, HEO S, HAN S, et al An image-based steel rebar size estimation and counting method using a convolutional neural network combined with homography[J]. Buildings, 2021, 11 (10): 463
doi: 10.3390/buildings11100463
|
|
|
[12] |
REN S, HE K, GIRSHICK R, et al Faster R-CNN: towards real-time object detection with region proposal networks[J]. IEEE Transactions on Pattern Analysis and Machine Intelligence, 2017, 39 (6): 1137- 1149
doi: 10.1109/TPAMI.2016.2577031
|
|
|
[13] |
HE K, GEORGIA G, PIOTR D, et al Mask R-CNN[J]. IEEE Transactions on Pattern Analysis and Machine Intelligence, 2020, 42 (2): 386- 397
doi: 10.1109/TPAMI.2018.2844175
|
|
|
[14] |
ZHAO K, KANG J, JUNG J, et al. Building extraction from satellite images using Mask R-CNN with building boundary regularization [C]// Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition . Salt Lake City: IEEE, 2018: 247-251.
|
|
|
[15] |
KIM B, CHO S Image-based concrete crack assessment using mask and region-based convolutional neural network[J]. Structural Control and Health Monitoring, 2019, 26 (8): e2381
|
|
|
[16] |
XU Y, LI D, XIE Q, et al Automatic defect detection and segmentation of tunnel surface using modified Mask R-CNN[J]. Measurement, 2021, 178: 109316
doi: 10.1016/j.measurement.2021.109316
|
|
|
[17] |
NIE S, JIANG Z, ZHANG H, et al. Inshore ship detection based on mask R-CNN [C]// IEEE International Geoscience and Remote Sensing Symposium . Valencia: IEEE, 2018: 693-696.
|
|
|
[18] |
LIU S, QI L, QIN H, et al. Path aggregation network for instance segmentation [C]// Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition . Salt Lake City: IEEE, 2018: 8759-8768.
|
|
|
[19] |
HUANG Z, ZHONG Z, SUN L, et al. Mask R-CNN with pyramid attention network for scene text detection [C]// IEEE Winter Conference on Applications of Computer Vision . Waikoloa: IEEE, 2019: 764-772.
|
|
|
[20] |
LI Z, ZHOU F. FSSD: Feature fusion single shot multibox detector [EB/OL]. [2023-06-30]. https:// arxiv.org/abs/1712.00960.
|
|
|
[21] |
VASWANI A, SHAZEER N, PARMAR N, et al Attention is all you need[J]. Advances in Neural Information Processing Systems, 2017, 30: 5999- 6009
|
|
|
[22] |
WOO S, PARK J, LEE J Y, et al. CBAM: convolutional block attention module [C]// Proceedings of the European Conference on Computer Vision . Switzerland: Springer, 2018: 3-19.
|
|
|
[23] |
YANG J, REN P, ZHANG D, et al. Neural aggregation network for video face recognition [C]// Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition . Honolulu: IEEE, 2017: 5216-5225.
|
|
|
[24] |
WANG Q, WU T, ZHENG H, et al. Hierarchical pyramid diverse attention networks for face recognition [C]// Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition . Piscataway: IEEE, 2020: 8326-8335.
|
|
|
[25] |
CHEN Y, DAI X, LIU M, et al. Dynamic convolution: attention over convolution kernels [C]// Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition . Seattle: IEEE, 2020: 11030-11039.
|
|
|
[26] |
XIE S, LIU S, CHEN Z, et al. Attentional ShapeContextNet for point cloud recognition [C]// Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition . Salt Lake City: IEEE, 2018: 4606-4615.
|
|
|
[27] |
ZAARANE A, SLIMANI I, AL OKAISHI W, et al Distance measurement system for autonomous vehicles using stereo camera[J]. Array, 2020, 5: 100016
doi: 10.1016/j.array.2020.100016
|
|
|
[28] |
PEPE M, FREGONESE L, SCAIONI M Planning airborne photogrammetry and remote-sensing missions with modern platforms and sensors[J]. European Journal of Remote Sensing, 2018, 51 (1): 412- 436
doi: 10.1080/22797254.2018.1444945
|
|
|
[29] |
TUOMINEN S, NÄSI R, HONKAVAARA E, et al Assessment of classifiers and remote sensing features of hyperspectral imagery and stereo-photogrammetric point clouds for recognition of tree species in a forest area of high species diversity[J]. Remote Sensing, 2018, 10 (5): 714
doi: 10.3390/rs10050714
|
|
|
[30] |
LI D, XU L, TANG X, et al 3D imaging of greenhouse plants with an inexpensive binocular stereo vision system[J]. Remote Sensing, 2017, 9 (5): 508
doi: 10.3390/rs9050508
|
|
|
[31] |
ZHOU Y, GALLEGO G, REBECQ H, et al. Semi-dense 3D reconstruction with a stereo event camera [C]// Proceedings of the European Conference on Computer Vision . Munich: Springer, 2018: 235-251.
|
|
|
|
Viewed |
|
|
|
Full text
|
|
|
|
|
Abstract
|
|
|
|
|
Cited |
|
|
|
|
|
Shared |
|
|
|
|
|
Discussed |
|
|
|
|