|
|
|
| Semantic segmentation of 3D point cloud based on boundary point estimation and sparse convolution neural network |
Jun YANG1,2( ),Chen ZHANG1 |
1. School of Electronic and Information Engineering, Lanzhou Jiaotong University, Lanzhou 730070, China
2. Faculty of Geomatics, Lanzhou Jiaotong University, Lanzhou 730070, China |
|
|
|
Abstract The large-scale point clouds are sparse, the traditional point cloud methods are insufficient in extracting rich contextual semantic features, and the semantic segmentation results have the problem of fuzzy object boundaries. A 3D point cloud semantic segmentation algorithm based on boundary point estimation and sparse convolution neural network was proposed, mainly including the voxel branch and the point branch. For the voxel branch, the original point cloud was voxelized, and then the contextual semantic features were obtained by sparse convolution. The initial semantic label of each point was obtained by voxelization. Finally, it was input into the boundary point estimation module to get the possible boundary points. For the point branch, the improved dynamic graph convolution module was first used to extract the local geometric features of the point cloud. Then, the local features were enhanced through the spatial attention module and the channel attention module in turn. Finally, the local geometric features obtained from the point branch and the contextual features obtained from the voxel branch were fused to enhance the richness of point cloud features. The semantic segmentation accuracy values of this algorithm on the S3DIS dataset and SemanticKITTI dataset were 69.5% and 62.7%, respectively. Experimental results show that the proposed algorithm can extract richer features of point clouds, accurately segment object boundary regions, and has good semantic segmentation ability for 3D point clouds.
|
|
Received: 20 May 2023
Published: 25 May 2024
|
|
|
| Fund: 国家自然科学基金资助项目（42261067）；兰州市人才创新创业资助项目（2020-RC-22）；甘肃省教育厅优秀研究生“创新之星”资助项目（2022CXZX-613）. |
基于边界点估计与稀疏卷积神经网络的三维点云语义分割
针对大规模点云具有稀疏性，传统点云方法提取上下文语义特征不够丰富，并且语义分割结果存在物体边界模糊的问题，提出基于边界点估计与稀疏卷积神经网络的三维点云语义分割算法，主要包括体素分支与点分支. 对于体素分支，将原始点云进行体素化后经过稀疏卷积得到上下文语义特征；进行解体素化得到每个点的初始语义标签；将初始语义标签输入到边界点估计模块中得到可能的边界点. 对于点分支，使用改进的动态图卷积模块提取点云局部几何特征；依次经过空间注意力模块与通道注意力模块增强局部特征；将点分支得到的局部几何特征与体素分支得到的上下文特征融合，增强点云特征的丰富性. 本算法在S3DIS数据集和SemanticKITTI数据集上的语义分割精度分别达到69.5%和62.7%. 实验结果表明，本研究算法能够提取到更丰富的点云特征，可以对物体的边界区域进行准确分割，具有较好的三维点云语义分割能力.
关键词：
点云数据,
语义分割,
注意力机制,
稀疏卷积,
体素化
|
|
| [1] |
SHI S, GUO C, JIANG L, et al. PV-RCNN: point-voxel feature set abstraction for 3D object detection [C]// Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition . Seattle: IEEE, 2020: 10526−10535.
|
|
|
| [2] |
CHABRA R, LENSSEN J, ILG E, et al. Deep local shapes: learning local SDF priors for detailed 3D reconstruction [C]// Proceedings of the European Conference on Computer Vision . Glasgow: Springer, 2020: 608−625.
|
|
|
| [3] |
HU W, ZHAO H, JIANG L, et al. Bidirectional projection network for cross dimension scene understanding [C]// Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition . [s. l.]: IEEE, 2021: 14373−14382.
|
|
|
| [4] |
DANG J S, YANG J LHPHGCNN: lightweight hierarchical parallel heterogeneous group convolutional neural networks for point cloud scene prediction[J]. IEEE Transactions on Intelligent Transportation Systems, 2022, 23 (10): 18903- 18915
doi: 10.1109/TITS.2022.3167910
|
|
|
| [5] |
QI C R, SU H, MO K, et al. PointNet: deep learning on point sets for 3D classification and segmentation [C]// Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition . Honolulu: IEEE, 2017: 77−85.
|
|
|
| [6] |
QI C R, YI L, SU H, et al. PointNet++: deep hierarchical feature learning on point sets in a metric space [C]// Advances in Neural Information Processing Systems. Long Beach: MIT Press, 2017: 5099−5108.
|
|
|
| [7] |
LAWIN F J, DANELLJAN M, TOSTEBERG P, et al. Deep projective 3D semantic segmentation [C]// International Conference on Computer Analysis of Images and Patterns . Ystad: Springer, 2017: 95−107.
|
|
|
| [8] |
BOULCH A, GUERRY J, SAUX B, et al SnapNet: 3D point cloud semantic labeling with 2D deep segmentation networks[J]. Computer and Graphics, 2018, 71: 189- 198
doi: 10.1016/j.cag.2017.11.010
|
|
|
| [9] |
GUERRY J, BOULCH A, LE S, et al. SnapNet-R: consistent 3D multi-view semantic labeling for robotics [C]// Proceedings of the IEEE International Conference on Computer Vision . Venice: IEEE, 2017: 669−678.
|
|
|
| [10] |
CORTINHAL T, TZELEPIS G, ERDAL E, et al. SalsaNext: fast, uncertainty-aware semantic segmentation of LiDAR point clouds [C]// International Symposium on Visual Computing . San Diego: Springer, 2020: 207−222.
|
|
|
| [11] |
ÇICEK O, ABDULKADIR A, LIENKAMP S S, et al. 3D U-Net: learning dense volumetric segmentation from sparse annotation [C]// Medical Image Computing and Computer-Assisted Intervention . Athens: Springer, 2016: 424−432.
|
|
|
| [12] |
WANG P S, LIU Y, GUO Y X, et al O-CNN: octree-based convolutional neural networks for 3D shape analysis[J]. ACM Transactions on Graphics, 2017, 36 (4): 1- 11
|
|
|
| [13] |
MENG H Y, GAO L, LAI Y K, et al. VV-Net: voxel VAE net with group convolutions for point cloud segmentation [C]// Proceedings of the IEEE/CVF International Conference on Computer Vision . Seoul: IEEE, 2019: 8499−8507.
|
|
|
| [14] |
LE T, DUAN Y. PointGrid: a deep network for 3D shape understanding [C]// Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition . Salt Lake City: IEEE, 2018: 9204−9214.
|
|
|
| [15] |
WANG Y, SUN Y, LIU Z, et al Dynamic graph CNN for learning on point clouds[J]. ACM Transactions on Graphics, 2018, 38 (5): 146- 158
|
|
|
| [16] |
KANG Z H, LI N PyramNet: point cloud pyramid attention network and graph embedding module for classification and segmentation[J]. Australian Journal of Intelligent Information Processing Systems, 2019, 16 (2): 35- 43
|
|
|
| [17] |
党吉圣, 杨军 多特征融合的三维模型识别与分割[J]. 西安电子科技大学学报, 2020, 47 (4): 149- 157
DANG Jisheng, YANG Jun 3D model recognition and segmentation based on multi-feature fusion[J]. Journal of Xidian University, 2020, 47 (4): 149- 157
|
|
|
| [18] |
HU Q Y, YANG B, XIE L H, et al. RandLA-Net: efficient semantic segmentation of large-scale point clouds [C]// Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition . Seattle: IEEE, 2020: 11105−11114.
|
|
|
| [19] |
LIU Z J, TANG H T, LIN Y J, et al. Point-voxel CNN for efficient 3D deep learning [C]// Advances in Neural Information Processing Systems . Vancouver: MIT Press, 2019: 963−973.
|
|
|
| [20] |
ZHANG F H, FANG J, WAH B, et al. Deep fusionnet for point cloud semantic segmentation [C]// Proceedings of the European Conference on Computer Vision . Glasgow: Springer, 2020: 644−663.
|
|
|
| [21] |
LIONG V E, NGUYEN T N T, Widjaja S, et al. AMVNet: assertion-based multi-view fusion network for LiDAR semantic segmentation [EB/OL]. (2020-12-09) [2023-02-12]. https://doi.org/10.48550/arXiv.2012.04934.
|
|
|
| [22] |
XU J Y, ZHANG R X, DOU J, et al. RPVNet: a deep and efficient range-point-voxel fusion network for LiDAR point cloud segmentation [C]// Proceedings of the IEEE/CVF International Conference on Computer Vision . Montreal: IEEE, 2021: 16004−16013.
|
|
|
| [23] |
RONNEBERGER O, FISCHER P, BROX T. U-Net: convolutional networks for biomedical image segmentation [C]// Medical Image Computing and Computer-Assisted Intervention . Munich: Springer, 2015: 234−241.
|
|
|
| [24] |
GRAHAM B, ENGELCKE M, MAATEN L. 3D semantic segmentation with submanifold sparse convolutional networks [C]// Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition . Salt Lake City: IEEE, 2018: 9224−9232.
|
|
|
| [25] |
杨军, 张琛. 融合双注意力机制和动态图卷积神经网络的三维点云语义分割 [EB/OL]. (2023-01-10) [2023-02-12]. https://bhxb.buaa.edu.cn/bhzk/article/doi/10.13700/j.bh.1001-5965.2022.0775.
|
|
|
| [26] |
ARMENI I, SENER O, ZAMIR A, et al. 3D semantic parsing of large-scale indoor spaces [C]// Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition . Las Vegas: IEEE, 2016: 1534−1543.
|
|
|
| [27] |
BEHLEY J, GARBADE M, MILIOTO A, et al. SemanticKITTI: a dataset for semantic scene understanding of LiDAR sequences [C]// Proceedings of the IEEE/CVF International Conference on Computer Vision . Seoul: IEEE, 2019: 9296−9306.
|
|
|
| [28] |
TATARCHENKO M, PARK J, KOLTUN V, et al. Tangent convolutions for dense prediction in 3D [C]// Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition . Salt Lake City: IEEE, 2018: 3887−3896.
|
|
|
| [29] |
LI Y, BU R, SUN M, et al. PointCNN: convolution on x-transformed points [C]// Advances in Neural Information Processing Systems . Montréal: MIT Press, 2018: 828−838.
|
|
|
| [30] |
LANDRIEU L, SIMONOVSKY M. Large-scale point cloud semantic segmentation with superpoint graphs [C]// Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition . Salt Lake City: IEEE, 2018: 4558−4567.
|
|
|
| [31] |
ZHAO H, JIANG L, FU C W, et al. PointWeb: enhancing local neighborhood features for point cloud processing [C]// Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition . Long Beach: IEEE, 2019: 5565−5573.
|
|
|
| [32] |
JIANG L, ZHAO H S, LIU S, et al. Hierarchical point-edge interaction network for point cloud semantic segmentation [C]// Proceedings of the IEEE/CVF International Conference on Computer Vision . Seoul: IEEE, 2019: 10432−10440.
|
|
|
| [33] |
WANG L, HUANG Y, HOU Y, et al. Graph attention convolution for point cloud semantic segmentation [C]// Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition . Long Beach: IEEE, 2019: 10296−10305.
|
|
|
| [34] |
AHN P, YANG J, YI E, et al Projection-based point convolution for efficient point cloud segmentation[J]. IEEE Access, 2022, 10: 15348- 15358
doi: 10.1109/ACCESS.2022.3144449
|
|
|
| [35] |
SHI Q, SAEED A, NICK B. Semantic segmentation for real point cloud scenes via bilateral augmentation and adaptive fusion [C]// Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition . [s. l.]: IEEE, 2021: 1757−1767.
|
|
|
| [36] |
THOMAS H, QI C R, DESCHAUD J E, et al. KPConv: flexible and deformable convolution for point clouds [C]// Proceedings of the IEEE/CVF International Conference on Computer Vision . Seoul: IEEE, 2019: 6410−6419.
|
|
|
| [37] |
WEI M, WEI Z, ZHOU H, et al AGConv: adaptive graph convolution on 3D point clouds[J]. IEEE Transactions on Pattern Analysis and Machine Intelligence, 2023, 45 (8): 9374- 9392
|
|
|
| [38] |
SHI H Y, LIN G S, WANG H, et al. SpSequenceNet: semantic segmentation network on 4D point clouds [C]// Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition . Seattle: IEEE, 2020: 4573–4582.
|
|
|
| [39] |
DANG J S, YANG J. HPGCNN: hierarchical parallel group convolutional neural networks for point clouds processing [C]// Proceedings of the Asian Conference on Computer Vision . Kyoto: Springer, 2020: 20−37.
|
|
|
| [40] |
MILIOTO A, VIZZO Ⅰ, BEHLEY J, et al. RangeNet++: fast and accurate LiDAR semantic segmentation [C]// IEEE/RSJ International Conference on Intelligent Robots and Systems . Macau: IEEE, 2019: 4213−4220.
|
|
|
| [41] |
ZHANG Y, ZHOU Z, DAIID P, et al. PolarNet: an improved grid representation for online LiDAR point clouds semantic segmentation [C]// Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition . Seattle: IEEE, 2020: 9598−9607.
|
|
|
| [42] |
ALONSO I, RIAZUELO L, MONTESANO L, et al 3D-MiniNet: learning a 2D representation from point clouds for fast and efficient 3D LiDAR semantic segmentation[J]. IEEE Robotics and Automation Letters, 2020, 5 (4): 5432- 5439
doi: 10.1109/LRA.2020.3007440
|
|
|
| [43] |
杨军, 李博赞 基于自注意力特征融合组卷积神经网络的三维点云语义分割[J]. 光学精密工程, 2022, 30 (7): 840- 853
YANG Jun, LI Bozan Semantic segmentation of 3D point cloud based on self-attention feature fusion group convolutional neural network[J]. Optics and Precision Engineering, 2022, 30 (7): 840- 853
doi: 10.37188/OPE.20223007.0840
|
|
|
|
Viewed |
|
|
|
Full text
|
|
|
|
|
Abstract
|
|
|
|
|
Cited |
|
|
|
|
| |
Shared |
|
|
|
|
| |
Discussed |
|
|
|
|
