Please wait a minute...
浙江大学学报(工学版)
浙江大学学报(工学版)  2025, Vol. 59 Issue (5): 879-889    DOI: 10.3785/j.issn.1008-973X.2025.05.001
计算机技术、信息工程     
面向点云理解的双邻域图卷积方法
李宗民1,2(),徐畅1,白云1,鲜世洋1,戎光彩1
1. 中国石油大学（华东） 青岛软件学院 计算机科学与技术学院，山东 青岛 266580
2. 青岛滨海学院 信息工程学院，山东 青岛 266580
Dual-neighborhood graph convolution method for point cloud understanding
Zongmin LI1,2(),Chang XU1,Yun BAI1,Shiyang XIAN1,Guangcai RONG1
1. College of Computer Science and Technology, Qingdao Institute of Software, China University of Petroleum (East China), Qingdao 266580, China
2. Information Engineering College, Qingdao Binhai University, Qingdao 266580, China
 全文: PDF(2494 KB)   HTML
摘要：

针对现有方法对局部点云结构建模时空间跨度有限以及传统特征聚合方法造成一定信息损失的问题，提出双邻域图卷积网络(DNGCN). 在原始点云中增加角度先验，以增强对点云局部几何结构的理解，捕捉局部细节. 对原始邻域进行扩展，在局域内设计双邻域图卷积，通过集成高斯自适应聚合，在提取较大感受野范围内显著特征的同时，充分保留原始邻域信息. 通过局部-全局信息交互来增大局部点的空间跨度，捕获远距离依赖关系. 本文方法在分类数据集ModelNet40和ScanObjectNN上分别取得了94.1%、89.6%的总体精度，与其他先进算法相比有显著提升，较DGCNN分别提升了1.2%、11.5%. 在部件分割数据集ShapeNetPart和语义分割数据集ScanNetv2、S3DIS上均获得优秀的性能，平均交并比分别为86.7%、74.9%和69.8%. 通过大量的实验，证明了该模型的有效性.
关键词： 点云特征;  图卷积网络;  几何增强;  局部全局交互;  注意力机制    
Abstract:

A dual-neighborhood graph convolutional network (DNGCN) was proposed in order to address the limitations of existing methods in modeling local point cloud structures with restricted spatial spans and the information loss caused by conventional feature aggregation strategies. Angular priors were incorporated into raw point coordinates in order to enhance geometric awareness for capturing fine-grained local structures. A dual-neighborhood graph convolution operator that integrated Gaussian adaptive aggregation was designed by extending the original neighborhood, enabling simultaneous extraction of salient features from enlarged receptive fields and preservation of intricate local details. A local-global cross-scale interaction mechanism was introduced to expand spatial perception spans and model long-range dependencies. The proposed method achieved an overall classification accuracy of 94.1% on ModelNet40 and 89.6% on ScanObjectNN, significantly outperforming other advanced algorithms. The increases were 1.2% and 11.5% respectively compared with DGCNN. Excellent performance was obtained on the ShapeNetPart dataset for part segmentation, as well as the ScanNetv2 and S3DIS datasets for semantic segmentation, with mean IoU scores of 86.7%, 74.9% and 69.8%, respectively. Experiments proved the effectiveness of the model.
Key words: point cloud feature    graph convolution network    geometric enhancement    local-global interaction    attention mechanism
收稿日期: 2024-07-03 出版日期: 2025-04-25
CLC:  TP 391  
基金资助: 国家重点研发计划资助项目(2019YFF0301800); 国家自然科学基金资助项目(61379106); 山东省自然科学基金资助项目(ZR2013FM036, ZR2015FM011).
作者简介: 李宗民（1965—），男，教授，博导，从事计算机图形学、模式识别的研究. orcid.org/0000-0003-4785-791X.E-mail：lizongmin@upc.edu.cn
服务　　
把本文推荐给朋友
加入引用管理器
E-mail Alert
作者相关文章　　
李宗民
徐畅
白云
鲜世洋
戎光彩

引用本文:

李宗民,徐畅,白云,鲜世洋,戎光彩. 面向点云理解的双邻域图卷积方法[J]. 浙江大学学报(工学版), 2025, 59(5): 879-889.

Zongmin LI,Chang XU,Yun BAI,Shiyang XIAN,Guangcai RONG. Dual-neighborhood graph convolution method for point cloud understanding. Journal of ZheJiang University (Engineering Science), 2025, 59(5): 879-889.

链接本文:

https://www.zjujournals.com/eng/CN/10.3785/j.issn.1008-973X.2025.05.001    或    https://www.zjujournals.com/eng/CN/Y2025/V59/I5/879

图 1  局域内角度表示的示例图
图 2  双邻域图卷积模块
图 3  DNG模块
图 4  DNG-Net整体网络框架
方法年份mAcc/%OA/%
PointNet[7]201786.289.2
PointNet++[8]2017—91.9
KPConv[18]2019—92.9
DGCNN[13]201990.292.9
PointASNL[11]2020—92.9
PointTransformer[31]202090.693.7
PointMixer[40]202191.493.6
CurveNet[41]2021—93.8
PointNeXT[42]202291.194.0
DGCNN+HyCoRe[43]202291.093.7
PointConT[44]2023—93.5
MKConv[45]2023—93.7
DNG-Net(本文方法)202491.394.1
表 1  在ModelNet40基准上测试的分类结果
方法年份mAcc/%OA/%
PointNet[7]201763.468.2
PointNet++[8]201775.477.9
DGCNN[13]201973.678.1
GBNet[20]202177.880.5
PointMLP[46]202284.485.7
RepSurf-U[21]202283.186.0
PointMLP+HyCoRe[43]202285.987.2
PointNeXT[42]202286.888.2
PointConT[44]202386.088.0
SPoTr[32]202386.888.6
DNG-Net(本文方法)202488.389.6
表 2  在ScanObjectNN基准上测试的分类结果
方法年份mIoUcls/%mIoUins/%
PointNet[7]201780.483.7
PointNet++[8]201781.985.1
KPConv[18]201985.086.2
DGCNN[13]201982.385.2
PointASNL[11]2020—86.1
PAConv[19]202184.686.1
CurveNet[41]2021—86.6
PointTransformer[31]202083.786.6
StratifiedFormer[16]202285.186.6
PointMLP[46]202284.686.1
Point2vec[47]202384.686.3
MKConv[45]2023—86.5
DNG-Net(本文方法)202484.786.7
表 3  在ShapeNetPart基准上测试的分割结果
图 5  ShapeNetPart数据集部件分割结果的可视化
方法年份P/106mIOUval/%mIOUtest/%
PointNet++[8]2017—53.555.7
KPConv[18]2019—69.268.6
PointTransformer[31]20217.870.6—
StratifiedFormer[16]202218.874.373.7
PTv3[34]202346.277.577.9
DNG-Net(本文方法)20249.275.674.9
表 4  在ScanNetV2基准上测试的分割结果
方法年份mAcc/%OA/%mIOU/%
KPConv[18]201972.8—67.1
MKConv[45]202375.189.667.7
RepSurf-U[21]202276.090.268.9
SPoTr[32]202376.490.770.8
DNG-Net(本文方法)202476.491.069.8
表 5  在S3DIS Area5基准上测试的分割结果
图 6  S3DIS数据集语义分割结果的可视化
模型LGEDNConvLGIOA/%
DNGCNGAA
A————92.8
B—√√√93.7
C√√√—93.8
D√——√93.5
E√—√√93.7
F√√√√94.1
表 6  在ModelNet40上不同组件的消融研究
模型局部几何增强LGE维度OA/%
A${{\boldsymbol{x}}_j} - {{\boldsymbol{x}}_i}$393.1
B${{\boldsymbol{x}}_i},{{\boldsymbol{x}}_j} - {{\boldsymbol{x}}_i}$693.3
C${{{{\boldsymbol{x}}}}_i},{{\boldsymbol{x}}_j} - {{\boldsymbol{x}}_i},{\delta _{{\boldsymbol{a}} }} - {\delta _{{\boldsymbol{b}} }},{\varphi _{{\boldsymbol{ a }}}} - {\varphi _{{\boldsymbol{b}} }}$894.1
D${{\boldsymbol{x}}_i},{{\boldsymbol{x}}_j} - {{\boldsymbol{x}}_i},{\delta _{{\boldsymbol{a}} }} - {\delta _{ {\boldsymbol{b}} }},{\varphi _{ {\boldsymbol{a}} }} - {\varphi _{{\boldsymbol{b}} }},l$993.5
E${{\boldsymbol{x}}_i},{{\boldsymbol{x}}_j} - {{\boldsymbol{x}}_i},{\delta _{{\boldsymbol{a}} }},{\delta _{{\boldsymbol{b}} }},{\varphi _{ {\boldsymbol{a}} }},{\varphi _{ {\boldsymbol{b}} }}$1093.7
F${{\boldsymbol{x}}_i},{{\boldsymbol{x}}_j},{{\boldsymbol{x}}_j} - {{\boldsymbol{x}}_i},{\delta _{ {\boldsymbol{a}} }} - {\delta _{ {\boldsymbol{b}} }},{\varphi _{ {\boldsymbol{a}} }} - {\varphi _{ {\boldsymbol{b}} }},l$1292.6
GSPT手工标注特征493.4
表 7  局部几何增强模块形式的研究
模型KOA/%
$N_{{S}_1} $$N_{{S}_2} $
A101293.4
A202093.7
A303093.3
B151293.2
B252093.5
B353093.0
C1101293.6
C2102093.8
C3103093.5
D11512—
D2152094.1
D3153093.6
表 8  双邻域不同K值的对比研究
模型特征融合方式OA/%
A拼接93.3
B求和93.5
C自适应融合94.1
表 9  特征融合方式的OA对比
m对应点数FLOPs/109OA/%
1, 2, 21024, 512, 2562.0793.3
1, 4, 41024, 256, 641.9294.1
1, 8, 81024, 128, 161.8693.7
表 10  关键点提取采样率m的效率研究
模型FLOPs/109P/106OA/%
PointNet[7]0.53.589.2
PointNet++[8]4.11.891.9
DGCNN[13]3.02.692.9
PointTransformer[31]18.49.693.7
CurveNet[41]0.32.193.8
PointMixer[40]—6.593.6
PointNeXT[42]6.54.594.0
PointMLP[46]15.713.294.1
DNG-Net(本文方法)1.924.994.1
表 11  在ModelNet40上的复杂度分析
1 JABOVEDOFF M, OPPIKOFER T, ABELLÁN A, et al Use of LIDAR in landslide investigations: a review[J]. Natural Hazards, 2012, 61 (1): 5- 28
doi: 10.1007/s11069-010-9634-2
2 RUSU R B, MARTON Z C, BLODOW N, et al Towards 3D point cloud based object maps for household environments[J]. Robotics and Autonomous Systems, 2008, 56 (11): 927- 941
doi: 10.1016/j.robot.2008.08.005
3 QI C R, LIU W, WU C, et al. Frustum pointnets for 3d object detection from rgb-d data [C]// Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition . Salt Lake City: IEEE, 2018: 918-927.
4 FAN H, SU H, GUIBAS L J. A point set generation network for 3d object reconstruction from a single image [C]// Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition . Honolulu: IEEE, 2017: 605-613.
5 SU H, MAJI S, KALOGERAKIS E, et al. Multi-view convolutional neural networks for 3d shape recognition [C]// Proceedings of the IEEE International Conference on Computer Vision . Santiago: IEEE, 2015: 945-953.
6 LI Y, PIRK S, SU H, et al Fpnn: field probing neural networks for 3d data[J]. Advances in Neural Information Processing Systems, 2016, 29: 307- 315
7 QI C R, SU H, MO K, et al. Pointnet: deep learning on point sets for 3d classification and segmentation [C]// Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition . Honolulu: IEEE, 2017: 652-660.
8 QI C R, YI L, SU H, et al. PointNet++: deep hierarchical feature learning on point sets in a metric space [C] // Proceedings of the 30th International Conference on Neural Information Processing Systems. Cambridge: MIT Press, 2017: 5099-5108.
9 LIU Y C, FAN B, XIANG S M, et al. Relation-shape convolutional neural network for point cloud analysis [C] // Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition. Long Beach: IEEE, 2019: 8895–8904.
10 HU Q Y, YANG B, XIE L H, et al. Randla-net: efficient semantic segmentation of large-scale point clouds [C] // Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition . Seattle: IEEE, 2020: 11108–11117.
11 YAN X, ZHENG C, LI Z, et al. Pointasnl: robust point clouds processing using nonlocal neural networks with adaptive sampling [C]// Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition . Seattle: IEEE, 2020: 5589-5598.
12 ZHAO H, JIANG L, FU C W, et al. Pointweb: enhancing local neighborhood features for point cloud processing [C]// Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition . Long Beach: IEEE, 2019: 5565-5573.
13 WANG Y, SUN Y, LIU Z, et al Dynamic graph CNN for learning on point clouds[J]. ACM Transactions on Graphics, 2019, 38 (5): 1- 12
14 SIMONOVSKY M, KOMODAKIS N. Dynamic edge-conditioned filters in convolutional neural networks on graphs [C]// Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition . Honolulu: IEEE, 2017: 3693-3702.
15 LIU Z, ZHOU S, SUO C, et al. Lpd-net: 3d point cloud learning for large-scale place recognition and environment analysis [C]// Proceedings of the IEEE/CVF International Conference on Computer Vision . Seoul: IEEE, 2019: 2831-2840.
16 LAI X, LIU J, JIANG L, et al. Stratified transformer for 3d point cloud segmentation [C]// Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition . New Orleans: IEEE, 2022: 8500-8509.
17 WU W, QI Z, FUXIN L. Pointconv: deep convolutional networks on 3d point clouds [C]// Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition . Long Beach: IEEE, 2019: 9621-9630.
18 THOMAS H, QI C R, DESCHAUD J E, et al. Kpconv: flexible and deformable convolution for point clouds [C]// Proceedings of the IEEE/CVF International Conference on Computer Vision. Seoul: IEEE, 2019: 6411-6420.
19 XU M, DING R, ZHAO H, et al. Paconv: position adaptive convolution with dynamic kernel assembling on point clouds [C]// Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition . Nashville: IEEE, 2021: 3173-3182.
20 QIU S, ANWAR S, BARNES N Geometric back-projection network for point cloud classification[J]. IEEE Transactions on Multimedia, 2021, 24: 1943- 1955
21 RAN H, LIU J, WANG C. Surface representation for point clouds [C]// Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition. New Orleans: IEEE, 2022: 18942-18952.
22 ZHANG R, WANG L, WANG Y, et al. Starting from non-parametric networks for 3D point cloud analysis [C]// Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition. Vancouver: IEEE, 2023: 5344-5353.
23 CHEN B, XIA Y, ZANG Y, et al. Decoupled local aggregation for point cloud learning [EB/OL]. (2023-08-31). https://doi.org/10.48550/arXiv.2308.16532.
24 DEFFERRARD M, BRESSON X, VANDERGHEYNST P Convolutional neural networks on graphs with fast localized spectral filtering[J]. Advances in Neural Information Processing Systems, 2016, 29: 3844- 3852
25 LI G, MULLER M, THABET A, et al. Deepgcns: can gcns go as deep as cnns? [C]// Proceedings of the IEEE/CVF International Conference on Computer Vision. Seoul: IEEE 2019: 9267-9276.
26 LIN Z H, HUANG S Y, WANG Y C F. Convolution in the cloud: learning deformable kernels in 3d graph convolution networks for point cloud analysis [C]// Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition . Seattle: IEEE, 2020: 1800-1809.
27 XU M, ZHOU Z, QIAO Y. Geometry sharing network for 3d point cloud classification and segmentation [C]// Proceedings of the AAAI Conference on Artificial Intelligence . Palo Alto: AAAI Press, 2020, 34(7): 12500-12507.
28 ZHOU H, FENG Y, FANG M, et al. Adaptive graph convolution for point cloud analysis [C]// Proceedings of the IEEE/CVF International Conference on Computer Vision . Montreal: IEEE, 2021: 4965-4974.
29 VASWANI A, SHAZEER N, PARMAR N, et al. Attention is all you need [C]// Proceedings of the 31st International Conference on Neural Information Processing Systems . Long Beach: Curran Associates Inc., 2017: 6000-6010.
30 GUO M H, CAI J X, LIU Z N, et al Pct: Point cloud transformer[J]. Computational Visual Media, 2021, 7 (2): 187- 199
doi: 10.1007/s41095-021-0229-5
31 ZHAO H, JIANG L, JIA J, et al. Point transformer [C]// Proceedings of the IEEE/CVF International Conference on Computer Vision. Montreal: IEEE, 2021: 16259-16268.
32 PARK J, LEE S, KIM S, et al. Self-positioning point-based transformer for point cloud understanding [C]// Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition . Vancouver: IEEE, 2023: 21814-21823.
33 RAN H, ZHUO W, LIU J, et al. Learning inner-group relations on point clouds [C]// Proceedings of the IEEE/CVF International Conference on Computer Vision . Montreal: IEEE, 2021: 15477-15487.
34 WU X Y, JIANG L, WANG P S, et al. Point Transformer V3: simpler, faster, stronger [C]// Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition. Seattle: IEEE, 2024: 4840-4851.
35 WU Z, SONG S, KHOSLA A, et al. 3d shapenets: a deep representation for volumetric shapes [C]// Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition. Los Alamitos: IEEE, 2015: 1912-1920.
36 UY M A, PHAM Q H, HUA B S, et al. Revisiting point cloud classification: a new benchmark dataset and classification model on real-world data [C]// Proceedings of the IEEE/CVF International Conference on Computer Vision. Seoul: IEEE, 2019: 1588-1597.
37 YI L, KIM V G, CEYLAN D, et al A scalable active framework for region annotation in 3d shape collections[J]. ACM Transactions on Graphics, 2016, 35 (6): 1- 12
38 DAI A, CHANG A X, SAVVA M, et al. ScanNet: richly-annotated 3D reconstructions of indoor scenes [C]// Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition. Honolulu: IEEE, 2017: 2432-2443.
39 ARMENI I, SENER O, ZAMIR A R, et al. 3d semantic parsing of large-scale indoor spaces [C]// Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition. Las Vegas: IEEE, 2016: 1534-1543.
40 CHOE J, PARK C, RAMEAU F, et al. Pointmixer: Mlp-mixer for point cloud understanding [C]// European Conference on Computer Vision. Cham: Springer, 2022: 620-640.
41 MUZAHID A A M, WAN W, SOHEL F, et al CurveNet: curvature-based multitask learning deep networks for 3D object recognition[J]. IEEE/CAA Journal of Automatica Sinica, 2020, 8 (6): 1177- 1187
42 QIAN G, LI Y, PENG H, et al. Pointnext: revisiting pointnet++ with improved training and scaling strategies [J]. Advances in Neural Information Processing Systems , 2022: 23192-23204.
43 MONTANARO A, VALSESIA D, MAGLI E Rethinking the compositionality of point clouds through regularization in the hyperbolic space[J]. Advances in Neural Information Processing Systems, 2022, 35: 33741- 33753
44 LIU Y, TIAN B, LV Y, et al Point cloud classification using content-based transformer via clustering in feature space[J]. IEEE/CAA Journal of Automatica Sinica, 2023, 11: 231- 239
45 WOO S, LEE D, HWANG S, et al MKConv: multidimensional feature representation for point cloud analysis[J]. Pattern Recognition, 2023, 143: 109800
doi: 10.1016/j.patcog.2023.109800
46 MA X, QIN C, YOU H X, et al. Rethinking network design and local geometry in point cloud: a simple residual MLP framework [EB/OL]. (2022-02-15). https://doi.org/10.48550/arXiv.2202.07123.
47 ZEID K A, SCHULT J, HERMANS A, et al. Point2Vec for self-supervised representation learning on point clouds [C]// DAGM German Conference on Pattern Recognition . Cham: Springer, 2023: 131-146.
[1] 刘登峰,郭文静,陈世海. 基于内容引导注意力的车道线检测网络[J]. 浙江大学学报(工学版), 2025, 59(3): 451-459.
[2] 姚明辉,王悦燕,吴启亮,牛燕,王聪. 基于小样本人体运动行为识别的孪生网络算法[J]. 浙江大学学报(工学版), 2025, 59(3): 504-511.
[3] 尹向雷,屈少鹏,解永芳,苏妮. 基于渐进特征融合及多尺度空洞注意力的遮挡鸟巢检测[J]. 浙江大学学报(工学版), 2025, 59(3): 535-545.
[4] 薛雅丽,贺怡铭,崔闪,欧阳权. 基于改进YOLOv5的SAR图像有向舰船目标检测算法[J]. 浙江大学学报(工学版), 2025, 59(2): 261-268.
[5] 杨冰,徐楚阳,姚金良,向学勤. 基于单目RGB图像的三维手部姿态估计方法[J]. 浙江大学学报(工学版), 2025, 59(1): 18-26.
[6] 李灿林,王新玥,马利庄,邵志文,张文娇. 融合注意力机制和结构线提取的图像卡通化[J]. 浙江大学学报(工学版), 2024, 58(8): 1728-1737.
[7] 李忠良,陈麒,石琳,杨朝,邹先明. 时间感知组合的动态知识图谱补全[J]. 浙江大学学报(工学版), 2024, 58(8): 1738-1747.
[8] 吴书晗,王丹,陈远方,贾子钰,张越棋,许萌. 融合注意力的滤波器组双视图图卷积运动想象脑电分类[J]. 浙江大学学报(工学版), 2024, 58(7): 1326-1335.
[9] 马现伟,范朝辉,聂为之,李东,朱逸群. 对失效传感器具备鲁棒性的故障诊断方法[J]. 浙江大学学报(工学版), 2024, 58(7): 1488-1497.
[10] 杨军,张琛. 基于边界点估计与稀疏卷积神经网络的三维点云语义分割[J]. 浙江大学学报(工学版), 2024, 58(6): 1121-1132.
[11] 李运堂,李恒杰,张坤,王斌锐,关山越,陈源. 基于新型编码解码网络的复杂输电线识别[J]. 浙江大学学报(工学版), 2024, 58(6): 1133-1141.
[12] 魏翠婷,赵唯坚,孙博超,刘芸怡. 基于改进Mask R-CNN与双目视觉的智能配筋检测[J]. 浙江大学学报(工学版), 2024, 58(5): 1009-1019.
[13] 刘毅,陈一丹,高琳,洪姣. 基于多尺度特征融合的轻量化道路提取模型[J]. 浙江大学学报(工学版), 2024, 58(5): 951-959.
[14] 邢志伟,朱书杰,李彪. 基于改进图卷积神经网络的航空行李特征感知[J]. 浙江大学学报(工学版), 2024, 58(5): 941-950.
[15] 宦海,盛宇,顾晨曦. 基于遥感图像道路提取的全局指导多特征融合网络[J]. 浙江大学学报(工学版), 2024, 58(4): 696-707.