结合全局信息和局部信息的三维网格分割框架

doi:10.3785/j.issn.1008-973X.2025.05.004

浙江大学学报(工学版)

2025, Vol. 59

Issue (5): 912-919 DOI: 10.3785/j.issn.1008-973X.2025.05.004

计算机技术、信息工程

结合全局信息和局部信息的三维网格分割框架

张梦瑶(

),周杰,李文婷,赵勇*(

)

中国海洋大学数学科学学院，山东青岛 266100

Three-dimensional mesh segmentation framework using global and local information

Mengyao ZHANG(

),Jie ZHOU,Wenting LI,Yong ZHAO*(

)

School of Mathematical Sciences, Ocean University of China, Qingdao 266100, China

全文: PDF(1497 KB) HTML

摘要：

针对Graph Transformer比较擅长捕获全局信息，但对局部精细信息的提取不够充分的问题，将图卷积神经网络(GCN)引入Graph Transformer中，得到Graph Transformer and GCN (GTG)模块，构建了能够结合全局信息和局部信息的网格分割框架. GTG模块利用Graph Transformer的全局自注意力机制和GCN的局部连接性质，不仅可以捕获全局信息，还能够加强局部精细信息的提取. 为了更好地保留边界区域的信息，设计边缘保持的粗化算法，可以使粗化过程仅作用在非边界区域. 利用边界信息对损失函数进行加权，提高了神经网络对边界区域的关注程度. 在实验方面，通过视觉效果和定量比较证明了采用本文算法能够获得高质量的分割结果，利用消融实验表明了GTG模块和边缘保持粗化算法的有效性.

关键词： 三维网格; 网格分割; Graph Transformer; 图卷积神经网络(GCN); 边缘保持的粗化算法

Abstract:

A Graph Transformer and GCN (GTG) block was obtained by introducing graph convolutional neural network (GCN) into Graph Transformer because Graph Transformer was good at capturing global information, but weak in extracting local fine-grained information. A mesh segmentation framework that combined both global and local information was constructed. Global self-attention mechanism of Graph Transformer and local connectivity properties of GCN were used in GTG block in order to capture global information and enhance the extraction of local fine-grained information. An edge-preserving coarsening algorithm was designed to constrain the coarsening to non-boundary regions in order to better preserve information in boundary regions. Boundary information was used to weight the loss function to enhance the neural network’s focus on boundary regions. In experiments, visual results and quantitative comparisons prove that the proposed algorithm can achieve high-quality segmentation results, and ablation study demonstrates the effectiveness of GTG block and edge-preserving coarsening algorithm.

Key words: 3D mesh mesh segmentation Graph Transformer graph convolutional neural network (GCN) edge-preserving coarsening algorithm

收稿日期: 2024-07-06 出版日期: 2025-04-25

CLC:

TP 391

基金资助: 山东省自然科学基金资助项目(ZR2018MF006); 浙江大学CAD&CG国家重点实验室开放课题资助项目(A2228); 青岛市自然科学基金资助项目(23-2-1-158-zyyd-jch).

通讯作者: 赵勇 E-mail: zhangmengyao@stu.ouc.edu.cn;zhaoyong@ouc.edu.cn

作者简介: 张梦瑶（1999—），女，硕士生，从事计算机图形学、深度学习的研究. orcid.org/0009-0001-2118-3171.E-mail：zhangmengyao@stu.ouc.edu.cn

	服务
	把本文推荐给朋友
	加入引用管理器
	E-mail Alert
	作者相关文章
	张梦瑶
	周杰
	李文婷
	赵勇

引用本文:

张梦瑶,周杰,李文婷,赵勇. 结合全局信息和局部信息的三维网格分割框架[J]. 浙江大学学报(工学版), 2025, 59(5): 912-919.

Mengyao ZHANG,Jie ZHOU,Wenting LI,Yong ZHAO. Three-dimensional mesh segmentation framework using global and local information. Journal of ZheJiang University (Engineering Science), 2025, 59(5): 912-919.

链接本文:

https://www.zjujournals.com/eng/CN/10.3785/j.issn.1008-973X.2025.05.004 或 https://www.zjujournals.com/eng/CN/Y2025/V59/I5/912

图 1 自注意力机制

图 2 节点的多尺度特征生成

图 3 Graph Transformer模块

图 4 GTG模块

图 5 Ant模型过分割结果的图结构

图 6 提出方法的网格分割框架

图 7 PSB中部分网格模型的分割结果

图 8 Plier模型的分割结果

图 9 Armadillo模型的分割结果

表 1 PSB数据集的分割准确率比较

表 2 消融实验的分割准确率

1	KALOGERAKIS E, HERTZMANN A, SINGH K Learning 3D mesh segmentation and labeling[J]. ACM Transactions on Graphics, 2010, 29 (4): 102
2	BENHABILES H, LAVOUÉ G, VANDEBORRE J, et al Learning boundary edges for 3D-mesh segmentation[J]. Computer Graphics Forum, 2011, 30 (8): 2170- 2182 doi: 10.1111/j.1467-8659.2011.01967.x
3	HU R, FAN L, LIU L Co-segmentation of 3D shapes via subspace clustering[J]. Computer Graphics Forum, 2012, 31 (5): 1703- 1713 doi: 10.1111/j.1467-8659.2012.03175.x
4	WANG Y, GONG M, WANG T, et al Projective analysis for 3D shape segmentation[J]. ACM Transactions on Graphics, 2013, 32 (6): 192
5	GUO K, ZOU D, CHEN X 3D mesh labeling via deep convolutional neural networks[J]. ACM Transactions on Graphics, 2015, 35 (1): 3
6	GEORGE D, XIE X, TAM G K 3D mesh segmentation via multi-branch 1D convolutional neural networks[J]. Graphical Models, 2018, 96: 1- 10 doi: 10.1016/j.gmod.2018.01.001
7	XIE Z, XU K, SHAN W, et al Projective feature learning for 3D shapes with multi-view depth images[J]. Computer Graphics Forum, 2015, 34 (7): 1- 11 doi: 10.1111/cgf.12740
8	KALOGERAKIS E, AVERKIOU M, MAJI S, et al. 3D shape segmentation with projective convolutional networks [C]// Proceedings of IEEE/CVF Conference on Computer Vision and Pattern Recognition . Honolulu: IEEE, 2017: 6630-6639.
9	KUNDU A, YIN X, FATHI A, et al. Virtual multi-view fusion for 3D semantic segmentation [C]// Proceedings of the European Conference on Computer Vision . Glasgow: Springer, 2020: 518-535.
10	WANG P, LIU Y, GUO Y, et al O-CNN: octree-based convolutional neural networks for 3D shape analysis[J]. ACM Transactions on Graphics, 2017, 36 (4): 72
11	WANG Z, LU F VoxSegNet: volumetric CNNs for semantic part segmentation of 3D shapes[J]. IEEE Transactions on Visualization and Computer Graphics, 2020, 26 (9): 2919- 2930 doi: 10.1109/TVCG.2019.2896310
12	HU Z, BAI X, SHANG J, et al. VMNet: voxel-mesh network for geodesic-aware 3D semantic segmentation [C]// Proceedings of the IEEE International Conference on Computer Vision . Venice: IEEE, 2021: 15468-15478.
13	XU H, DONG M, ZHONG Z. Directionally convolutional networks for 3D shape segmentation [C]// Proceedings of the IEEE International Conference on Computer Vision . Venice: IEEE, 2017: 2717-2726.
14	HUANG J, ZHANG H, YI L, et al. TextureNet: consistent local parametrizations for learning from high-resolution signals on meshes [C]// Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition . Long Beach: IEEE, 2019: 4440-4449.
15	FENG Y, FENG Y, YOU H, et al. MeshNet: mesh neural network for 3D shape representation [C]// Proceedings of the AAAI Conference on Artificial Intelligence . Palo Alto: AAAI Press, 2019: 8279-8286.
16	HANOCKA R, HERTZ A, FISH N, et al MeshCNN: a network with edge[J]. ACM Transactions on Graphics, 2019, 38 (4): 90
17	HU S, LIU Z, GUO M, et al Subdivision-based mesh convolution networks[J]. ACM Transactions on Graphics, 2022, 41 (3): 25
18	PÉREZ D, SHEN Y, LI J Mesh convolutional networks with face and vertex feature operators[J]. IEEE Transactions on Visualization and Computer Graphics, 2023, 29 (3): 1678- 1690 doi: 10.1109/TVCG.2021.3129156
19	LAHAV A, TAL A MeshWalker: deep mesh understanding by random walks[J]. ACM Transactions on Graphics, 2020, 39 (6): 263
20	SHARP N, ATTAIKI S, CRANE K, et al DiffusionNet: discretization agnostic learning on surfaces[J]. ACM Transactions on Graphics, 2022, 41 (3): 27
21	QIAO Y, GAO L, YANG J, et al Learning on 3D meshes with Laplacian encoding and pooling[J]. IEEE Transactions on Visualization and Computer Graphics, 2022, 28 (2): 1317- 1327 doi: 10.1109/TVCG.2020.3014449
22	DONG Q, WANG Z, LI M, et al Laplacian2Mesh: Laplacian-based mesh understanding[J]. IEEE Transactions on Visualization and Computer Graphics, 2023, 30 (7): 4349- 4361
23	WONG C. Heat diffusion based multi-scale and geometric structure-aware transformer for mesh segmentation [C]// Proceedings of IEEE/CVF Conference on Computer Vision and Pattern Recognition . Vancouver: IEEE, 2023: 4413-4422.
24	YI L, SU H, GUO X, et al. SyncSpecCNN: synchronized spectral CNN for 3D shape segmentation [C]// Proceedings of IEEE/CVF Conference on Computer Vision and Pattern Recognition . Honolulu: IEEE, 2017: 6584-6592.
25	SCHULT J, ENGELMANN F, KONTOGIANNI T, et al. DualConvMesh-net: joint geodesic and Euclidean convolutions on 3D meshes [C]// Proceedings of IEEE/CVF Conference on Computer Vision and Pattern Recognition . Seattle: IEEE, 2020: 8609-8619.
26	LI X, YANG J, ZHANG F. Laplacian mesh transformer: dual attention and topology aware network for 3D mesh classification and segmentation [C]// Proceedings of the European Conference on Computer Vision . Tel Aviv: Springer, 2022: 541-560.
27	ROY B. Neural shape diameter function for efficient mesh segmentation [C]// Proceedings of ACM SIGGRAPH Annual Conference Posters . Los Angeles: ACM, 2023.
28	XU X, LIU C, ZHENG Y 3D tooth segmentation and labeling using deep convolutional neural networks[J]. IEEE Transactions on Visualization and Computer Graphics, 2019, 25 (7): 2336- 2348 doi: 10.1109/TVCG.2018.2839685
29	LI Y, HE X, JIANG Y, et al MeshFormer: high-resolution mesh segmentation with graph transformer[J]. Computer Graphics Forum, 2022, 41 (7): 38- 49
30	LV J, CHEN X, HUANG J, et al Semi-supervised mesh segmentation and labeling[J]. Computer Graphics Forum, 2012, 31 (7): 2241- 2248 doi: 10.1111/j.1467-8659.2012.03217.x
31	SHU Z, SHEN X, XIN S, et al Scribble-based 3D shape segmentation via weakly-supervised learning[J]. IEEE Transactions on Visualization and Computer Graphics, 2020, 26 (8): 2671- 2682 doi: 10.1109/TVCG.2019.2892076
32	SHU Z, YANG S, WU H, et al 3D shape segmentation using soft density peak clustering and semi-supervised learning[J]. Computer-Aided Design, 2022, 145: 103181 doi: 10.1016/j.cad.2021.103181
33	SHU Z, QI C, XIN S, et al Unsupervised 3D shape segmentation and co-segmentation via deep learning[J]. Computer-Aided Geometric Design, 2016, 43: 39- 52 doi: 10.1016/j.cagd.2016.02.015
34	LIANG Y, ZHAO S, YU B, et al. MeshMAE: masked autoencoders for 3D mesh data analysis [C]// Proceedings of the European Conference on Computer Vision . Tel Aviv: Springer, 2022: 37-54.
35	JIAO X, CHEN Y, YANG X SCMS-Net: self-supervised clustering-based 3D meshes segmentation network[J]. Computer-Aided Design, 2023, 160: 103512 doi: 10.1016/j.cad.2023.103512
36	LOUKAS A Graph reduction with spectral and cut guarantees[J]. Journal of Machine Learning Research, 2019, 20 (116): 1- 42
37	ZHANG Z, LIU Q, HU Q, et al. Hierarchical graph transformer with adaptive node sampling [C]// Proceedings of the Conference on Neural Information Proceeding Systems . New Orleans: MIT Press, 2022, 35: 21171-21183.
38	WU L, HOU Y, XU J, et al Robust mesh segmentation using feature-aware region fusion[J]. Sensor, 2023, 23 (1): 416
39	SHAPIRA L, SHAMIR A, COHEN-OR D Consistent mesh partitioning and skeletonisation using the shape diameter function[J]. The Visual Computer, 2008, 24 (4): 249- 259 doi: 10.1007/s00371-007-0197-5
40	BEN-CHEN M, GOTSMAN C. Characterizing shape using conformal factors [C]// Proceedings of the Eurographics Conference on 3D Object Retrieval . Crete: Springer, 2008: 1-8.
41	SUN J, OVSJANIKOV M, GUIBAS L A concise and provably informative multi-scale signature based on heat diffusion[J]. Computer Graphics Forum, 2009, 28 (5): 1383- 1392 doi: 10.1111/j.1467-8659.2009.01515.x
42	CHEN X, GOLOVINSKIY A, FUNKHOUSER T A benchmark for 3D mesh segmentation[J]. ACM Transactions on Graphics, 2009, 28 (3): 73

[1]	舒振宇, 汪国昭. 基于张量投票的快速网格分割算法[J]. J4, 2011, 45(6): 999-1005.
[2]	钱江陈志杨叶修梓张三元. 基于参数化技术的网格分割[J]. J4, 2008, 42(8): 1370-1375.

Viewed

Full text

Abstract

Cited

Shared

Discussed