A 3D mesh segmentation algorithm based on graph attention network

doi:10.3785/j.issn.1008-9497.2023.06.016

Journal of Zhejiang University (Science Edition)

2023, Vol. 50

Issue (6): 811-819 DOI: 10.3785/j.issn.1008-9497.2023.06.016

CSIAM-GDC 2023

A 3D mesh segmentation algorithm based on graph attention network

Wenting LI(

),Lulu WU,Jie ZHOU,Yong ZHAO(

)

School of Mathematical Sciences，Ocean University of China，Qingdao 266100，Shandong Province，China

Download:

HTML( 2 )

PDF(1665KB)
Export: BibTeX | EndNote (RIS)

Abstract

Improving the segmentation quality of 3D meshes is always an important problem to computer graphics. To handle this problem, this paper proposes a shape-aware graph attention network. The shape-aware graph attention coefficient is defined to better reflect the similarity between nodes, which not only expands the attention coefficient obtained by network learning with the help of edge features between nodes, but also introduces the attention coefficient related to the local shape information of nodes. On the other hand, the network architecture is adjusted by taking both shape features and labels of 3D mesh model as the input of graph attention network, which enables the participation of labels in network training and verification stages. Residual connection is further employed to make the network output more stable. A large number of experiments show that the proposed algorithm can obtain accurate segmentation boundaries. Compared with the existing classical segmentation algorithms on PSB dataset, the proposed algorithm improves 2% in accuracy, and achieves better Rand index. The reasonableness of the algorithm is proved by ablation experiment.

Key words： mesh segmentation graph attention coefficient edge feature local shape information network architecture

Received: 21 June 2023 Published: 30 November 2023

CLC:

TP 391.41

Corresponding Authors: Yong ZHAO E-mail: lwt991212@163.com;zhaoyong@ouc.edu.cn

	Service
	E-mail this article
	Add to my bookshelf
	Add to citation manager
	E-mail Alert
	RSS
	Articles by authors
	Wenting LI
	Lulu WU
	Jie ZHOU
	Yong ZHAO

Cite this article:

Wenting LI,Lulu WU,Jie ZHOU,Yong ZHAO. A 3D mesh segmentation algorithm based on graph attention network. Journal of Zhejiang University (Science Edition), 2023, 50(6): 811-819.

URL:

https://www.zjujournals.com/sci/EN/Y2023/V50/I6/811

基于图注意力网络的三维网格分割算法

针对三维网格模型分割质量提升问题，提出了感知几何的图注意力网络。首先，定义了感知几何的图注意力系数，利用节点之间的边特征扩展由网络学习得到的注意力系数，引入与节点局部几何信息相关的注意力系数，更好地反映节点之间的相似性。然后，通过调整网络架构，将三维网格模型的几何特征与标签信息共同作为图注意力网络的输入，使标签信息参与网络训练和验证，并通过残差形式的线性连接实现网络的更稳定输出。大量实验结果表明，本文算法能够获得精确的分割边界，其在PSB数据集上的分割准确率较现有经典算法提升约2个百分点，也取得了更好的兰德指数。同时，通过消融实验验证了算法的合理性。

关键词： 网格分割, 图注意力系数, 边特征, 局部几何信息, 网络架构

Fig.1 Aggregation of node features in local graph structure

Fig.2 Over-segmentation result of Armadillo

Fig.3 Graph structure construction in local region

Fig.4 Architecture of graph attention network

Fig.5 Segmentation results of 3D meshes for PSB dataset

Fig.6 Comparison of segmentation results with different algorithms

Fig.7 Comparison of segmentation results under different attention coefficients

Table 1 Comparison of segmentation accuracy of different algorithm

Table 2 Comparison of Rand index of different algorithm

模型类别	网络学习得到的注意力系数 $e i j L$	感知几何的图注意力系数 $e i j$
平均值	97.2	98.2
Human	91.5	94.3
Cup	98.9	99.8
Glasses	98.3	98.6
Ant	98.6	99.1
Chair	97.0	98.8
Octopus	98.5	98.9
Table	99.1	99.7
Teddy	97.8	98.8
Armadillo	94.9	95.5

Table 3 Comparison of segmentation accuracy under different attention coefficients


[1]	SHLAFMAN S， TAL A， KATZ S. Metamorphosis of polyhedral surfaces using decomposition［J］. Computer Graphics Forum， 2002， 21（3）： 219-228. DOI：10.1111/ 1467-8659.00581 doi: 10.1111/ 1467-8659.00581

[2]	LAI Y K， HU S M， MARTIN R R， et al. Fast mesh segmentation using random walks［C］// Proceedings of the 2008 ACM Symposium on Solid and Physical Modeling. New York： ACM， 2008： 183-191. DOI：10.1145/ 1364901.1364927 doi: 10.1145/ 1364901.1364927

[3]	CHEN X B， GOLOVINSKIY A， FUNKHOUSER T. A benchmark for 3D mesh segmentation［J］. ACM Transactions on Graphics， 2009， 28（3）： 73. DOI：10. 1145/1531326.1531379 doi: 10. 1145/1531326.1531379

[4]	LIU H R， LIU W Y， LATECKI L J. Convex shape decomposition［C］// Proceedings of the 2010 IEEE Conference on Computer Vision and Pattern Recognition. Piscataway： IEEE， 2010： 97-104. DOI：10.1109/CVPR.2010. 5540225 doi: 10.1109/CVPR.2010. 5540225

[5]	ZHENG Y Y， TAI C L. Mesh decomposition with cross-boundary brushes［J］. Computer Graphics Forum， 2010， 29（2）： 527-535. DOI：10.1111/j.1467-8659.2009. 01622.x doi: 10.1111/j.1467-8659.2009. 01622.x

[6]	FAN L B， LIC L， LIU K. Paint mesh cutting［J］. Computer Graphics Forum， 2011， 30（2）： 603-612. DOI：10.1111/j.1467-8659.2011.01895.x doi: 10.1111/j.1467-8659.2011.01895.x

[7]	ZHENG Y Y， TAI C L， AU O K. Dot scissor： A single-click interface for mesh segmentation［J］. IEEE Transactions on Visualization and Computer Graphics， 2012， 18（8）： 1304-1312. DOI：10.1109/ TVCG.2011.140 doi: 10.1109/ TVCG.2011.140

[8]	TONG W H， YANG X K， PAN M D， et al. Spectral mesh segmentation via L0 gradient minimization［J］. IEEE Transactions on Visualization and Computer Graphics， 2020， 26（4）： 1807-1820. DOI：10.1109/TVCG.2018. 2882212 doi: 10.1109/TVCG.2018. 2882212

[9]	LIN C， LIU L J， LI C J， et al. SEG-MAT： 3D shape segmentation using medial axis transform［J］. IEEE Transactions on Visualization and Computer Graphics， 2022， 28（6）： 2430-2444. DOI：10.1109/ TVCG.2020.3032566 doi: 10.1109/ TVCG.2020.3032566

[10]	KALOGERAKIS E， HERTZMANN A， SINGH K. Learning 3D mesh segmentation and labeling［J］. ACM Transactions on Graphics， 2010， 29（4）： No.102. DOI：10. 1145/1778765.1778839 doi: 10. 1145/1778765.1778839

[11]	BENHABILES H， LAVOUÉ G， VANDEBORRE J P， et al. Learning boundary edges for 3D-mesh segmentation［J］. Computer Graphics Forum， 2011， 30（8）： 2170-2182. DOI：10.1111/j.1467-8659.2011. 01967.x doi: 10.1111/j.1467-8659.2011. 01967.x

[12]	WANG Y H， GONG M L， WANG T H， et al. Projective analysis for 3D shape segmentation［J］. ACM Transactions on Graphics， 2013， 32（6）： 192. DOI：10. 1145/2508363.2508393 doi: 10. 1145/2508363.2508393

[13]	XU W W， SHI Z X， XU M L， et al. Transductive 3D shape segmentation using sparse reconstruction［J］. Computer Graphics Forum， 2014， 33（5）： 107-115. DOI：10.1111/cgf. 12436 doi: 10.1111/cgf. 12436

[14]	XIE Z G， XU K， LIU L G， et al. 3D shape segmentation and labeling via extreme learning machine［J］. Computer Graphics Forum， 2014， 33（5）： 85-95. DOI：10.1111/cgf. 12434 doi: 10.1111/cgf. 12434

[15]	KALOGERAKIS E， AVERKIOU M， MAJI S， et al. 3D shape segmentation with projective convolutional networks［C］// Proceedings of the 2017 IEEE Conference on Computer Vision and Pattern Recognition. Piscataway： IEEE， 2017： 6630-6639. DOI：10. 1109/ CVPR.2017.702 doi: 10. 1109/ CVPR.2017.702

[16]	KUNDU A， YIN X Q， FATHI A， et al. Virtual multi-view fusion for 3D semantic segmentation［C］// Proceedings of the 16th European Conference on Computer Vision. Berlin： Springer， 2020： 518-535. DOI：10.1007/978-3-030-58586-0\_31. doi:10.1007/978-3-030-58586-0_31 doi: 10.1007/978-3-030-58586-0_31

[17]	GUO K， ZOU D Q， CHEN X W. 3D mesh labeling via deep convolutional neural networks［J］. ACM Transactions on Graphics， 2015， 35（1）： 3. DOI：10.1145/2835487 doi: 10.1145/2835487

[18]	GEORGE D， XIE X H， TAM G K L. 3D mesh segmentation via multi-branch 1D convolutional neural networks［J］. Graphical Models， 2018， 96： 1-10. DOI：10.1016/j.gmod. 2018.01.001 doi: 10.1016/j.gmod. 2018.01.001

[19]	GEORGE D， XIE X H， LAI Y K， et al. A deep learning driven active framework for segmentation of large 3D shape collections［J］. Computer-Aided Design， 2022， 144： 103179. DOI：10.1016/j.cad.2021.103179 doi: 10.1016/j.cad.2021.103179

[20]	LI Y， HE X Y， JIANG Y K， et al. MeshFormer： High-resolution mesh segmentation with graph transformer［J］. Computer Graphics Forum， 2022， 41（7）： 38-49. DOI：10.1111/cgf.14655 doi: 10.1111/cgf.14655

[21]	HANOCKA R， HERTZ A， FISH N， et al. MeshCNN： A network with edge［J］. ACM Transactions on Graphics， 2019， 38（4）： 90. DOI：10.1145/3306346. 332295 doi: 10.1145/3306346. 332295

[22]	HUANG J W， ZHANG H T， YI L， et al. TextureNet： Consistent local parametrizations for learning from high-resolution signals on meshes［C］// Proceedings of the 2019 IEEE Conference on Computer Vision and Pattern Recognition. Piscataway： IEEE， 2019： 4440-4449. DOI：10.1109/CVPR. 2019.00457 doi: 10.1109/CVPR. 2019.00457

[23]	FENG Y T， FENG Y F， YOU H X， et al. MeshNet： Mesh neural network for 3D shape representation［C］// Proceedings of the 23th AAAI Conference on Artificial Intelligence. Palo Alto： AAAI， 2019： 8279-8286. DOI：10. 1609/aaai.v33i01.33018279 doi: 10. 1609/aaai.v33i01.33018279

[24]	HE W C， JIANG Z， ZHANG C M， et al. CurvaNet： Geometric deep learning based on directional curvature for 3D shape analysis［C］// Proceedings of the 26th ACM SIGKDD Conference on Knowledge Discovery and Data Mining. New York： ACM， 2020： 2214-2224. DOI：10.1145/3394486.3403272 doi: 10.1145/3394486.3403272

[25]	LAHAV A， TAL A. MeshWalker： Deep mesh understanding by random walks［J］. ACM Transactions on Graphics， 2020， 39（6）： 263. DOI：10.1145/3414685. 3417806 doi: 10.1145/3414685. 3417806

[26]	HU S M， LIU Z N， GUO M H， et al. Subdivision-based mesh convolution networks［J］. ACM Transactions on Graphics， 2022， 41（3）： No.25. DOI：10.1145/3506694 doi: 10.1145/3506694

[27]	QIAO Y L， GAO L， YANG J， et al. Learning on 3D meshes with Laplacian encoding and pooling［J］. IEEE Transactions on Visualization and Computer Graphics， 2022， 28（2）： 1317-1327. DOI：10.1109/ TVCG.2020.3014449 doi: 10.1109/ TVCG.2020.3014449

[28]	SHARP N， ATTAIKI S， CRANE K， et al. DiffusionNet： Discretization agnostic learning on surfaces［J］. ACM Transactions on Graphics， 2022， 41（3）： No. 27. DOI：10.1145/3507905 doi: 10.1145/3507905

[29]	PÉREZ D， SHEN Y Z， LI J. Mesh convolutional networks with face and vertex feature operators［J］. IEEE Transactions on Visualization and Computer Graphics， 2023， 29（3）： 1678-1690. DOI：10.1109/ TVCG.2021.3129156 doi: 10.1109/ TVCG.2021.3129156

[30]	VELICKOVIC P， CUCURULL G， CASANOVA A， et al. Graph attention networks［C］// Proceedings of the 6th International Conference on Learning Representations. Vancouver： Open Review， 2018： 1-12.

[31]	KIM D， OH A. How to find your friendly neighborhood： Graph attention design with self-supervision［C］// Proceedings of the 9th International Conference on Learning Representations. Vancouver： Open Review， 2021：1-25.

[32]	WU L L， HOU Y， XU J L， et al. Robust mesh segmentation using feature-aware region fusion［J］. Sensors， 2023， 23（1）： No.416. DOI：10.3390/s23010416 doi: 10.3390/s23010416

[33]	MEYER M， DESBRUN M， SCHRÖDER P， et al. Discrete differential-geometry operators for triangulated 2-manifolds［C］// Proceedings of the 3rd International Workshop “Visualization and Mathematics”. Berlin： Springer， 2002： 35-57. DOI：10.1007/978-3-662-05105-4\_ 2. doi:10.1007/978-3-662-05105-4_2 doi: 10.1007/978-3-662-05105-4_2

[34]	SHAPIRA L， SHAMIR A， COHEN-OR D. Consistent mesh partitioning and skeletonisation using the shape diameter function［J］. The Visual Computer， 2008， 24（4）： 249-259. DOI：10.1007/s00371-007-0197-5 doi: 10.1007/s00371-007-0197-5

[35]	HILAGA M， SHINAGAWA Y， KOMURA T. Topology matching for fully automatic similarity estimation of 3D shapes［C］// Proceedings of the 28th Annual Conference on Computer Graphics and Interactive Techniques. New York： ACM， 2001： 203-212. DOI：10.1145/383259. 383282 doi: 10.1145/383259. 383282

[36]	BENCHEN M， GOTSMAN C. Characterizing shape using conformal factors［C］// Proceedings of the 1st Eurographics Workshop on 3D Object Retrieval. New York： ACM， 2008： 1-8. DOI：10.2312/3dor/3dor08/001-008 doi: 10.2312/3dor/3dor08/001-008

[37]	SUN J， OVSJANIKOV M， GUIBAS L. A concise and provably informative multi-scale signature based on heat diffusion［J］. Computer Graphics Forum， 2009， 28（5）： 1383-1392. DOI：10.1111/j.1467-8659.2009. 01515.x doi: 10.1111/j.1467-8659.2009. 01515.x

[38]	ZHANG G G， MA Q H， JIAO L C， et al. AttAN： Attention adversarial networks for 3D point cloud semantic segmentation［C］// Proceedings of the 29th International Joint Conference on Artificial Intelligence. San Mateo： Morgan Kaufmann， 2020： 789-796. DOI：10. 24963/ijcai.2020/110 doi: 10. 24963/ijcai.2020/110

[39]	GOLOVINSKIY A， FUNKHOUSER T. Randomized cuts for 3D mesh analysis［J］. ACM Transactions on Graphics， 2008， 27（5）： No.145. DOI：10.1145/1409060. 1409098 doi: 10.1145/1409060. 1409098

[1]	Hongyu FAN,Pengbo BO. Parametric tread pattern model retrieval based on geometric features[J]. Journal of Zhejiang University (Science Edition), 2023, 50(6): 803-810.

[2]	Xinjing LI,Wanbin PAN,Ye YANG,Yigang WANG,Cheng LIN. A double-level intelligent improvement approach for overhangs on side loss[J]. Journal of Zhejiang University (Science Edition), 2023, 50(6): 781-794.

[3]	Yujie LIU,Yafu YUAN,Xiaorui SUN,Zongmin LI. A point cloud processing network combining global and local information[J]. Journal of Zhejiang University (Science Edition), 2023, 50(6): 770-780.

[4]	Feng DOU,Huiwen MA,Xinyang XIE,Wanwen YANG,Xue SHI,Li HAN,Bin LIN. Unsupervised generalized functional map learning for arbitrary 3D shape dense correspondence[J]. Journal of Zhejiang University (Science Edition), 2023, 50(6): 736-744.

[5]	Zechu ZHANG,Weilong PENG,Keke TANG,Zhaoyang YU,Asad Khan,Meie FANG. Reconstructing tooth meshes by pyramid diffeomorphic deformation from CBCT images[J]. Journal of Zhejiang University (Science Edition), 2023, 50(6): 701-710.

[6]	Hanyang MAO,Chen PENG,Chen LI,Changbo WANG. Neural marching cubes for open surfaces[J]. Journal of Zhejiang University (Science Edition), 2023, 50(6): 692-700.

[7]	Kehua SU,Bailüe LIU,Na LEI,Kehan LI,Xianfeng GU. Focus+Context visualization based on optimal mass transportation[J]. Journal of Zhejiang University (Science Edition), 2023, 50(6): 681-691.

[8]	Shengjun LIU,Zi TENG,Haibo WANG,Xinru LIU. Two-dimensional shape intrinsic symmetry detection algorithm based on functional map[J]. Journal of Zhejiang University (Science Edition), 2023, 50(6): 668-680.

[9]	Zerun LIU,Yufei YIN,Wenhao XUE,Rui GUO,Lechao CHENG. A review of conditional image generation based on diffusion models[J]. Journal of Zhejiang University (Science Edition), 2023, 50(6): 651-667.

[10]	Yuhua FANG,Feng YE. MFDC-Net: A breast cancer pathological image classification algorithm incorporating multi-scale feature fusion and attention mechanism[J]. Journal of Zhejiang University (Science Edition), 2023, 50(4): 455-464.

[11]	Ruiqi YU,Yuhua LIU,Xilong SHEN,Ruyu ZHAI,Xiang ZHANG,Zhiguang ZHOU. Representation learning driven multiple graph sampling[J]. Journal of Zhejiang University (Science Edition), 2022, 49(3): 271-279.

[12]	Ying ZHONG,Song WANG,Hao WU,Zepeng CHENG,Xuejun LI. SEMMA-Based visual exploration of cyber security event[J]. Journal of Zhejiang University (Science Edition), 2022, 49(2): 131-140.

[13]	Jintai ZHU, Jihua YE, Feng GUO, Lu JIANG, Aiwen JIANG. FSAGN:An expression recognition method based on independent selection of video key frames[J]. Journal of Zhejiang University (Science Edition), 2022, 49(2): 141-150.

[14]	Qiang ZHU,Chaoyi WANG,Jiqing ZHANG,Baocai YIN,Xiaopeng WEI,Xin YANG. UAV target tracking algorithm based on event camera[J]. Journal of Zhejiang University (Science Edition), 2022, 49(1): 10-18.

[15]	Meng YANG,Shu DING,Yuntao MA,Jiayi XIE,Ruifeng DUAN. Dynamic simulation method of wheat rust based on texture feature[J]. Journal of Zhejiang University (Science Edition), 2022, 49(1): 1-9.

Viewed

Full text

Abstract

Cited

Shared

Discussed