Please wait a minute...
浙江大学学报(理学版)  2023, Vol. 50 Issue (6): 736-744    DOI: 10.3785/j.issn.1008-9497.2023.06.008
第26届全国计算机辅助设计与图形学学术会议专题     
广义无监督函数映射学习的三维形状密集对应方法
窦丰(),马会文,谢昕洋,杨万文,石雪,韩丽,林彬()
辽宁师范大学 计算机与人工智能学院,辽宁 大连 116081
Unsupervised generalized functional map learning for arbitrary 3D shape dense correspondence
Feng DOU(),Huiwen MA,Xinyang XIE,Wanwen YANG,Xue SHI,Li HAN,Bin LIN()
School of Computing and Artificial Intelligence,Liaoning Normal University,Dalian 116081,Liaoning Province,China
 全文: PDF(1685 KB)   HTML( 1 )
摘要:

提出了一种新颖的广义无监督函数映射学习的三维形状密集对应方法。首先,基于多层感知器(multilayer perceptron,MLP)和残差网络,直接学习深度点特征。其次,计算点云的近似测地线距离,并对其进行特征分解,建立特征嵌入空间,引入注意力机制,有效学习广义基函数表示。再次,结合点特征与广义基函数生成三维形状的深度特征表示。最后,建立无监督的函数映射网络框架,获取形状之间的密集对应表示。提出的三元正则优化机制,联合重构损失、特征损失和形状匹配的距离损失,在特征域和空间域上有效提升了学习性能及形状对应的精度。实验结果表明,广义基函数表示与无监督函数映射学习机制适用于任意三维形状,突破了现有方法只适用于连续二维流形的局限,在任意三维形状匹配中取得了更优的性能。

关键词: 无监督学习形状对应函数映射深度学习    
Abstract:

This paper proposes a novel dense correspondence method based on generalized unsupervised learning. First, multilayer perceptron (MLP) and residual network are constructed to learn deep point features. Secondly, the approximate geodesic distance of the point cloud is calculated and a feature embedding space is established through feature decomposition. By employing the attention mechanism, it effectively learns the generalized basis function representation. Furthermore, the proposed method combines point features with generalized basis function to generate deep feature representations of 3D shapes. Finally, an unsupervised function mapping network is constructed to obtain dense corresponding representations between shapes. We also propose a tri-regularization mechanism that combines reconstruction loss, descriptor loss, and distance loss for shape matching, effectively improving learning performance and shape corresponding accuracy from the feature and spatial domains. Extensive experimental results have shown that the generalized basis function representation and unsupervised functional map learning mechanism are suitable for arbitrary 3D shapes, breaking through the limitations of previous methods on continuous 2D manifolds, it achieves better performance in arbitrary 3D shape matching.

Key words: unsupervised learning    shape correspondence    functional maps    deep learning
收稿日期: 2023-06-12 出版日期: 2023-11-30
CLC:  TP 391.41  
基金资助: 辽宁省科技厅应用基础研究计划项目(2023JH2/101300190)
通讯作者: 林彬     E-mail: fengdou_df@163.com;13998561021@163.com
作者简介: 窦丰(1998—),ORCID:https://orcid.org/0009-0005-8491-5850,女,硕士研究生,主要从事计算机图形学研究,E-mail: fengdou_df@163.com.
服务  
把本文推荐给朋友
加入引用管理器
E-mail Alert
RSS
作者相关文章  
窦丰
马会文
谢昕洋
杨万文
石雪
韩丽
林彬

引用本文:

窦丰,马会文,谢昕洋,杨万文,石雪,韩丽,林彬. 广义无监督函数映射学习的三维形状密集对应方法[J]. 浙江大学学报(理学版), 2023, 50(6): 736-744.

Feng DOU,Huiwen MA,Xinyang XIE,Wanwen YANG,Xue SHI,Li HAN,Bin LIN. Unsupervised generalized functional map learning for arbitrary 3D shape dense correspondence. Journal of Zhejiang University (Science Edition), 2023, 50(6): 736-744.

链接本文:

https://www.zjujournals.com/sci/CN/10.3785/j.issn.1008-9497.2023.06.008        https://www.zjujournals.com/sci/CN/Y2023/V50/I6/736

图1  对比不同特征基的模型重构
图2  网络框架
图3  任意三维形状的密集对应
数据集重构误差
欧氏距离马氏距离近似测地线距离
SHREC20110.0890.0680.032
ModelNet102.11212.320.358
表1  不同特征基的重构误差
图4  引入注意力机制前后的对应形状
图5  残缺模型的密集对应
图6  非连通模型匹配可视化
学习模式方法精度/%
有监督学习FMNet70.93
DGFM85.06
GCNN93.82
3D-CODED73.45
本文方法98.54
无监督学习Unsup FMNet40.08
Heat49.33
SURFMNet98.32
CorrNet45.56
PMF Gauss92.85
PMF Heat94.55
本文方法88.20
表2  不同方法在SHREC2011数据集上的形状对应性能
1 HUANG Q X, ZHANG G X, GAO L, et al. An optimization approach for extracting and encoding consistent maps in a shape collection [J]. ACM Transactions on Graphics, 2012, 31(6): 1-11. DOI:10.1145/2366145.2366186
doi: 10.1145/2366145.2366186
2 KIM V G, LI W, MITRA N J, et al. Exploring collections of 3D models using fuzzy correspondences[J]. ACM Transactions on Graphics, 2012, 31(4):1-11. DOI:10.1145/2185520.2185550
doi: 10.1145/2185520.2185550
3 HUANG Q X, GUIBAS L. Consistent shape maps via semi definite programming[J]. Computer Graphics Forum, 2013, 32(5): 177-186. doi:10.1111/cgf.12184
doi: 10.1111/cgf.12184
4 SAHILLIOĞLU Y, YEMEZ Y. Multiple shape correspondence by dynamic programming[J]. Computer Graphics Forum, 2015, 33(7): 121-130. DOI:10.1111/cgf.12480
doi: 10.1111/cgf.12480
5 SHTERN A, KIMMEL R. Spectral gradient fields embedding for nonrigid shape matching[J]. Computer Vision and Image Understanding, 2015, 140: 21-29. DOI:10.1016/j.cviu.2015.02.004
doi: 10.1016/j.cviu.2015.02.004
6 ALHASHIM I, XU K, ZHUANG Y, et al. Deformation-driven topology-varying 3D shape correspondence[J]. ACM Transactions on Graphics, 2015, 34(6): 1-13. DOI:10.1145/2816795.2818088
doi: 10.1145/2816795.2818088
7 MARON H, DYM N, KEZURER I, et al. Point registration via efficient convex relaxation[J]. ACM Transactions on Graphics, 2016, 35(4): 7373. DOI:10.1145/2897824.2925913
doi: 10.1145/2897824.2925913
8 DYKE R M, LAI Y K, ROSIN P L, et al. Non-rigid registration under anisotropic deformations[J]. Computer Aided Geometric Design, 2019, 71: 142-156. DOI:10.1016/j.cagd.2019.04.014
doi: 10.1016/j.cagd.2019.04.014
9 LITMAN R, BRONSTEIN A M. Learning spectral descriptors for deformable shape correspondence[J]. IEEE Transactions on Pattern Analysis & Machine Intelligence, 2014, 36(1): 171-180. DOI:10.1109/TPAMI.2013.148
doi: 10.1109/TPAMI.2013.148
10 ZHOU T, KRAHENBUHL P, AUBRY M, et al. Learning dense correspondence via 3D-guided cycle consistency[C]// 2016 IEEE Conference on Computer Vision and Pattern Recognition. Las Vegas: IEEE, 2016: 117-126. DOI:10.1109/CVPR.2016.20
doi: 10.1109/CVPR.2016.20
11 LIM I, DIELEN A, CAMPEN M, et al. A simple approach to intrinsic correspondence learning on unstructured 3D meshes[C]// Proceedings of the European Conference on Computer Vision (ECCV). Munich: Springer-Verlag, 2018: 349-362. DOI:10.1007/978-3-030-11015-4_26
doi: 10.1007/978-3-030-11015-4_26
12 LIM I, DIELEN A, CAMPEN M, et al. A simple approach to intrinsic correspondence learning on unstructured 3D meshes[C]// Proceedings of the European Conference on Computer Vision (ECCV). Munich: Springer-Verlag, 2018: 349-362. DOI:10.1007/978-3-030-11015-4_26
doi: 10.1007/978-3-030-11015-4_26
13 OVSJANIKOV M, BEN-CHEN M, SOLOMON J, et al. Functional maps: A flexible representation of maps between shapes[J]. ACM Transactions on Graphics, 2012, 31(4): 1-11. DOI:10.1145/2185520.2185526
doi: 10.1145/2185520.2185526
14 RODOLÀ E, MOELLER M, CREMERS D. Regularized pointwise map recovery from functional correspondence[J]. Computer Graphics Forum, 2017, 36(7): 700-711. DOI:10.1111/cgf.13160
doi: 10.1111/cgf.13160
15 KOVNATSKY A, BRONSTEIN M M, BRESSON X, et al. Functional correspondence by matrix completion[C]// 2015 IEEE Conference on Computer Vision and Pattern Recognition (CVPR). Boston: IEEE, 2015: 905-914. DOI:10.1109/CVPR.2015.7298692
doi: 10.1109/CVPR.2015.7298692
16 NOGNENG D, OVSJANIKOV M. Informative descriptor preservation via commutativity for shape matching[J]. Computer Graphics Forum, 2017, 36(2): 259-267. DOI:10.1111/cgf.13124
doi: 10.1111/cgf.13124
17 DONATI N, SHARMA A, OVSJANIKOV M. Deep geometric functional maps: Robust feature learning for shape correspondence[C]// 2020 IEEE/CVF Conference on Computer Vision and Pattern Recognition (CNPR). Seattle: IEEE, 2020: 8592-8601. DOI:10.1109/CVPR42600.2020.00862 .
doi: 10.1109/CVPR42600.2020.00862
18 LITANY O, REMEZ T, RODOLA E, et al. Deep functional maps: Structured prediction for dense shape correspondence[C]// Proceedings of the IEEE International Conference on Computer Vision(ICCV). Venice: IEEE, 2017: 5659-5667. DOI:10.1109/ICCV.2017.603
doi: 10.1109/ICCV.2017.603
19 HALIMI O, LITANY O, RODOLA E R, et al. Unsupervised learning of dense shape correspondence[C]// 2019 IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR). Long Beach: IEEE, 2019: 4365-4374. DOI:10.1109/CVPR.2019.00450
doi: 10.1109/CVPR.2019.00450
20 ROUFOSSE J M, SHARMA A, OVSJANIKOV M. Unsupervised deep learning for structured shape matching[C]// 2019 IEEE/CVF International Conference on Computer Vision (ICCV). Seoul: IEEE, 2019: 1617-1627. DOI:10.1109/ICCV.2019. 00170
doi: 10.1109/ICCV.2019. 00170
21 ROSEN K H. Discrete Mathematics and Its Applications[M]. 5th ed. New York: McGraw-Hill Science/Engineering/Math,2003.
22 ABADI M, AGARWAL A, BARHAM P, et al. TensorFlow: Large-scale Machine Learning on Heterogeneous Distributed Systems[Z]. (2016-03-14). https://arxiv.org/abs/1603.04467.
23 MASCI J, BOSCAINI D, BRONSTEIN M, et al. Geodesic convolutional neural networks on riemannian manifolds[C]// Proceedings of the IEEE International Conference on Computer Vision(ICCV). Santiago: IEEE, 2015: 37-45. DOI:10. 1109/ICCVW.2015.112
doi: 10. 1109/ICCVW.2015.112
24 GROUEIX T, FISHER M, KIM V G, et al. 3D-coded: 3D correspondences by deep deformation[C]// Proceedings of the European Conference on Computer Vision (ECCV). Munich: Springer, 2018: 230-246. DOI:10.1007/978-3-030-01216-8_15
doi: 10.1007/978-3-030-01216-8_15
25 AYGÜN M, LÄHNER Z, CREMERS D. Unsupervised dense shape correspondence using heat kernels[C]// 2020 International Conference on 3D Vision (3DV). Fukuoka: IEEE, 2020: 573-582. DOI:10.1109/3DV50981.2020.00067
doi: 10.1109/3DV50981.2020.00067
26 ZENG Y M, QIAN Y, ZHU Z Y, et al. CorrNet3D: Unsupervised end-to-end learning of dense correspondence for 3D point clouds[C]// 2021 IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR). Nashville: IEEE, 2021: 6048-6057. DOI:10.1109/CVPR46437.2021.00599
doi: 10.1109/CVPR46437.2021.00599
27 RODOLÀ E, LÄHNER Z, BRONSTEIN A M, et al. Functional maps representation on product manifolds[J]. Computer Graphics Forum, 2019, 38(1): 678-689. DOI:10.1111/cgf.13598
doi: 10.1111/cgf.13598
28 VESTNER M, LITMAN R, RODOLA E, et al. Product manifold filter: Non-rigid shape correspondence via kernel density estimation in the product space[C]// 2017 IEEE Conference on Computer Vision and Pattern Recognition (CVPR). Honolulu: IEEE, 2017: 3327-3336. DOI:10.1109/CVPR.2017.707
doi: 10.1109/CVPR.2017.707
[1] 刘圣军,滕子,王海波,刘新儒. 基于函数映射的二维形状内蕴对称检测算法[J]. 浙江大学学报(理学版), 2023, 50(6): 668-680.
[2] 徐圣嘉,苏程,朱孔阳,章孝灿. 基于深度学习的岩石薄片矿物自动识别方法[J]. 浙江大学学报(理学版), 2022, 49(6): 743-752.
[3] 刘华玲,张国祥,马俊. 图嵌入算法研究进展[J]. 浙江大学学报(理学版), 2022, 49(4): 443-456.
[4] 钱立辉, 王斌, 郑云飞, 章佳杰, 李马丁, 于冰. 基于图像深度预测的景深视频分类算法[J]. 浙江大学学报(理学版), 2021, 48(3): 282-288.
[5] 陈园琼, 邹北骥, 张美华, 廖望旻, 黄嘉儿, 朱承璋. 医学影像处理的深度学习可解释性研究进展[J]. 浙江大学学报(理学版), 2021, 48(1): 18-29.
[6] 傅颖颖, 张丰, 杜震洪, 刘仁义. 融合图卷积神经网络和注意力机制的PM2.5小时浓度多步预测[J]. 浙江大学学报(理学版), 2021, 48(1): 74-83.
[7] 李君轶, 任涛, 陆路正. 游客情感计算的文本大数据挖掘方法比较研究[J]. 浙江大学学报(理学版), 2020, 47(4): 507-520.
[8] 陈善雄, 王小龙, 韩旭, 刘云, 王明贵. 一种基于深度学习的古彝文识别方法[J]. 浙江大学学报(理学版), 2019, 46(3): 261-269.
[9] 黄婕, 张丰, 杜震洪, 刘仁义, 曹晓裴. 基于RNN-CNN集成深度学习模型的PM2.5小时浓度预测[J]. 浙江大学学报(理学版), 2019, 46(3): 370-379.
[10] 胡伟俭, 陈为, 冯浩哲, 张天平, 朱正茂, 潘巧明. 应用于平扫CT图像肺结节检测的深度学习方法综述[J]. 浙江大学学报(理学版), 2017, 44(4): 379-384.