Please wait a minute...
Journal of Zhejiang University (Science Edition)  2023, Vol. 50 Issue (6): 736-744    DOI: 10.3785/j.issn.1008-9497.2023.06.008
CCF CAD/CG 2023     
Unsupervised generalized functional map learning for arbitrary 3D shape dense correspondence
Feng DOU(),Huiwen MA,Xinyang XIE,Wanwen YANG,Xue SHI,Li HAN,Bin LIN()
School of Computing and Artificial Intelligence,Liaoning Normal University,Dalian 116081,Liaoning Province,China
Download: HTML( 1 )   PDF(1685KB)
Export: BibTeX | EndNote (RIS)      

Abstract  

This paper proposes a novel dense correspondence method based on generalized unsupervised learning. First, multilayer perceptron (MLP) and residual network are constructed to learn deep point features. Secondly, the approximate geodesic distance of the point cloud is calculated and a feature embedding space is established through feature decomposition. By employing the attention mechanism, it effectively learns the generalized basis function representation. Furthermore, the proposed method combines point features with generalized basis function to generate deep feature representations of 3D shapes. Finally, an unsupervised function mapping network is constructed to obtain dense corresponding representations between shapes. We also propose a tri-regularization mechanism that combines reconstruction loss, descriptor loss, and distance loss for shape matching, effectively improving learning performance and shape corresponding accuracy from the feature and spatial domains. Extensive experimental results have shown that the generalized basis function representation and unsupervised functional map learning mechanism are suitable for arbitrary 3D shapes, breaking through the limitations of previous methods on continuous 2D manifolds, it achieves better performance in arbitrary 3D shape matching.



Key wordsunsupervised learning      shape correspondence      functional maps      deep learning     
Received: 12 June 2023      Published: 30 November 2023
CLC:  TP 391.41  
Corresponding Authors: Bin LIN     E-mail: fengdou_df@163.com;13998561021@163.com
Cite this article:

Feng DOU,Huiwen MA,Xinyang XIE,Wanwen YANG,Xue SHI,Li HAN,Bin LIN. Unsupervised generalized functional map learning for arbitrary 3D shape dense correspondence. Journal of Zhejiang University (Science Edition), 2023, 50(6): 736-744.

URL:

https://www.zjujournals.com/sci/EN/Y2023/V50/I6/736


广义无监督函数映射学习的三维形状密集对应方法

提出了一种新颖的广义无监督函数映射学习的三维形状密集对应方法。首先,基于多层感知器(multilayer perceptron,MLP)和残差网络,直接学习深度点特征。其次,计算点云的近似测地线距离,并对其进行特征分解,建立特征嵌入空间,引入注意力机制,有效学习广义基函数表示。再次,结合点特征与广义基函数生成三维形状的深度特征表示。最后,建立无监督的函数映射网络框架,获取形状之间的密集对应表示。提出的三元正则优化机制,联合重构损失、特征损失和形状匹配的距离损失,在特征域和空间域上有效提升了学习性能及形状对应的精度。实验结果表明,广义基函数表示与无监督函数映射学习机制适用于任意三维形状,突破了现有方法只适用于连续二维流形的局限,在任意三维形状匹配中取得了更优的性能。


关键词: 无监督学习,  形状对应,  函数映射,  深度学习 
Fig.1 Compare the reconstruction of different feature bases
Fig.2 Network framework
Fig.3 Dense correspondence of arbitrary 3D shapes
数据集重构误差
欧氏距离马氏距离近似测地线距离
SHREC20110.0890.0680.032
ModelNet102.11212.320.358
Table 1 Reconstruction errors of different feature bases
Fig.4 The shape of corresponds before and after adding attention mechanism
Fig.5 Intensive correspondence of incomplete models
Fig.6 Unconnected model matching visualization
学习模式方法精度/%
有监督学习FMNet70.93
DGFM85.06
GCNN93.82
3D-CODED73.45
本文方法98.54
无监督学习Unsup FMNet40.08
Heat49.33
SURFMNet98.32
CorrNet45.56
PMF Gauss92.85
PMF Heat94.55
本文方法88.20
Table 2 The shape of different methods corresponds to the performance
[1]   HUANG Q X, ZHANG G X, GAO L, et al. An optimization approach for extracting and encoding consistent maps in a shape collection [J]. ACM Transactions on Graphics, 2012, 31(6): 1-11. DOI:10.1145/2366145.2366186
doi: 10.1145/2366145.2366186
[2]   KIM V G, LI W, MITRA N J, et al. Exploring collections of 3D models using fuzzy correspondences[J]. ACM Transactions on Graphics, 2012, 31(4):1-11. DOI:10.1145/2185520.2185550
doi: 10.1145/2185520.2185550
[3]   HUANG Q X, GUIBAS L. Consistent shape maps via semi definite programming[J]. Computer Graphics Forum, 2013, 32(5): 177-186. doi:10.1111/cgf.12184
doi: 10.1111/cgf.12184
[4]   SAHILLIOĞLU Y, YEMEZ Y. Multiple shape correspondence by dynamic programming[J]. Computer Graphics Forum, 2015, 33(7): 121-130. DOI:10.1111/cgf.12480
doi: 10.1111/cgf.12480
[5]   SHTERN A, KIMMEL R. Spectral gradient fields embedding for nonrigid shape matching[J]. Computer Vision and Image Understanding, 2015, 140: 21-29. DOI:10.1016/j.cviu.2015.02.004
doi: 10.1016/j.cviu.2015.02.004
[6]   ALHASHIM I, XU K, ZHUANG Y, et al. Deformation-driven topology-varying 3D shape correspondence[J]. ACM Transactions on Graphics, 2015, 34(6): 1-13. DOI:10.1145/2816795.2818088
doi: 10.1145/2816795.2818088
[7]   MARON H, DYM N, KEZURER I, et al. Point registration via efficient convex relaxation[J]. ACM Transactions on Graphics, 2016, 35(4): 7373. DOI:10.1145/2897824.2925913
doi: 10.1145/2897824.2925913
[8]   DYKE R M, LAI Y K, ROSIN P L, et al. Non-rigid registration under anisotropic deformations[J]. Computer Aided Geometric Design, 2019, 71: 142-156. DOI:10.1016/j.cagd.2019.04.014
doi: 10.1016/j.cagd.2019.04.014
[9]   LITMAN R, BRONSTEIN A M. Learning spectral descriptors for deformable shape correspondence[J]. IEEE Transactions on Pattern Analysis & Machine Intelligence, 2014, 36(1): 171-180. DOI:10.1109/TPAMI.2013.148
doi: 10.1109/TPAMI.2013.148
[10]   ZHOU T, KRAHENBUHL P, AUBRY M, et al. Learning dense correspondence via 3D-guided cycle consistency[C]// 2016 IEEE Conference on Computer Vision and Pattern Recognition. Las Vegas: IEEE, 2016: 117-126. DOI:10.1109/CVPR.2016.20
doi: 10.1109/CVPR.2016.20
[11]   LIM I, DIELEN A, CAMPEN M, et al. A simple approach to intrinsic correspondence learning on unstructured 3D meshes[C]// Proceedings of the European Conference on Computer Vision (ECCV). Munich: Springer-Verlag, 2018: 349-362. DOI:10.1007/978-3-030-11015-4_26
doi: 10.1007/978-3-030-11015-4_26
[12]   LIM I, DIELEN A, CAMPEN M, et al. A simple approach to intrinsic correspondence learning on unstructured 3D meshes[C]// Proceedings of the European Conference on Computer Vision (ECCV). Munich: Springer-Verlag, 2018: 349-362. DOI:10.1007/978-3-030-11015-4_26
doi: 10.1007/978-3-030-11015-4_26
[13]   OVSJANIKOV M, BEN-CHEN M, SOLOMON J, et al. Functional maps: A flexible representation of maps between shapes[J]. ACM Transactions on Graphics, 2012, 31(4): 1-11. DOI:10.1145/2185520.2185526
doi: 10.1145/2185520.2185526
[14]   RODOLÀ E, MOELLER M, CREMERS D. Regularized pointwise map recovery from functional correspondence[J]. Computer Graphics Forum, 2017, 36(7): 700-711. DOI:10.1111/cgf.13160
doi: 10.1111/cgf.13160
[15]   KOVNATSKY A, BRONSTEIN M M, BRESSON X, et al. Functional correspondence by matrix completion[C]// 2015 IEEE Conference on Computer Vision and Pattern Recognition (CVPR). Boston: IEEE, 2015: 905-914. DOI:10.1109/CVPR.2015.7298692
doi: 10.1109/CVPR.2015.7298692
[16]   NOGNENG D, OVSJANIKOV M. Informative descriptor preservation via commutativity for shape matching[J]. Computer Graphics Forum, 2017, 36(2): 259-267. DOI:10.1111/cgf.13124
doi: 10.1111/cgf.13124
[17]   DONATI N, SHARMA A, OVSJANIKOV M. Deep geometric functional maps: Robust feature learning for shape correspondence[C]// 2020 IEEE/CVF Conference on Computer Vision and Pattern Recognition (CNPR). Seattle: IEEE, 2020: 8592-8601. DOI:10.1109/CVPR42600.2020.00862 .
doi: 10.1109/CVPR42600.2020.00862
[18]   LITANY O, REMEZ T, RODOLA E, et al. Deep functional maps: Structured prediction for dense shape correspondence[C]// Proceedings of the IEEE International Conference on Computer Vision(ICCV). Venice: IEEE, 2017: 5659-5667. DOI:10.1109/ICCV.2017.603
doi: 10.1109/ICCV.2017.603
[19]   HALIMI O, LITANY O, RODOLA E R, et al. Unsupervised learning of dense shape correspondence[C]// 2019 IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR). Long Beach: IEEE, 2019: 4365-4374. DOI:10.1109/CVPR.2019.00450
doi: 10.1109/CVPR.2019.00450
[20]   ROUFOSSE J M, SHARMA A, OVSJANIKOV M. Unsupervised deep learning for structured shape matching[C]// 2019 IEEE/CVF International Conference on Computer Vision (ICCV). Seoul: IEEE, 2019: 1617-1627. DOI:10.1109/ICCV.2019. 00170
doi: 10.1109/ICCV.2019. 00170
[21]   ROSEN K H. Discrete Mathematics and Its Applications[M]. 5th ed. New York: McGraw-Hill Science/Engineering/Math,2003.
[22]   ABADI M, AGARWAL A, BARHAM P, et al. TensorFlow: Large-scale Machine Learning on Heterogeneous Distributed Systems[Z]. (2016-03-14). https://arxiv.org/abs/1603.04467.
[23]   MASCI J, BOSCAINI D, BRONSTEIN M, et al. Geodesic convolutional neural networks on riemannian manifolds[C]// Proceedings of the IEEE International Conference on Computer Vision(ICCV). Santiago: IEEE, 2015: 37-45. DOI:10. 1109/ICCVW.2015.112
doi: 10. 1109/ICCVW.2015.112
[24]   GROUEIX T, FISHER M, KIM V G, et al. 3D-coded: 3D correspondences by deep deformation[C]// Proceedings of the European Conference on Computer Vision (ECCV). Munich: Springer, 2018: 230-246. DOI:10.1007/978-3-030-01216-8_15
doi: 10.1007/978-3-030-01216-8_15
[25]   AYGÜN M, LÄHNER Z, CREMERS D. Unsupervised dense shape correspondence using heat kernels[C]// 2020 International Conference on 3D Vision (3DV). Fukuoka: IEEE, 2020: 573-582. DOI:10.1109/3DV50981.2020.00067
doi: 10.1109/3DV50981.2020.00067
[26]   ZENG Y M, QIAN Y, ZHU Z Y, et al. CorrNet3D: Unsupervised end-to-end learning of dense correspondence for 3D point clouds[C]// 2021 IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR). Nashville: IEEE, 2021: 6048-6057. DOI:10.1109/CVPR46437.2021.00599
doi: 10.1109/CVPR46437.2021.00599
[27]   RODOLÀ E, LÄHNER Z, BRONSTEIN A M, et al. Functional maps representation on product manifolds[J]. Computer Graphics Forum, 2019, 38(1): 678-689. DOI:10.1111/cgf.13598
doi: 10.1111/cgf.13598
[28]   VESTNER M, LITMAN R, RODOLA E, et al. Product manifold filter: Non-rigid shape correspondence via kernel density estimation in the product space[C]// 2017 IEEE Conference on Computer Vision and Pattern Recognition (CVPR). Honolulu: IEEE, 2017: 3327-3336. DOI:10.1109/CVPR.2017.707
doi: 10.1109/CVPR.2017.707
[1] Shengjia XU,Cheng SU,Kongyang ZHU,Xiaocan ZHANG. Automatic identification of mineral in petrographic thin sections based on images using a deep learning method[J]. Journal of Zhejiang University (Science Edition), 2022, 49(6): 743-752.
[2] Hualing LIU,Guoxiang ZHANG,Jun MA. Research progress of graph embedding algorithms[J]. Journal of Zhejiang University (Science Edition), 2022, 49(4): 443-456.
[3] QIAN Lihui, WANG Bin, ZHENG Yunfei, ZHANG Jiajie, LI Mading, YU Bing. Depth of field videos classification based on image depth prediction[J]. Journal of Zhejiang University (Science Edition), 2021, 48(3): 282-288.
[4] CHEN Yuanqiong, ZOU Beiji, ZHANG Meihua, LIAO Wangmin, HUANG Jiaer, ZHU Chengzhang. A review on deep learning interpretability in medical image processing[J]. Journal of Zhejiang University (Science Edition), 2021, 48(1): 18-29.
[5] FU Yingying, ZHANG Feng, DU Zhenhong, LIU Renyi. Multi-step prediction of PM2.5 hourly concentration by fusing graph convolution neural network and attention mechanism[J]. Journal of Zhejiang University (Science Edition), 2021, 48(1): 74-83.
[6] LI Junyi, REN Tao, LU Luzheng. A comparative study of big text data mining methods on tourist emotion computing[J]. Journal of Zhejiang University (Science Edition), 2020, 47(4): 507-520.
[7] Shanxiong CHEN, Xiaolong WANG, Xu HAN, Yun LIU, Minggui WANG. A recognition method of Ancient Yi character based on deep learning[J]. Journal of Zhejiang University (Science Edition), 2019, 46(3): 261-269.
[8] Jie HUANG, Feng ZHANG, Zhenhong DU, Renyi LIU, Xiaopei CAO. Hourly concentration prediction of PM2.5 based on RNN-CNN ensemble deep learning model[J]. Journal of Zhejiang University (Science Edition), 2019, 46(3): 370-379.
[9] HU Weijian, CHEN Wei, FENG Haozhe, ZHANG Tianping, ZHU Zhengmao, PAN Qiaoming. A survey of depth learning methods for detecting lung nodules by CT images[J]. Journal of Zhejiang University (Science Edition), 2017, 44(4): 379-384.