Please wait a minute...
Journal of ZheJiang University (Engineering Science)  2022, Vol. 56 Issue (10): 1935-1947    DOI: 10.3785/j.issn.1008-973X.2022.10.005
    
Unsupervised co-calculation on correspondence of three-dimensional shape collections
Jun YANG1,2(),Jin-tai LI1,Zhi-ming GAO1
1. School of Electronic and Information Engineering, Lanzhou Jiaotong University, Lanzhou 730070, China
2. Faculty of Geomatics, Lanzhou Jiaotong University, Lanzhou 730070, China
Download: HTML     PDF(3261KB) HTML
Export: BibTeX | EndNote (RIS)      

Abstract  

A new method of unsupervised collaborative calculation of the correspondence between the three-dimensional (3D) shape collections was proposed aiming at the problem that the calculation accuracy of the correspondence between the 3D shape collections of the near-isometric non-rigid was not high. The feature extraction module of the 3D point cloud was used to obtain the features after fusing low-dimensional features with richer location and detail information and high-dimensional features with richer semantic information. The extracted fusion features were converted into spectral descriptors in the unsupervised deep functional maps module, and the matrix of the functional map was calculated. The optimal function mapping matrix was obtained by applying weighted regularization constraints to the matrix. The optimal objective function was solved by combining the cycle-consistency constraint and functional maps theory in the shape collection correspondence cooperative calculation module in order to obtain the optimal shape collection correspondence. The experimental results showed that the geodesic errors of the 3D shape collection correspondence constructed by this algorithm on the FAUST, SCAPE and TOSCA datasets were smaller than the current commonly used methods. The mapping results are more smoother and the correspondence is more accurate, which has good generalization ability.



Key wordsshape correspondence      three-dimensional shape collection      unsupervised learning      deep functional map      cycle-consistency     
Received: 18 June 2021      Published: 25 October 2022
CLC:  TP 391  
Fund:  国家自然科学基金资助项目(61862039, 42261067);甘肃省科技计划资助项目(20JR5RA429);2021年度中央引导地方科技发展资金资助项目(2021-51);兰州市人才创新创业资助项目(2020-RC-22);兰州交通大学天佑创新团队资助项目(TY202002);甘肃省教育厅优秀研究生“创新之星”资助项目(2021CXZX-614)
Cite this article:

Jun YANG,Jin-tai LI,Zhi-ming GAO. Unsupervised co-calculation on correspondence of three-dimensional shape collections. Journal of ZheJiang University (Engineering Science), 2022, 56(10): 1935-1947.

URL:

https://www.zjujournals.com/eng/10.3785/j.issn.1008-973X.2022.10.005     OR     https://www.zjujournals.com/eng/Y2022/V56/I10/1935


无监督的三维模型簇对应关系协同计算

针对近似等距的非刚性变换的三维模型簇对应关系计算准确率不高的问题,提出采用无监督的三维模型簇对应关系协同计算的新方法. 通过三维点云特征提取模块,获取将位置、细节信息更丰富的低维特征与语义信息更丰富的高维特征相融合后的特征. 在无监督深度函数映射模块中,将提取到的融合特征转换为谱描述符,计算函数映射矩阵,对该矩阵施加加权正则化约束项,得到最优的函数映射矩阵. 在模型簇对应关系协同计算模块中,结合循环一致性约束与函数映射理论,求解最优的目标函数,得到最优的模型簇对应关系. 实验结果表明,所提算法在FAUST、SCAPE和TOSCA 3个数据集上所构建的模型簇对应关系测地误差均小于目前主流方法,映射结果更加平滑,对应关系更加准确,具有良好的泛化能力.


关键词: 对应关系,  三维模型簇,  无监督学习,  深度函数映射,  循环一致性 
Fig.1 Cycle-consistency constraint
Fig.2 Unsupervised DGCNN cycle-consistency functional map network framework
Fig.3 Comparison of consistent correspondences of human shape collections constructed by different methods on FAUST dataset
Fig.4 Comparison of consistent correspondences of human shape collections constructed by different methods on SCAPE dataset
Fig.5 Comparison of consistent correspondences of human shape collections constructed by different methods on TOSCA dataset
Fig.6 Comparison of consistent correspondences of cat shape collections constructed by different methods on TOSCA dataset
Fig.7 Comparison of consistent correspondences of dog shape collections constructed by different methods on TOSCA dataset
Fig.8 Comparison of consistent correspondences of horse shape collections constructed by different methods on TOSCA dataset
Fig.9 Comparison of consistent correspondences of centaur shape collections constructed by different methods on TOSCA dataset
模型 $\overline G _{\rm{e}} $
文献[12]算法 文献[24]算法 文献[27]算法 文献[28]算法 本文算法
人体模型簇(FAUST) 0.039 0.058 0.124 0.097 0.026
人体模型簇(SCAPE) 0.107 0.174 0.449 0.217 0.046
人体模型簇(TOSCA) 0.046 0.036 0.168 0.157 0.025
猫模型 0.087 0.139 0.329 0.131 0.023
狗模型 0.334 0.169 0.393 0.244 0.034
马模型 0.375 0.258 0.284 0.129 0.037
半人马模型簇 0.069 0.053 0.171 0.065 0.028
Tab.1 Average geodesic errors of different methods
Fig.10 Geodesic error curves of different algorithms on FAUST, SCAPE and TOSCA datasets
数据集 Ce
文献[12]算法 文献[24]算法 文献[27]算法 文献[28]算法 本文算法
FAUST 0.30 0 0.54 0.28 0
SCAPE 0.08 0 0.68 0.12 0
TOSCA 0.26 0 0.58 0.32 0
Tab.2 Cycle error on correspondence of shape collections of different algorithms
算法 tp/s tr/s tt/s
文献[27]算法 300 3600 3900
文献[28]算法 5400 5400
本文算法 180 7200 7380
Tab.3 Comparison of running time between different methods
三维点云
特征提取模块
无监督深度
函数映射模块
模型簇对应关系
协同计算模块
Ge
FAUST SCAPE TOSCA
0.059 0.127 0.103
0.103 0.205 0.086
0.185 0.288 0.176
0.026 0.046 0.025
Tab.4 Comparison of ablation experiments to verify effectiveness of each module
[1]   SAHILLIOGLU Y Recent advances in shape correspondence[J]. The Visual Computer, 2020, 36 (8): 1705- 1721
doi: 10.1007/s00371-019-01760-0
[2]   BESL P J, MCKAY N D Method for registration of 3D shapes[J]. IEEE Transactions on Pattern Analysis and Machine Intelligence, 1992, 14 (2): 239- 256
doi: 10.1109/34.121791
[3]   OVSJANIKOV M, BEN C M, SOLOMON J, et al Functional maps: a flexible representation of maps between shapes[J]. ACM Transactions on Graphics, 2012, 31 (4): 30
[4]   OVSJANIKOV M, CORMAN E, BRONSTEIN M, et al. Computing and processing correspondences with functional maps [EB/OL]. [2021-06-01]. http://hal.inria.fr/hal-01664767/file/siggraph17_course_notes.pdf.
[5]   CHEN M, WANG C, QIN H Jointly learning shape descriptors and their correspondence via deep triplet CNNs[J]. Computer Aided Geometric Design, 2018, 62 (5): 192- 205
[6]   GROUEIX T, FISHER M, KIM V G, et al. 3D-CODED: 3D correspondences by deep deformation [C]// European Conference on Computer Vision. Munich: Springer, 2018: 235-251.
[7]   GINZBURG D, RAVIV D. Cyclic functional mapping: self-supervised correspondence between non-isometric deformable shapes [M]. Cham: Springer, 2020.
[8]   AYGUN M, LAHNER Z, CREMERS D. Unsupervised dense shape correspondence using heat kernels [C]// International Conference on 3D Vision. Fukuoka: IEEE, 2020: 573-582.
[9]   KIM V G, LI W, MITRA N J, et al Exploring collections of 3D models using fuzzy correspondences[J]. ACM Transactions on Graphics, 2012, 31 (4): 54
[10]   HUANG Q X, GUO X An optimization approach for extracting and encoding consistent maps in a shape collection[J]. ACM Transactions on Graphics, 2012, 31 (6): 654- 660
[11]   HUANG Q, WANG F, GUIBAS L Functional map networks for analyzing and exploring large shape collections[J]. ACM Transactions on Graphics, 2014, 33 (4): 1- 11
[12]   HUANG R, REN J, WONKA P, et al Consistent zoomout: efficient spectral map synchronization[J]. Computer Graphics Forum, 2020, 39 (5): 265- 278
doi: 10.1111/cgf.14084
[13]   REN J, POULENARD A, WONKA P, et al Continuous and orientation-preserving correspondences via functional maps[J]. ACM Transactions on Graphics, 2019, 37 (6): 248- 263
[14]   杨军, 闫寒 校准三维模型基矩阵的函数映射的对应关系计算[J]. 武汉大学学报·信息科学版, 2018, 43 (10): 1518- 1525
YANG Jun, YAN Han Correspondence calculation of functional maps for calibrating the base matrix of 3D model[J]. Geomatics and Information Science of Wuhan University, 2018, 43 (10): 1518- 1525
[15]   WU Y, YANG J, ZHAO J L Partial 3D shape functional correspondence via fully spectral eigenvalue alignment and upsampling refinement[J]. Computers and Graphics, 2020, 92 (6): 99- 113
doi: 10.1016/j.cag.2020.09.004
[16]   EISENBERGER M, LAHNER Z, CREMERS D. Smooth shells: multi-scale shape registration with functional maps [C]// Conference on Computer Vision and Pattern Recognition. Seattle: IEEE, 2020: 12265-12274.
[17]   LITANY O, REMEZ T, RODOLA E, et al. Deep functional maps: structured prediction for dense shape correspondence [C]// International Conference on Computer Vision. Venice: IEEE, 2017: 5659-5667.
[18]   HALIMI O, LITANY O, RODOLA E, et al. Unsupervised learning of dense shape correspondence [C]// Conference on Computer Vision and Pattern Recognition. Los Angeles: IEEE, 2019: 4370-4379.
[19]   DONATI N, SHARMA A, OVSJANIKOV M. Deep geometric functional maps: robust feature learning for shape correspondence [C]// Conference on Computer Vision and Pattern Recognition. Seattle: IEEE, 2020: 8592-8601.
[20]   HUANG Q, GUIBAS L Consistent shape maps via semidefinite programming[J]. Computer Graphics Forum, 2013, 32 (5): 177- 186
doi: 10.1111/cgf.12184
[21]   KIM V G, LI W, MITRA N J, et al Learning part-based templates from large collections of 3D shapes[J]. ACM Transactions on Graphics, 2013, 32 (4): 1- 12
[22]   COSMO L, RODOLA E, ALBARELLI A, et al Consistent partial matching of shape collections via sparse modeling[J]. Computer Graphics Forum, 2017, 36 (1): 209- 221
doi: 10.1111/cgf.12796
[23]   杨军, 雷鸣 结合函数映射与循环一致性约束的模型簇对应关系计算[J]. 激光与光电子学进展, 2019, 56 (8): 123- 133
YANG Jun, LEI Ming Correspondence calculation of shape collections by functional maps combined with cycle-consistency constraints[J]. Laser and Optoelectronics Progress, 2019, 56 (8): 123- 133
[24]   BERNARD F, THUNBERG J, SWOBODA, et al. HiPPI: higher-order projected power iterations for scalable multi-matching [C]// International Conference on Computer Vision. Seoul: IEEE, 2019: 10284-10293.
[25]   HUANG Q X, SU H, GUIBAS L Fine-grained semi-supervised labeling of large shape collections[J]. ACM Transactions on Graphics, 2013, 32 (6): 1- 10
[26]   FISH N, KAICK O V, BERMANO A, et al Structure-oriented networks of shape collections[J]. ACM Transactions on Graphics, 2016, 35 (6): 1- 14
[27]   HUANG Q, LIANG Z, WANG H, et al Tensor maps for synchronizing heterogeneous shape collections[J]. ACM Transactions on Graphics, 2019, 38 (4): 1- 18
[28]   GROUEIX T, FISHER M, KIM V G, et al Unsupervised cycle-consistent deformation for shape matching[J]. Computer Graphics Forum, 2019, 38 (5): 123- 133
doi: 10.1111/cgf.13794
[29]   WANG Y, SUN Y, LIU Z, et al Dynamic graph CNN for learning on point clouds[J]. ACM Transactions on Graphics, 2019, 38 (5): 1- 12
[30]   WANG Y, SOLOMON J M. Deep closest point: learning representations for point cloud registration [C]// International Conference on Computer Vision. Seoul: IEEE, 2019: 3523-3532.
[31]   GINZBURG D, DAN R. Dual geometric graph network (DG2N): iterative network for deformable shape alignment [C]// International Conference on 3D Vision. [S. l.]: IEEE, 2021: 1341-1350.
[32]   REN J, PANINE M, WONKA P, et al Structured regularization of functional map computations[J]. Computer Graphics Forum, 2019, 38 (5): 39- 53
doi: 10.1111/cgf.13788
[33]   BOGO F, ROMERO J, LOPER M, et al. FAUST: Dataset and evaluation for 3D mesh registration [C]// Conference on Computer Vision and Pattern Recognition. Columbus: IEEE, 2014: 3794-3801.
[34]   ANGUELOV D, SRINIVASAN P, KOLLER D, et al SCAPE: shape completion and animation of people[J]. ACM Transactions on Graphics, 2005, 24 (3): 408- 416
doi: 10.1145/1073204.1073207
[35]   BRONSTEIN A M, BRONSTEIN M M, KIMMEL R. Numerical geometry of non-rigid shapes [M]. Berlin: Springer, 2008.
[1] Rong ZHANG,Wei ZHANG. Fire detection algorithm based on improved GhostNet-FCOS[J]. Journal of ZheJiang University (Engineering Science), 2022, 56(10): 1891-1899.
[2] Wan-liang WANG,Tie-jun WANG,Jia-cheng CHEN,Wen-bo YOU. Medical image segmentation method combining multi-scale and multi-head attention[J]. Journal of ZheJiang University (Engineering Science), 2022, 56(9): 1796-1805.
[3] Xiao-liang YIN,Cheng QIAN. Global approximation of complex model based on adaptive sampling[J]. Journal of ZheJiang University (Engineering Science), 2022, 56(9): 1815-1823.
[4] Hai-jun WANG,Sheng-yan ZHANG,Yu-jie DU. UAV object tracking algorithm based on response and filter deviation-aware regularization[J]. Journal of ZheJiang University (Engineering Science), 2022, 56(9): 1824-1832.
[5] Kun HAO,Kuo WANG,Bei-bei WANG. Lightweight underwater biological detection algorithm based on improved Mobilenet-YOLOv3[J]. Journal of ZheJiang University (Engineering Science), 2022, 56(8): 1622-1632.
[6] Fei SUN,Xiao-run LI,Liao-ying ZHAO,Shao-qi YU. Anomaly detection algorithm based on FrFT transform and total variation regularization[J]. Journal of ZheJiang University (Engineering Science), 2022, 56(7): 1276-1284.
[7] Wei-hong NIU,Xiao-feng HUANG,Wei QI,Hai-bing YIN,Cheng-gang YAN. Fast stepwise all zero block detection algorithm for H.266/VVC[J]. Journal of ZheJiang University (Engineering Science), 2022, 56(7): 1285-1293, 1319.
[8] Shi-lin ZHANG,Hong-nan GUO,Xuan LIU. Person and vehicle re-identification based on energy model[J]. Journal of ZheJiang University (Engineering Science), 2022, 56(7): 1416-1424.
[9] Xun ZHANG,Jian-sheng LI,Wen OUYANG,Run-ze CHEN,Zhen JI,Kai ZHENG. Efficient convolution operators integrating motion information and tracking evaluation[J]. Journal of ZheJiang University (Engineering Science), 2022, 56(6): 1135-1143, 1167.
[10] Guo-hua ZHOU,Jian-wei LU,Tong-guang NI,Xue-long HU. Hierarchical nonlinear subspace dictionary learning[J]. Journal of ZheJiang University (Engineering Science), 2022, 56(6): 1159-1167.
[11] You-wei WANG,Shuang TONG,Li-zhou FENG,Jian-ming ZHU,Yang LI,Fu CHEN. New inductive microblog rumor detection method based on graph convolutional network[J]. Journal of ZheJiang University (Engineering Science), 2022, 56(5): 956-966.
[12] Xiao-gao XU,Ying-jie XIA,Si-yu ZHU,Li KUANG. Cooperative control algorithm of multi-intersection variable-direction lanes based on reinforcement learning[J]. Journal of ZheJiang University (Engineering Science), 2022, 56(5): 987-994, 1005.
[13] Li HE,Shan-min PANG. Face reconstruction from voice based on age-supervised learning and face prior information[J]. Journal of ZheJiang University (Engineering Science), 2022, 56(5): 1006-1016.
[14] Cheng CHEN,Hao ZHANG,Yong-qiang LI,Yuan-jing FENG. Knowledge graph link prediction based on relational generative graph attention network[J]. Journal of ZheJiang University (Engineering Science), 2022, 56(5): 1025-1034.
[15] Xue-qin ZHANG,Tian-ren LI. Breast cancer pathological image classification based on Cycle-GAN and improved DPN network[J]. Journal of ZheJiang University (Engineering Science), 2022, 56(4): 727-735.