Please wait a minute...
浙江大学学报(工学版)  2022, Vol. 56 Issue (10): 1935-1947    DOI: 10.3785/j.issn.1008-973X.2022.10.005
自动化技术、信息工程     
无监督的三维模型簇对应关系协同计算
杨军1,2(),李金泰1,高志明1
1. 兰州交通大学 电子与信息工程学院,甘肃 兰州 730070
2. 兰州交通大学 测绘与地理信息学院,甘肃 兰州 730070
Unsupervised co-calculation on correspondence of three-dimensional shape collections
Jun YANG1,2(),Jin-tai LI1,Zhi-ming GAO1
1. School of Electronic and Information Engineering, Lanzhou Jiaotong University, Lanzhou 730070, China
2. Faculty of Geomatics, Lanzhou Jiaotong University, Lanzhou 730070, China
 全文: PDF(3261 KB)   HTML
摘要:

针对近似等距的非刚性变换的三维模型簇对应关系计算准确率不高的问题,提出采用无监督的三维模型簇对应关系协同计算的新方法. 通过三维点云特征提取模块,获取将位置、细节信息更丰富的低维特征与语义信息更丰富的高维特征相融合后的特征. 在无监督深度函数映射模块中,将提取到的融合特征转换为谱描述符,计算函数映射矩阵,对该矩阵施加加权正则化约束项,得到最优的函数映射矩阵. 在模型簇对应关系协同计算模块中,结合循环一致性约束与函数映射理论,求解最优的目标函数,得到最优的模型簇对应关系. 实验结果表明,所提算法在FAUST、SCAPE和TOSCA 3个数据集上所构建的模型簇对应关系测地误差均小于目前主流方法,映射结果更加平滑,对应关系更加准确,具有良好的泛化能力.

关键词: 对应关系三维模型簇无监督学习深度函数映射循环一致性    
Abstract:

A new method of unsupervised collaborative calculation of the correspondence between the three-dimensional (3D) shape collections was proposed aiming at the problem that the calculation accuracy of the correspondence between the 3D shape collections of the near-isometric non-rigid was not high. The feature extraction module of the 3D point cloud was used to obtain the features after fusing low-dimensional features with richer location and detail information and high-dimensional features with richer semantic information. The extracted fusion features were converted into spectral descriptors in the unsupervised deep functional maps module, and the matrix of the functional map was calculated. The optimal function mapping matrix was obtained by applying weighted regularization constraints to the matrix. The optimal objective function was solved by combining the cycle-consistency constraint and functional maps theory in the shape collection correspondence cooperative calculation module in order to obtain the optimal shape collection correspondence. The experimental results showed that the geodesic errors of the 3D shape collection correspondence constructed by this algorithm on the FAUST, SCAPE and TOSCA datasets were smaller than the current commonly used methods. The mapping results are more smoother and the correspondence is more accurate, which has good generalization ability.

Key words: shape correspondence    three-dimensional shape collection    unsupervised learning    deep functional map    cycle-consistency
收稿日期: 2021-06-18 出版日期: 2022-10-25
CLC:  TP 391  
基金资助: 国家自然科学基金资助项目(61862039, 42261067);甘肃省科技计划资助项目(20JR5RA429);2021年度中央引导地方科技发展资金资助项目(2021-51);兰州市人才创新创业资助项目(2020-RC-22);兰州交通大学天佑创新团队资助项目(TY202002);甘肃省教育厅优秀研究生“创新之星”资助项目(2021CXZX-614)
作者简介: 杨军(1973—),男,教授,博导,从事深度学习、计算机图形学、遥感图像解译等研究. orcid.org/0000-0001-6403-3408. E-mail: yangj@mail.lzjtu.cn
服务  
把本文推荐给朋友
加入引用管理器
E-mail Alert
作者相关文章  
杨军
李金泰
高志明

引用本文:

杨军,李金泰,高志明. 无监督的三维模型簇对应关系协同计算[J]. 浙江大学学报(工学版), 2022, 56(10): 1935-1947.

Jun YANG,Jin-tai LI,Zhi-ming GAO. Unsupervised co-calculation on correspondence of three-dimensional shape collections. Journal of ZheJiang University (Engineering Science), 2022, 56(10): 1935-1947.

链接本文:

https://www.zjujournals.com/eng/CN/10.3785/j.issn.1008-973X.2022.10.005        https://www.zjujournals.com/eng/CN/Y2022/V56/I10/1935

图 1  循环一致性约束
图 2  无监督动态图卷积循环一致函数映射网络框架
图 3  FAUST数据集上不同方法构建的人体模型簇一致对应关系结果的对比
图 4  SCAPE数据集上不同方法构建的人体模型簇一致对应关系结果的对比
图 5  TOSCA数据集上不同方法构建的人体模型簇一致对应关系结果的对比
图 6  TOSCA数据集上不同方法构建的猫模型簇一致对应关系结果的对比
图 7  TOSCA数据集上不同方法构建的狗模型簇一致对应关系结果的对比
图 8  TOSCA数据集上不同方法构建的马模型簇一致对应关系结果的对比
图 9  TOSCA数据集上不同方法构建的半人马模型簇一致对应关系结果对比
模型 $\overline G _{\rm{e}} $
文献[12]算法 文献[24]算法 文献[27]算法 文献[28]算法 本文算法
人体模型簇(FAUST) 0.039 0.058 0.124 0.097 0.026
人体模型簇(SCAPE) 0.107 0.174 0.449 0.217 0.046
人体模型簇(TOSCA) 0.046 0.036 0.168 0.157 0.025
猫模型 0.087 0.139 0.329 0.131 0.023
狗模型 0.334 0.169 0.393 0.244 0.034
马模型 0.375 0.258 0.284 0.129 0.037
半人马模型簇 0.069 0.053 0.171 0.065 0.028
表 1  不同算法的平均测地误差
图 10  不同算法在FAUST、SCAPE和TOSCA数据集上的测地误差曲线
数据集 Ce
文献[12]算法 文献[24]算法 文献[27]算法 文献[28]算法 本文算法
FAUST 0.30 0 0.54 0.28 0
SCAPE 0.08 0 0.68 0.12 0
TOSCA 0.26 0 0.58 0.32 0
表 2  不同算法的模型簇对应关系的循环误差
算法 tp/s tr/s tt/s
文献[27]算法 300 3600 3900
文献[28]算法 5400 5400
本文算法 180 7200 7380
表 3  不同方法的运行时间比较
三维点云
特征提取模块
无监督深度
函数映射模块
模型簇对应关系
协同计算模块
Ge
FAUST SCAPE TOSCA
0.059 0.127 0.103
0.103 0.205 0.086
0.185 0.288 0.176
0.026 0.046 0.025
表 4  验证各模块有效性的消融实验对比
1 SAHILLIOGLU Y Recent advances in shape correspondence[J]. The Visual Computer, 2020, 36 (8): 1705- 1721
doi: 10.1007/s00371-019-01760-0
2 BESL P J, MCKAY N D Method for registration of 3D shapes[J]. IEEE Transactions on Pattern Analysis and Machine Intelligence, 1992, 14 (2): 239- 256
doi: 10.1109/34.121791
3 OVSJANIKOV M, BEN C M, SOLOMON J, et al Functional maps: a flexible representation of maps between shapes[J]. ACM Transactions on Graphics, 2012, 31 (4): 30
4 OVSJANIKOV M, CORMAN E, BRONSTEIN M, et al. Computing and processing correspondences with functional maps [EB/OL]. [2021-06-01]. http://hal.inria.fr/hal-01664767/file/siggraph17_course_notes.pdf.
5 CHEN M, WANG C, QIN H Jointly learning shape descriptors and their correspondence via deep triplet CNNs[J]. Computer Aided Geometric Design, 2018, 62 (5): 192- 205
6 GROUEIX T, FISHER M, KIM V G, et al. 3D-CODED: 3D correspondences by deep deformation [C]// European Conference on Computer Vision. Munich: Springer, 2018: 235-251.
7 GINZBURG D, RAVIV D. Cyclic functional mapping: self-supervised correspondence between non-isometric deformable shapes [M]. Cham: Springer, 2020.
8 AYGUN M, LAHNER Z, CREMERS D. Unsupervised dense shape correspondence using heat kernels [C]// International Conference on 3D Vision. Fukuoka: IEEE, 2020: 573-582.
9 KIM V G, LI W, MITRA N J, et al Exploring collections of 3D models using fuzzy correspondences[J]. ACM Transactions on Graphics, 2012, 31 (4): 54
10 HUANG Q X, GUO X An optimization approach for extracting and encoding consistent maps in a shape collection[J]. ACM Transactions on Graphics, 2012, 31 (6): 654- 660
11 HUANG Q, WANG F, GUIBAS L Functional map networks for analyzing and exploring large shape collections[J]. ACM Transactions on Graphics, 2014, 33 (4): 1- 11
12 HUANG R, REN J, WONKA P, et al Consistent zoomout: efficient spectral map synchronization[J]. Computer Graphics Forum, 2020, 39 (5): 265- 278
doi: 10.1111/cgf.14084
13 REN J, POULENARD A, WONKA P, et al Continuous and orientation-preserving correspondences via functional maps[J]. ACM Transactions on Graphics, 2019, 37 (6): 248- 263
14 杨军, 闫寒 校准三维模型基矩阵的函数映射的对应关系计算[J]. 武汉大学学报·信息科学版, 2018, 43 (10): 1518- 1525
YANG Jun, YAN Han Correspondence calculation of functional maps for calibrating the base matrix of 3D model[J]. Geomatics and Information Science of Wuhan University, 2018, 43 (10): 1518- 1525
15 WU Y, YANG J, ZHAO J L Partial 3D shape functional correspondence via fully spectral eigenvalue alignment and upsampling refinement[J]. Computers and Graphics, 2020, 92 (6): 99- 113
doi: 10.1016/j.cag.2020.09.004
16 EISENBERGER M, LAHNER Z, CREMERS D. Smooth shells: multi-scale shape registration with functional maps [C]// Conference on Computer Vision and Pattern Recognition. Seattle: IEEE, 2020: 12265-12274.
17 LITANY O, REMEZ T, RODOLA E, et al. Deep functional maps: structured prediction for dense shape correspondence [C]// International Conference on Computer Vision. Venice: IEEE, 2017: 5659-5667.
18 HALIMI O, LITANY O, RODOLA E, et al. Unsupervised learning of dense shape correspondence [C]// Conference on Computer Vision and Pattern Recognition. Los Angeles: IEEE, 2019: 4370-4379.
19 DONATI N, SHARMA A, OVSJANIKOV M. Deep geometric functional maps: robust feature learning for shape correspondence [C]// Conference on Computer Vision and Pattern Recognition. Seattle: IEEE, 2020: 8592-8601.
20 HUANG Q, GUIBAS L Consistent shape maps via semidefinite programming[J]. Computer Graphics Forum, 2013, 32 (5): 177- 186
doi: 10.1111/cgf.12184
21 KIM V G, LI W, MITRA N J, et al Learning part-based templates from large collections of 3D shapes[J]. ACM Transactions on Graphics, 2013, 32 (4): 1- 12
22 COSMO L, RODOLA E, ALBARELLI A, et al Consistent partial matching of shape collections via sparse modeling[J]. Computer Graphics Forum, 2017, 36 (1): 209- 221
doi: 10.1111/cgf.12796
23 杨军, 雷鸣 结合函数映射与循环一致性约束的模型簇对应关系计算[J]. 激光与光电子学进展, 2019, 56 (8): 123- 133
YANG Jun, LEI Ming Correspondence calculation of shape collections by functional maps combined with cycle-consistency constraints[J]. Laser and Optoelectronics Progress, 2019, 56 (8): 123- 133
24 BERNARD F, THUNBERG J, SWOBODA, et al. HiPPI: higher-order projected power iterations for scalable multi-matching [C]// International Conference on Computer Vision. Seoul: IEEE, 2019: 10284-10293.
25 HUANG Q X, SU H, GUIBAS L Fine-grained semi-supervised labeling of large shape collections[J]. ACM Transactions on Graphics, 2013, 32 (6): 1- 10
26 FISH N, KAICK O V, BERMANO A, et al Structure-oriented networks of shape collections[J]. ACM Transactions on Graphics, 2016, 35 (6): 1- 14
27 HUANG Q, LIANG Z, WANG H, et al Tensor maps for synchronizing heterogeneous shape collections[J]. ACM Transactions on Graphics, 2019, 38 (4): 1- 18
28 GROUEIX T, FISHER M, KIM V G, et al Unsupervised cycle-consistent deformation for shape matching[J]. Computer Graphics Forum, 2019, 38 (5): 123- 133
doi: 10.1111/cgf.13794
29 WANG Y, SUN Y, LIU Z, et al Dynamic graph CNN for learning on point clouds[J]. ACM Transactions on Graphics, 2019, 38 (5): 1- 12
30 WANG Y, SOLOMON J M. Deep closest point: learning representations for point cloud registration [C]// International Conference on Computer Vision. Seoul: IEEE, 2019: 3523-3532.
31 GINZBURG D, DAN R. Dual geometric graph network (DG2N): iterative network for deformable shape alignment [C]// International Conference on 3D Vision. [S. l.]: IEEE, 2021: 1341-1350.
32 REN J, PANINE M, WONKA P, et al Structured regularization of functional map computations[J]. Computer Graphics Forum, 2019, 38 (5): 39- 53
doi: 10.1111/cgf.13788
33 BOGO F, ROMERO J, LOPER M, et al. FAUST: Dataset and evaluation for 3D mesh registration [C]// Conference on Computer Vision and Pattern Recognition. Columbus: IEEE, 2014: 3794-3801.
34 ANGUELOV D, SRINIVASAN P, KOLLER D, et al SCAPE: shape completion and animation of people[J]. ACM Transactions on Graphics, 2005, 24 (3): 408- 416
doi: 10.1145/1073204.1073207
35 BRONSTEIN A M, BRONSTEIN M M, KIMMEL R. Numerical geometry of non-rigid shapes [M]. Berlin: Springer, 2008.
[1] 张融,张为. 基于改进GhostNet-FCOS的火灾检测算法[J]. 浙江大学学报(工学版), 2022, 56(10): 1891-1899.
[2] 王万良,王铁军,陈嘉诚,尤文波. 融合多尺度和多头注意力的医疗图像分割方法[J]. 浙江大学学报(工学版), 2022, 56(9): 1796-1805.
[3] 殷小亮,钱承. 基于自适应采样的复杂模型全局近似[J]. 浙江大学学报(工学版), 2022, 56(9): 1815-1823.
[4] 王海军,张圣燕,杜玉杰. 响应和滤波器偏差感知约束的无人机目标跟踪算法[J]. 浙江大学学报(工学版), 2022, 56(9): 1824-1832.
[5] 郝琨,王阔,王贝贝. 基于改进Mobilenet-YOLOv3的轻量级水下生物检测算法[J]. 浙江大学学报(工学版), 2022, 56(8): 1622-1632.
[6] 孙菲,厉小润,赵辽英,余绍奇. 基于FrFT变换和全变分正则化的异常检测算法[J]. 浙江大学学报(工学版), 2022, 56(7): 1276-1284.
[7] 牛伟宏,黄晓峰,祁伟,殷海兵,颜成钢. H.266/VVC分步全零块判决快速算法[J]. 浙江大学学报(工学版), 2022, 56(7): 1285-1293, 1319.
[8] 张师林,郭红南,刘轩. 基于能量模型的行人与车辆再识别方法[J]. 浙江大学学报(工学版), 2022, 56(7): 1416-1424.
[9] 张迅,李建胜,欧阳文,陈润泽,汲振,郑凯. 融合运动信息和跟踪评价的高效卷积算子[J]. 浙江大学学报(工学版), 2022, 56(6): 1135-1143, 1167.
[10] 周国华,卢剑伟,倪彤光,胡学龙. 层次型非线性子空间字典学习[J]. 浙江大学学报(工学版), 2022, 56(6): 1159-1167.
[11] 王友卫,童爽,凤丽洲,朱建明,李洋,陈福. 基于图卷积网络的归纳式微博谣言检测新方法[J]. 浙江大学学报(工学版), 2022, 56(5): 956-966.
[12] 徐小高,夏莹杰,朱思雨,邝砾. 基于强化学习的多路口可变车道协同控制方法[J]. 浙江大学学报(工学版), 2022, 56(5): 987-994, 1005.
[13] 何立,庞善民. 结合年龄监督和人脸先验的语音-人脸图像重建[J]. 浙江大学学报(工学版), 2022, 56(5): 1006-1016.
[14] 陈成,张皞,李永强,冯远静. 关系生成图注意力网络的知识图谱链接预测[J]. 浙江大学学报(工学版), 2022, 56(5): 1025-1034.
[15] 张雪芹,李天任. 基于Cycle-GAN和改进DPN网络的乳腺癌病理图像分类[J]. 浙江大学学报(工学版), 2022, 56(4): 727-735.