|
|
Feature fusion based constrained local model for three-dimensional facial landmark localization |
Xiang-hao CHENG1,2(),Fei-peng DA1,2,*(),Liang WANG1,2 |
1. School of Automation, Southeast University, Nanjing 210096, China 2. Key Laboratory of Measurement and Control of Complex Systems of Engineering, Ministry of Education, Southeast University, Nanjing 210096, China |
|
|
Abstract An algorithm for automatic detection of landmarks on three-dimensional faces was proposed by using a feature fusion based constrained local model. A classifier based on depth information and a classifier based on local shape information of three-dimensional meshes were trained for each landmark. The responses of two classifiers were merged and the regularized landmark mean shift algorithm was applied on fitting for the localization of landmarks. Traversing candidates of each landmark is usually necessary in three-dimensional facial landmark localization to generate candidate combinations. The problem of time overhead for nested loops that increases rapidly by using model fitting instead of exhaustive search was solved. The approach was evaluated based on three-dimensional face databases: FRGC v2.0 and Bosphorus. The mean error of every landmark in the FRGC v2.0 is between 2.48 mm to 4.12 mm. The overall detection success rate is 97.3%, among which 97.6% for neutral expression, 97.4% for mild, 95.5% for extreme. On the Bosphorus database, the success rate of 94%, 95% and 89% was respectively achieved under three different poses. The experimental results show that the proposed approach is comparable to state-of-the-art methods in terms of its accuracy, and good robustness is achieved against expression and small pose variation.
|
Received: 15 March 2018
Published: 28 March 2019
|
|
Corresponding Authors:
Fei-peng DA
E-mail: 220151407@seu.edu.cn;dafp@seu.edu.cn
|
基于融合约束局部模型的三维人脸特征点定位
提出基于特征融合约束局部模型的三维人脸特征点定位算法. 该算法对每个特征点分别使用三维网格的深度信息和网格局部形状信息训练分类器,对分类器的响应进行融合. 使用基于融合响应的正则化特征点均值漂移算法进行模型拟合,实现特征点定位. 三维人脸特征点定位经常需要对每个特征点的候选点集进行遍历产生候选点组合,该算法使用模型拟合代替穷举搜索,避免了嵌套循环带来的快速增长的时间开销. 使用FRGC v2.0和Bosphorus数据库,对算法进行实验评估. FRGC v2.0库上的特征点平均误差为2.48~4.12 mm,总体检测成功率为97.3%,其中中性、温和及极端表情下的检测成功率分别为97.6%、97.4%和95.5%. Bosphorus库上3种姿态下的检测成功率分别是94%、95%和89%. 实验结果表明,提出方法具有较好的效果,对表情和小幅度的姿态变化具有较好的鲁棒性.
关键词:
三维人脸特征点定位,
约束局部模型,
特征融合,
深度信息,
局部形状信息,
正则化特征点均值漂移
|
|
[1] |
郭梦丽, 达飞鹏, 邓星, 等 基于关键点和局部特征的三维人脸识别[J]. 浙江大学学报: 工学版, 2017, 51 (3): 584- 589 GUO Meng-li, DA Fei-peng, DENG Xing, et al 3D face recognition based on keypoints and local feature[J]. Journal of Zhejiang University: Engineering Science, 2017, 51 (3): 584- 589
|
|
|
[2] |
汤兰兰, 盖绍彦, 达飞鹏, 等 基于网格纵横局部二值模式的三维人脸识别[J]. 仪器仪表学报, 2016, 37 (6): 1413- 1420 TANG Lan-lan, GAI Shao-yan, DA Fei-peng, et al A 3D face recognition method based on the local binary pattern from vertical and horizontal on the mesh[J]. Chinese Journal of Scientific Instrument, 2016, 37 (6): 1413- 1420
doi: 10.3969/j.issn.0254-3087.2016.06.027
|
|
|
[3] |
DENG X, DA F, SHAO H Efficient 3D face recognition using local covariance descriptor and Riemannian kernel sparse coding[J]. Computers and Electrical Engineering, 2017, 62 (8): 81- 91
|
|
|
[4] |
DERKACH D, SUKNO F M. Local shape spectrum analysis for 3D facial expression recognition [C] // Proceedings of IEEE International Conference on Automatic Face and Gesture Recognition (FG). Washington DC: IEEE, 2017: 41–47.
|
|
|
[5] |
ZHAO Q, OKADA K, ROSENBAUM K, et al Digital facial dysmorphology for genetic screening: Hierarchical constrained local model using ICA[J]. Medical Image Analysis, 2014, 18 (5): 699- 710
doi: 10.1016/j.media.2014.04.002
|
|
|
[6] |
COOTES T F, TAYLOR C J, COOPER D H, et al Active shape models-their training and application[J]. Computer Vision and Image Understanding, 1995, 61 (1): 38- 59
doi: 10.1006/cviu.1995.1004
|
|
|
[7] |
EDWARDS G J, COOTES T F, TAYLOR C J. Face recognition using active appearance models [C] // Proceedings of European Conference on Computer Vision (ECCV). Berlin: Springer, 1998: 581–595.
|
|
|
[8] |
CRISTINACCE D, COOTES T Automatic feature localisation with constrained local models[J]. Pattern Recognition, 2008, 41 (10): 3054- 3067
doi: 10.1016/j.patcog.2008.01.024
|
|
|
[9] |
BALTRU?AITIS T, ROBINSON P, MORENCY L P. 3D constrained local model for rigid and non-rigid facial tracking [C] // Proceedings of IEEE Conference on Computer Vision and Pattern Recognition (CVPR). Rhode Island: IEEE, 2012: 2610–2617.
|
|
|
[10] |
SARAGIH J M, LUCEY S, COHN J F Deformable model fitting by regularized landmark mean-shift[J]. International Journal of Computer Vision, 2011, 91 (2): 200- 215
doi: 10.1007/s11263-010-0380-4
|
|
|
[11] |
CHENG S, ZAFEIRIOU S, ASTHANA A, et al. 3D facial geometric features for constrained local model [C] // Proceedings of IEEE International Conference on Image Processing (ICIP). Paris: IEEE, 2014: 1425–1429.
|
|
|
[12] |
NAIR P, CAVALLARO A 3-D face detection, landmark localization, and registration using a point distribution model[J]. IEEE Transactions on Multimedia, 2009, 11 (4): 611- 623
doi: 10.1109/TMM.2009.2017629
|
|
|
[13] |
PERAKIS P, PASSALIS G, THEOHARIS T, et al 3D facial landmark detection under large yaw and expression variations[J]. IEEE Transactions on Pattern Analysis and Machine Intelligence, 2013, 35 (7): 1552- 1564
doi: 10.1109/TPAMI.2012.247
|
|
|
[14] |
CREUSOT C, PEARS N, AUSTIN J A machine-learning approach to keypoint detection and landmarking on 3D meshes[J]. International Journal of Computer Vision, 2013, 102 (1-3): 146- 179
doi: 10.1007/s11263-012-0605-9
|
|
|
[15] |
SUKNO F M, WADDINGTON J L, WHELAN P F 3-D facial landmark localization with asymmetry patterns and shape regression from incomplete local features[J]. IEEE Transactions on Cybernetics, 2015, 45 (9): 1717- 1730
doi: 10.1109/TCYB.2014.2359056
|
|
|
[16] |
GOWER J C. Generalized procrustes analysis [J]. Psychometrika, 1975, 40(1): 33-51.
|
|
|
[17] |
MOGHADDAM B, PENTLAND A Probabilistic visual learning for object representation[J]. IEEE Transactions on Pattern Analysis and Machine Intelligence, 1997, 19 (7): 696- 710
doi: 10.1109/34.598227
|
|
|
[18] |
JOHNSON A E. Spin-images: a representation for 3D surface matching [D]. Pittsburgh: Carnegie Mellon University, 1997.
|
|
|
[19] |
PHILLIPS P J, FLYNN P J, SCRUGGS T, et al. Overview of the face recognition grand challenge [C] // Proceedings of IEEE Computer Society Conference on Computer Vision and Pattern Recognition (CVPR). San Diego: IEEE, 2005: 947–954.
|
|
|
[20] |
SAVRAN A, ALYüZ N, DIBEKLIO?LU H, et al. Bosphorus database for 3D face analysis [C] // Proceedings of European Workshop on Biometrics and Identity Management (BIOID). Berlin: Springer, 2008: 47–56.
|
|
|
|
Viewed |
|
|
|
Full text
|
|
|
|
|
Abstract
|
|
|
|
|
Cited |
|
|
|
|
|
Shared |
|
|
|
|
|
Discussed |
|
|
|
|