Please wait a minute...
Journal of ZheJiang University (Engineering Science)  2019, Vol. 53 Issue (7): 1374-1379    DOI: 10.3785/j.issn.1008-973X.2019.07.017
Automatic Technology, Computer Technology     
Multi-view facial landmark location method based on cascade shape regression
Gao-li SANG(),Guo-bin WANG,Rong ZHU,Jia-jia SONG
Department of Mathematics and Information Engineering, Jiaxing University, Jiaxing 314001, China
Download: HTML     PDF(728KB) HTML
Export: BibTeX | EndNote (RIS)      

Abstract  

A new cascade shape regression based facial landmark location method was proposed in order to improve the low landmark location accuracy and perform landmark location with large pose variations in a unified frame. The face area was divided according to the degree of occlusion in order to improve the accuracy of landmark location, and the shape regression was separately trained for each block. The visible/invisible attribute was introduced to the landmark's definition in order to locate the landmarks of any pose under the unified frame. The calculation and the strategy of feature extraction were improved in order to guarantee the performance of the proposed algorithm. The experimental results on the Multi-PIE, AFLW, COFW and 300-W databases show that the proposed method not only has strong robustness to pose and occlusion, but also achieves good results under other uncontrollable factors.



Key wordslandmark location      cascaded shape regression      regional division      visible/invisible attribute      multi-view     
Received: 21 May 2018      Published: 25 June 2019
CLC:  TP 391  
Cite this article:

Gao-li SANG,Guo-bin WANG,Rong ZHU,Jia-jia SONG. Multi-view facial landmark location method based on cascade shape regression. Journal of ZheJiang University (Engineering Science), 2019, 53(7): 1374-1379.

URL:

http://www.zjujournals.com/eng/10.3785/j.issn.1008-973X.2019.07.017     OR     http://www.zjujournals.com/eng/Y2019/V53/I7/1374


基于级联形状回归的多视角人脸特征点定位

针对当前人脸特征点定位精度低、且当人脸图像存在较大姿态变化时不能在同一模型框架下实现任意姿态人脸图像的面部特征点精确定位问题,提出基于级联形状回归的对姿态、遮挡都鲁棒的人脸特征点定位方法. 为了提高定位的准确度,提出按姿态偏转造成的遮挡程度对人脸区域进行分块,针对每一块分别训练形状估计回归器;为了能够在同一框架下实现任意姿态人脸的特征点定位,在特征点形状定义中引入特征点的可见/不可见属性;为了提高该算法的性能,在特征的计算方法和计算策略上分别进行改进. 在Multi-PIE、AFLW、COFW和300-W数据库上的实验结果表明,提出算法不但对姿态、遮挡具有很强的鲁棒性,而且在其他不可控因素影响下取得很好的效果.


关键词: 特征点定位,  级联形状回归,  区域划分,  可见/不可见属性,  多视角 
Fig.1 Diagram of cascade shape regression
Fig.2 Flow chart of proposed multi-view facial landmark location method based on improved cascade shape regression
Fig.3 Face region division and position and sequence of landmark
方法 e
CPR[8] 6.89
GCPR 4.14
Tab.1 Aerage location errors of visible landmarks achieved by proposed and baseline methods on Multi-PIE database
Fig.4 Location errors of proposed and baseline methods under different posed on Multi-PIE database
Fig.5 Some example results by proposed method on three different databases
方法 e
可见点 可见+不可见
CPR[8] ? 8.55
PIFA[11] ? 6.52
GCPR 4.45 5.60
Tab.2 Average location errors of visible and invisible landmarks achieved by proposed and baseline methods on AFLW database
Fig.6 Location errors of proposed and baseline methods under different posed on AFLW database
方法 e
可见点 可见+不可见
CPR[8] 9.25
RCPR[9] 8.50
CRC[15] 7.30
SDM[16] 6.69 7.70
TCDCN[17] 8.05
GCPR 5.10 5.40
Tab.3 Average location errors of visible and invisible landmarks achieved by proposed and baseline methods on COFW database
方法 普通集 挑战集 合集
RCPR[9] 6.18 17.26 8.35
SDM[16] 5.57 15.4 7.50
TCDCN[17] 4.80 8.60 5.54
ECT[18] 4.68 8.69 5.47
GCPR 4.50 7.40 5.22
Tab.4 Average location errors of landmarks achieved by proposed and baseline methods on 300-W database
[1]   WANG N, GAO X, TAO D, et al Facial feature point detection: a comprehensive survey[J]. Neurocomputing, 2018, 275: 50- 65
doi: 10.1016/j.neucom.2017.05.013
[2]   DU C, WU Q, YANG J, et al. SVM based ASM for facial landmarks location [C] // 8th IEEE International Conference on Computer and Information Technology. Chongqing: IEEE, 2008: 321-326.
[3]   石正权, 赵启军, 陈虎 基于cpr和clm的多视角人脸特征点定位方法[J]. 计算机技术与发展, 2015, 25 (11): 1- 5
SHI Zheng-quan, ZHAO Qi-jun, CHEN Hu A multi-view facial landmark localization method based on CPR and CLM[J]. Computer Technology and Development, 2015, 25 (11): 1- 5
[4]   CAUNCE A, TAYLOR C J, COOTES T F. Improved 3D model search for facial feature location and pose estimation in 2D images [C] // Proceedings of BMVC. Aberystwyth: BMVA, 2010: 1-10.
[5]   YU X, HUANG J, ZHANG S, et al. Pose-free facial landmark fitting via optimized part mixtures and cascaded deformable shape model [C] // 2013 IEEE International Conference on Computer Vision (ICCV). Sydney: IEEE, 2013: 1944-1951.
[6]   JOURABLOO A, LIU X. Large-pose face alignsment via CNN-based dense 3D model fitting [C] // Proceedings of CVPR. Las Vegas: IEEE, 2016: 4188-4196.
[7]   OZUYSAL M, CALONDER M, LEPETIT V, et al Fast keypoint recognition using random ferns[J]. IEEE Transactions on Pattern Analysis and Machine Intelligence, 2010, 32 (3): 448- 461
doi: 10.1109/TPAMI.2009.23
[8]   DOLLAR P, WELINDER P, PERONA P. Cascaded pose regression [C] // Proceedings of CVPR. San Francisco: IEEE, 2010: 1078-1085.
[9]   CAO X, WEI Y, WEN F, et al Face alignment by explicit shape regression[J]. International Journal of Computer Vision, 2014, 107 (2): 177- 190
doi: 10.1007/s11263-013-0667-3
[10]   CHAI X, WANG Q, ZHAO Y, et al Robust facial landmark detection based on initializing multiple poses[J]. International Journal of Advanced Robotic Systems, 2016, 13 (5): 1729881416662793
[11]   JOURABLOO A, LIU X. Pose-invariant 3D face alignment [C] // Proceedings of ICCV. Santiago: IEEE, 2015: 3694-3702.
[12]   WU J, TRIVEDI M M A two-stage head pose estimation framework and evaluation[J]. Pattern Recognition, 2008, 41 (3): 1138- 1158
doi: 10.1016/j.patcog.2007.07.017
[13]   KOESTINGER M, WOHLHART P, ROTH P M, et al. Annotated facial landmarks in the wild: a large-scale, real-world database for facial landmark localization [C] // Proceedings of ICCV. Barcelona: IEEE, 2011: 2144-2151.
[14]   SAGONAS C, TZIMIROPOULOS G, ZAFEIRIOU S, et al. 300 faces in-the-wild challenge: the first facial landmark localization challenge [C] // IEEE International Conference on Computer Vision Workshops. Washington: IEEE, 2013: 397–403.
[15]   FENG Z H, HUBER P, KITTLER J, et al Random cascaded-regression copse for robust facial landmark detection[J]. IEEE Signal Processing Letters, 2015, 22 (1): 76- 80
[16]   XIONG X, DE LA TORRE F. Supervised descent method and its applications to face alignment [C] // Proceedings of CVPR. Portland: IEEE, 2013: 532-539.
[17]   ZHANG Z, LUO P, LOY C C, et al Learning deep representation for face alignment with auxiliary attributes[J]. IEEE Transactions on Pattern Analysis and Machine Intelligence, 2016, 38 (5): 918- 930
doi: 10.1109/TPAMI.2015.2469286
[1] ZHU Wen-qiao, DIAO Chang-yu, XU Duan-qing, LU Dong-ming. Multi-view three-dimensional reconstruction using continuous symmetric disparity[J]. Journal of ZheJiang University (Engineering Science), 2014, 48(1): 85-91.