|
|
|
| Face frontalization based on generative adversarial network |
Hui-ya HU1,2( ),Shao-yan GAI1,2,3,Fei-peng DA1,2,3,*() |
1. Key Laboratory of Measurement and Control of Complex System of Engineering, Ministry of Education, Southeast University, Nanjing 210096, China
2. School of Automation, Southeast University, Nanjing 210096, China
3. Shenzhen Institute of Southeast University, Shenzhen 518000, China |
|
|
|
Abstract The original deep convolutional generative adversarial network (DCGAN) structure was replaced with boundary equilibrium generative adversarial network (BEGAN) by drawing on the experience of two-pathway generative adversarial network (TP-GAN) in order to improve the effect of frontalizing the rotated faces. The classifier used for discriminating face identity was added to the traditional two player network in order to form the three-player structure. Adding classifier is more effective for maintaining the consistency of face identity compared with adding constraints in the generator loss function. The adopted BEGAN can improve the training efficiency aiming at the problem that TP-GAN are complex to train and prone to the mode collapse. The experimental results on the multiple images across pose, illumination and expression (Multi-PIE) and labeled faces in the wild (LFW) dataset show that the proposed method can generate high-quality frontal face images with retaining the identity information.
|
|
Received: 03 January 2020
Published: 05 January 2021
|
|
|
|
Corresponding Authors:
Fei-peng DA
E-mail: 220171559@seu.edu.cn;dafp@seu.edu.cn
|
基于生成对抗网络的偏转人脸转正
为了提高偏转人脸转正的效果，借鉴双通道生成对抗网络（TP-GAN）双通道生成的思想，将原始网络中的深度卷积生成对抗网络（DCGAN）替换成边界均衡生成对抗网络（BEGAN）. 在传统两者对抗的网络结构中加入判别人脸身份的分类器，形成三者对抗的网络结构. 经实验对比可知，与在生成器损失函数中添加约束相比，结构上加入分类器对人脸身份一致性的保持更加有效. TP-GAN存在训练复杂、模式崩溃等难题，使用BEGAN的网络结构，可以避免这些问题，提高训练效率. 在Multi-PIE数据集及LFW上的实验结果表明，利用提出的方法能够高效地生成高质量的正面人脸图片，且保留人脸的身份特征.
关键词：
人脸生成,
分类器,
模式崩溃,
生成对抗网络（GAN）
|
|
| [1] |
SUN Y, CHEN Y, WANG X, et al. Deep learning face representation by joint identification-verification [C] // International Conference on Neural Information Processing Systems. Montreal: MIT Press, 2014.
|
|
|
| [2] |
SUN Y, WANG X G, TANG X O. Deep learning face representation from predicting 10, 000 classes [C] // IEEE Conference on Computer Vision and Pattern Recognition. Ohio: IEEE, 2014: 1891-1898.
|
|
|
| [3] |
LI S, LIU X, CHAI X, et al Maximal likelihood correspondence estimation for face recognition across pose[J]. IEEE Transactions on Image Processing, 2014, 23 (10): 4587- 4600
doi: 10.1109/TIP.2014.2351265
|
|
|
| [4] |
WISKOTT L, FELLOUS J M, NORBERT K, et al Face recognition by elastic bunch graph matching[J]. IEEE Transactions on Pattern Analysis and Machine Intelligence, 1996, 19 (7): 456- 463
|
|
|
| [5] |
BISWASS S, AGGARWAL G, FLYNN P J, et al Pose-robust recognition of low-resolution face images[J]. IEEE Transactions on Pattern Analysis and Machine Intelligence, 2013, 35 (12): 3037- 3049
doi: 10.1109/TPAMI.2013.68
|
|
|
| [6] |
CHEN D, CAO X, WEN F, et al. Blessing of dimensionality: high-dimensional feature and its efficient compression for face verification [C]// IEEE Conference on Computer Vision and Pattern Recognition. Portland: IEEE, 2013: 3025-3032.
|
|
|
| [7] |
ZHANG Y, SHAO M, WONG E K, et al. Random faces guided sparse many-to-one encoder for pose-invariant face recognition [C]// Proceedings of the 2013 IEEE International Conference on Computer Vision. Sidney: IEEE, 2013: 2416-2423.
|
|
|
| [8] |
WU X, HE R, SUN Z, et al A light CNN for deep face representation with noisy labels[J]. IEEE Transactions on Information Forensics and Security, 2018, 13 (11): 2884- 2896
doi: 10.1109/TIFS.2018.2833032
|
|
|
| [9] |
SCHROFF F, KALENICHENKO D, PHILBIN J. Facenet: a unified embedding for face recognition and clustering [C]// IEEE Conference on Computer Vision and Pattern Recognition. Boston: IEEE, 2015: 815-823.
|
|
|
| [10] |
PRABHU U, HEO J, SAVVIDES M Unconstrained pose-invariant face recognition using 3D generic elastic models[J]. IEEE Transactions on Pattern Analysis and Machine Intelligence, 2011, 33 (10): 1952- 1961
doi: 10.1109/TPAMI.2011.123
|
|
|
| [11] |
BLANZ V, VETTER T. A morphable model for the synthesis of 3D faces [C]// Proceedings of the 26th Annual Conference on Computer Graphics and Interactive Techniques. Los Angeles: ACM, 1999: 187-194.
|
|
|
| [12] |
ALDRIAN O, SMITH W A P, et al Inverse] rendering of faces with a 3D morphable model[J]. IEEE Transactions on Pattern Analysis and Machine Intelligence, 2013, 35 (5): 1080- 1093
doi: 10.1109/TPAMI.2012.206
|
|
|
| [13] |
HUANG R, SHU Z, TIANYU L, et al. Beyond face rotation: global and local perception GAN for photorealistic and identity preserving frontal view synthesis [C]// IEEE International Conference on Computer Vision. Venice: IEEE, 2017: 2439-2448.
|
|
|
| [14] |
ZHAO J, CHENG Y, XIU Y, et al. Towards pose invariant face recognition in the wild [C]// IEEE Conference on Computer Vision and Pattern Recognition. Salt Lake City: IEEE, 2018: 2207-2216.
|
|
|
| [15] |
YIN X, YU X, SOHN K, et al. Towards large-pose face frontalization in the wild [C]// IEEE International Conference on Computer Vision. Venice: IEEE, 2017: 3990-3999.
|
|
|
| [16] |
ZHU X, LEI Z, YAN J J, et al. High-fidelity pose and expression normalization for face recognition in the wild [C]// IEEE Conference on Computer Vision and Pattern Recognition. Boston: IEEE, 2015: 787-796.
|
|
|
| [17] |
SHEN Y J, LUO P, YAN J J, et al. FaceID-GAN: learning a symmetry three-player GAN for identity-preserving face synthesis [C]// IEEE Conference on Computer Vision and Pattern Recognition. Salt Lake City: IEEE, 2018: 821-830.
|
|
|
| [18] |
BERTHELOT D, SCHUMM T, METZ L. BEGAN: boundary equilibrium generative adversarial networks [EB/OL]. (2017-05-31). https://arxiv.org/pdf/1703.10717.pdf.
|
|
|
| [19] |
ISOLA P, ZHU J Y, ZHOU T, et al. Image-to-image translation with conditional adversarial networks [C]// IEEE Conference on Computer Vision and Pattern Recognition. Honolulu: IEEE, 2017: 5967–5976.
|
|
|
| [20] |
ARJOVSKY M, CHINTALA S, BOTTOU L. Wasserstein GAN [EB/OL]. (2017-12-06). https://arxiv.org/pdf/1701.07875.pdf.
|
|
|
| [21] |
HASSNER T, HAREL S, PAZ, et al. Effective face frontalization in unconstrained images [C]// IEEE Conference on Computer Vision and Pattern Recognition. Boston: IEEE, 2015: 787-796.
|
|
|
|
Viewed |
|
|
|
Full text
|
|
|
|
|
Abstract
|
|
|
|
|
Cited |
|
|
|
|
| |
Shared |
|
|
|
|
| |
Discussed |
|
|
|
|
