Please wait a minute...
浙江大学学报(工学版)
JOURNAL OF ZHEJIANG UNIVERSITY (ENGINEERING SCIENCE)  2017, Vol. 51 Issue (12): 2299-2310    DOI: 10.3785/j.issn.1008-973X.2017.12.001
Computerand Communication Technology     
Portrait skin beautification technology using multiple feature masks
LU Xiao-hui1, WANG Jin1, LU Guo-dong1, ZHANG Dong-liang2
1. State Key Laboratory of Fluid Power Transmission and Control, Zhejiang University, Hangzhou 310027, China;
2. International Design Institute, Zhejiang University, Hangzhou 310027, China
Download:   PDF(23024KB) HTML
Export: BibTeX | EndNote (RIS)      

Abstract  

Portrait skin beautification technology using multiple feature masks was proposed to solve detail blurring and character losing problem in existing portrait skin beautification technologies. The first mask was called facial apparent blemish feature mask. This mask was made up of high pass and superposition of hard light, which was used for removing apparent facial blemishes beforehand. The second mask called portrait background feature mask was made up of skin color detection. The third mask called facial key feature mask was made up of Face++ algorithm. The second mask along with the third mask can help to solve the problem that filtering may blur facial key features and portrait background features. The fourth mask called stereo vision control mask was made up of high pass. This mask can restore facial stereo vision features after portrait skin beautification. Experimental results show that this method could wipe off facial blemishes by setting a smaller portrait skin beautification coefficient and retain realistic features, such as facial key features, as well as facial stereoscopic effect with a larger portrait skin beautification coefficient. Therefore, the contradictory problem that portrait skin beautification should either keep the clarity of details or keep the degree of removing facial blemishes was solved. This technology can achieve the purpose of facial beautification.

Received: 27 October 2016      Published: 22 November 2017
CLC:  TP391.41  
Service
E-mail this article
Add to my bookshelf
Add to citation manager
E-mail Alert
RSS
Articles by authors
Cite this article:

LU Xiao-hui, WANG Jin, LU Guo-dong, ZHANG Dong-liang. Portrait skin beautification technology using multiple feature masks. JOURNAL OF ZHEJIANG UNIVERSITY (ENGINEERING SCIENCE), 2017, 51(12): 2299-2310.

URL:

http://www.zjujournals.com/eng/10.3785/j.issn.1008-973X.2017.12.001     OR     http://www.zjujournals.com/eng/Y2017/V51/I12/2299


采用多重特征蒙板的人像皮肤美化技术

为了解决现有人像皮肤美化技术中出现的细节模糊以及特征丢失问题，提出一种基于构建多重特征蒙板的人像皮肤美化技术.第一重蒙板为利用高反差保留和强光迭加构建的面部显著瑕疵特征蒙板，基于该蒙板可预先去除人像面部的显著瑕疵；第二重蒙板为利用肤色检测构建的人像背景特征蒙板；第三重蒙板为利用Face++算法构建的面部关键区域特征蒙板；第二、三重蒙板的结合使用可以解决人像的背景特征及面部关键区域特征由于后续滤波处理而变模糊的问题；第四重蒙板为利用高反差保留构建的面部立体视觉控制蒙板，该蒙板可在之前的皮肤美化处理基础上还原人像的面部立体视觉特征.实验结果表明，该技术使用较小的皮肤美化系数即可去除瑕疵，并能在皮肤美化系数较大的情况下保留人像关键区域以及立体细节等真实感信息特征，从而解决人像皮肤美化过程中细节清晰度与瑕疵去除程度无法兼得的矛盾，最终达到人像美化的目的.

[1] CHEN D, REN S, WEI Y, et al. Joint cascade face detection and alignment[C]//ECCV. Zurich:Springer, 2014:109-122.
[2] ZHANG K, ZHANG Z, LI Z, et al. Joint face detection and alignment using multi-task cascaded convolutional networks[J]. IEEE Signal Processing Letters, 2016,23(10):1499-1503.
[3] RANJAN R, PATEL V M, CHELLAPPA R. HyperFace:a deep multi-task learning framework for face detection, landmark localization, pose estimation, and gender recognition[J]. IEEE Transactions On Pattern Analysis and Machine Intelligence, 2016(3):1-10.
[4] ZHOU E, FAN H, CAO Z, et al. Extensive facial landmark localization with coarse-to-fine convolutional network cascade[C]//IEEE International Conference on Computer Vision Workshops. Moscow:IEEE, 2013:386-391.
[5] LIN P H, CHEN B H, CHENG F C, et al. A morphological mean filter for impulse noise removal[J]. Journal of Display Technology, 2016, 12(4):344-350.
[6] KULKARNI R K, LAHOTI C B, MEHER S. Impulse denoising using improved progressive switching median filter[C]//ICWET'10 International Conference and Workshop on Emerging Trends in Technology. New York:ACM, 2010:586-590.
[7] BAEK J, JACOBS D E. Accelerating spatially varying Gaussian filters[J]. Acm Transactions on Graphics, 2010, 29(6):81-95.
[8] ADAMS A, GELFAND N, DOLSON J, et al. Gaussian KD-trees for fast high-dimensional filtering[J]. ACM Transactions on Graphics, 2009, 28(3):1-12.
[9] CARLOS A B M. Filtering the shadows from poorlyilluminated photos[J]. ACM Symposium on Applied Computing, 2010:1599-1600.
[10] FUJITA S, FUKUSHIMA N, KIMURA M, et al. Randomized redundant DCT:efficient denoising byusing random subsampling of DCT patches[C]//SIGGRAPH Asia 2015 Technical Briefs. New York:ACM, 2015:7.
[11] LAPARRA V, GUTIERREZ J, CAMPS V G, et al. Image denoising with kernels based on natural image relations[J]. Journal of Machine Learning Research, 2010, 11(1):873-903.
[12] DABOV K, FOI A, KATKOVNIK V, et al. Image denoising by sparse 3D transform-domain collaborative filtering[J]. IEEE Transactions on Image Processing, 2007, 16(8):2080-2095.
[13] SUGIMOTO K, KAMATA S I. Compressive bilateral filtering[J]. IEEE Transactions on Image Processing A Publication of the IEEE Signal Processing Society, 2015, 24(11):3357-3369.
[14] LEE C, SCHRAMM M T, BOUTIN M, et al. Analgorithm for automatic skin smoothing in digitalportraits[J]. IEEE International Conference on Image Processing Pages, 2009:3149-3152.
[15] LIANG L, JIN L, LI X. Facial skin beautificationusing adaptive region-aware masks[J]. IEEE Transactions on Cybernetics, 2014, 44(12):2600-2612.
[16] BRAND M, PLETSCHER P. A conditional random field for automatic photo editing[C]//IEEE Conference on Computer Vision and Pattern Recognition.Ancholage:IEEE, 2008:1-7.
[17] YANG Y, ZHAO H, YOU L, et al. Semantic portrait color transfer with internet images[J]. Multimedia Tools and Applications, 2017, 76(1):523-541.
[18] LIAO Q, JIN X, ZENG W. Enhancing the symmetry and proportion of 3D face geometry[J]. IEEE Transactions on Visualization and Computer Graphics, 2012,18(10):1704-1716.
[19] GUO D, SIM T. Digital face makeup by example[C]//IEEE Conference on Computer Vision and Pattern Recognition. Miami:IEEE, 2009:73-79.
[20] GUO G. Digital anti-aging in face images[C]//IEEE International Conference on Computer Vision. Barcelona:IEEE, 2011:6-13.
[21] GONZALEZ R C, WOODS R E. Digital image processing[M]. 3rd ed. New Jersey:Prentice Hall, 2002:305-308.
[22] DADGOSTAR F, SARRAFZADEH A. An adaptive real-time skin detector based on Hue thresholding:a comparison on two motion tracking methods[J]. Pattern Recognition Letters, 2006, 27(12):1342-1352.
[23] MARKUS W. Caltech:frontal face dataset[EB/OL].[1999-01-01]. http://www.vision.caltech.edu/html-files/.
[24] FEI. Face database[EB/OL].[2006-01-01]. http://fei.edu.br/~cet/facedatabase.html.
[25] KAE A, SOHN K, LEE H, et al. Augmenting CRFs with Boltzmann machine shape priors for image labeling[C]//IEEE Conference on Computer Vision and Pattern Recognition. Portland:IEEE, 2013:2019-2026.
[1] ZHENG Zhou, ZHANG Xue-chang, ZHENG Si-ming, SHI Yue-ding. Liver segmentation in CT images based on region-growing and unified level set method[J]. JOURNAL OF ZHEJIANG UNIVERSITY (ENGINEERING SCIENCE), 2018, 52(12): 2382-2396.
[2] CA Jin-Hui, ZHANG Guang-Xin, CAI Hui. Theory and application of connectivity maskbased reconstruction
opening operator[J]. JOURNAL OF ZHEJIANG UNIVERSITY (ENGINEERING SCIENCE), 2010, 44(4): 675-680.
[3] WANG Xuan-Yin, LIANG Dong-Tai. Surface defect detection based on multivariate image analysis[J]. JOURNAL OF ZHEJIANG UNIVERSITY (ENGINEERING SCIENCE), 2010, 44(3): 448-452.
[4] SUN Zhi-Hai, KONG Mo-Ceng, SHU Shan-An. Improved algorithm of subtractive clustering for object location  in video sequences[J]. JOURNAL OF ZHEJIANG UNIVERSITY (ENGINEERING SCIENCE), 2010, 44(3): 458-462.
[5] MAO Feng, ZHANG Shu-Wei, HUANG Chang-Lin. Inspection of foreign substances in mould using image scattergrams and multi|resolution analysis[J]. JOURNAL OF ZHEJIANG UNIVERSITY (ENGINEERING SCIENCE), 2009, 43(10): 1749-1756.
[6] FAN Xiang, XIA Shun-ren. Feature based automatic stitching of microscopic images[J]. JOURNAL OF ZHEJIANG UNIVERSITY (ENGINEERING SCIENCE), 2009, 43(7): 1182-1186.
[7] CHEN Cheng, PENG Huo-Ting, XIAO Dun. Video foreground segmentation with camera movement[J]. JOURNAL OF ZHEJIANG UNIVERSITY (ENGINEERING SCIENCE), 2009, 43(6): 975-977.
[8] ZHANG Dong-Mei, LIU Li-Gang. Angle-filtering based smoothing algorithm for planar graphs[J]. JOURNAL OF ZHEJIANG UNIVERSITY (ENGINEERING SCIENCE), 2009, 43(6): 1042-1046.