Please wait a minute...
浙江大学学报(工学版)  2020, Vol. 54 Issue (10): 1936-1944    DOI: 10.3785/j.issn.1008-973X.2020.10.010
计算机技术     
摄像机主动位姿协同的人脸正视图像获取方法
王雯涛(),李佳田*(),吴华静,高鹏,阿晓荟,朱志浩
昆明理工大学 国土资源工程学院,云南 昆明 650093
Camera active pose cooperation for obtaining face frontal images
Wen-tao WANG(),Jia-tian LI*(),Hua-jing WU,Peng GAO,Xiao-hui A,Zhi-hao ZHU
Faculty of Land Resources Engineering, Kunming University of Science and Technology, Kunming 650093, China
 全文: PDF(1751 KB)   HTML
摘要:

为了主动调整摄像机的位置姿态以在物方空间降低由人脸旋转带来的影响,提出人脸-摄像机主动位姿协同方法. 利用工业摄像机与同步带类运动控制器搭建位姿协同环境,根据单像空间后方交会方法求解运动人脸姿态角和相对位置,计算人脸、摄像机和运动控制器各坐标系之间的映射关系,摄像机运动由相邻采样时刻摄像机相对位置和人脸姿态角计算得出的摄像机协同位姿控制,主动、实时地获取人脸正视图像. 实验表明,运动轴上的摄像机点位误差小于10 mm,取得的人脸正视图像提高了人脸纠正的精准度与鲁棒性.

关键词: 人脸纠正人脸正视图像摄影测量人脸姿态角位姿协同运动控制器    
Abstract:

A face-to-camera active pose cooperation method was proposed in order to reduce the impact of face rotation in object space by actively adjusting the position and attitude of the camera. Environment of pose cooperation was constructed by using an industrial camera and a synchronous belt-like motion controller. The angle of a moving face pose and the relative position between the camera and the face were solved by a single image space resection. The mapping relationship between the coordinate systems of the face, the camera and the motion controller was calculated. The camera motion was controlled by the camera's coordinated pose control calculated from the relative position of the camera and the face attitude angle at the adjacent sampling time, and the face frontal image was acquired actively and real-time. The experimental results showed that the camera point position error on the motion axis was less than 10 mm, and the extracted frontal image improved the accuracy and robustness of face correction.

Key words: correction of face images    face frontal images    photogrammetry    face pose angle    pose cooperation    motion controller
收稿日期: 2019-08-28 出版日期: 2020-10-28
CLC:  TP 302  
基金资助: 国家自然科学基金资助项目(41561082)
通讯作者: 李佳田     E-mail: 413274795@qq.com;ljtwcx@163.com
作者简介: 王雯涛(1995—),男,硕士生,从事摄影测量的研究. orcid.org/0000-0002-9380-1485. E-mail: 413274795@qq.com
服务  
把本文推荐给朋友
加入引用管理器
E-mail Alert
作者相关文章  
王雯涛
李佳田
吴华静
高鹏
阿晓荟
朱志浩

引用本文:

王雯涛,李佳田,吴华静,高鹏,阿晓荟,朱志浩. 摄像机主动位姿协同的人脸正视图像获取方法[J]. 浙江大学学报(工学版), 2020, 54(10): 1936-1944.

Wen-tao WANG,Jia-tian LI,Hua-jing WU,Peng GAO,Xiao-hui A,Zhi-hao ZHU. Camera active pose cooperation for obtaining face frontal images. Journal of ZheJiang University (Engineering Science), 2020, 54(10): 1936-1944.

链接本文:

http://www.zjujournals.com/eng/CN/10.3785/j.issn.1008-973X.2020.10.010        http://www.zjujournals.com/eng/CN/Y2020/V54/I10/1936

图 1  世界坐标系到摄像机坐标系转换示意图
图 2  摄像机运动控制器
图 3  初始采样时刻各个坐标系的位置关系图
图 4  人脸与摄像机位置关系的示意图
特征点 XW YW ZW
左眼左角点 ?220.0 165.0 ?130.0
嘴巴左角点 ?145.0 ?145.0 ?120.0
右眼右角点 220.0 165.0 ?130.0
嘴巴右角点 145.0 ?145.0 ?120.0
下巴 0.0 ?325.0 ?60.0
鼻尖 0.0 0.0 0.0
表 1  标准人脸模型特征点的三维坐标
ti 人脸姿态 φ φ' ω ω' κ κ'
t1 0.0306 0.0307 ?0.2621 ?0.2613 3.1195 3.1200
t2 0.0308 0.0305 ?0.3072 ?0.3064 3.1186 3.1179
t3 0.0319 0.0311 ?0.8121 ?0.8112 3.1178 3.1173
t4 0.0305 0.0312 ?1.1629 ?1.1638 3.1260 3.1256
t5 0.0316 0.0315 ?0.7629 ?0.7628 2.8835 2.8829
t6 0.5354 0.5353 ?0.2771 ?0.2770 3.1225 3.1217
t7 ?0.3146 ?0.3148 ?0.2551 ?0.2555 3.1135 3.1138
表 2  不同采样时刻人脸姿态
ti XS ${X_S^{'}}$ ${Y_S}$ ${Y_S^{'}}$ ZS ${Z_S^{'}}$
t1 10.230 11.610 10.511 8.261 2449.88 2442.07
t2 80.299 78.629 21.900 18.840 2449.88 2450.25
t3 53.597 52.457 46.397 44.217 2449.87 2451.95
t4 41.106 39.416 35.589 39.119 2449.07 2456.19
t5 109.310 106.660 42.763 47.613 2450.17 2455.20
t6 14.464 16.014 65.387 61.317 2449.77 2452.04
t7 4.230 4.140 ?33.453 ?37.803 2449.97 2444.28
表 3  不同采样时刻摄像机相对位置
位置 X Y Z φ ω κ
P1 107.33 82.02 2452.59 0.061 ?0.29 3.124
P2 115.85 84.06 2457.74 0.058 ?0.281 3.121
P3 123.13 84.02 2459.14 0.064 ?0.317 3.054
P4 128.66 83.86 2462.48 0.072 ?0.371 3.078
P5 131.64 83.93 2464.75 0.061 ?0.359 3.049
P6 135.84 82.31 2467.73 0.072 ?0.395 3.061
P7 149.44 83.45 2468.52 0.073 ?0.402 3.13
P8 161.56 84.75 2469.77 0.069 ?0.456 3.044
P9 161.87 83.13 2470.87 0.061 ?0.467 3.127
P10 168.81 82.54 2479.31 0.072 0.005 3.073
表 4  人脸姿态及摄像机相对位置
位置变化 x y u
Tx/mm P/PPU Ty/mm P/PPU Rω/rad P/PPU
P1P2 5.809 1452.25 0.049 12.25 0.0837 59.757
P2P3 7.168 1792.00 ?0.018 ?4.50 0.0361 48.966
P3P4 10.735 2683.75 0.142 35.50 0.0697 52.631
P4P5 4.731 1182.75 0.099 24.75 0.0588 49.678
P6P7 7.847 1961.75 0.054 13.50 0.0751 68.308
P7P8 13.574 3393.5 0.098 24.50 0.0772 58.230
P8P9 15.684 3921 0.093 23.25 0.0315 29.013
P9P10 10.944 2736 0.026 6.50 0.0352 38.888
表 5  摄像机位置变化量与步进电机脉冲输入量
图 5  摄像机在x、y轴的位置误差曲线
ti Ec/mm Ep/mm Em/mm
t1 ?7.58 ?6.445 ?1.135
t2 3.16 2.740 0.42
t3 8.14 7.308 0.832
t4 6.92 6.181 0.739
t5 ?6.17 ?5.615 ?0.555
t6 4.37 3.852 0.518
t7 6.46 5.713 0.747
t8 7.78 6.990 0.790
t9 ?2.57 ?2.370 ?0.200
t10 2.66 2.501 0.159
表 6  摄像机位置x轴误差分析
图 6  不同平移速度下的摄像机y轴位置误差分析
图 7  不同角速度下的摄像机u轴位置误差分析
图 8  人脸图像与正视人脸图像的对齐对比分析
图 9  不同偏转角度下人脸图像与正视图像的对齐对比分析
图 10  不同偏转角度对正视图像人脸对齐的影响
1 CAO X, WEI Y, WEN F, et al Face alignment by explicit shape regression[J]. International Journal of Computer Vision, 2014, 107 (2): 177- 190
doi: 10.1007/s11263-013-0667-3
2 ZHANG K, ZHANG Z, LI Z, et al Joint face detection and alignment using multitask cascaded convolutional networks[J]. IEEE Signal Processing Letters, 2016, 23 (10): 1499- 1503
doi: 10.1109/LSP.2016.2603342
3 CHEN D, REN S, WEI Y, et al. Joint cascade face detection and alignment [C] // European Conference on Computer Vision. Switzerland: Springer, 2014: 109-122.
4 LIAO S, JAIN A K, LI S Z Partial face recognition: alignment-free approach[J]. IEEE Transactions on Pattern Analysis and Machine Intelligence, 2013, 35 (5): 1193- 1205
doi: 10.1109/TPAMI.2012.191
5 MCNEILL G, VIJAYAKUMAR S. Hierarchical procrustes matching for shape retrieval [C] // 2006 IEEE Computer Society Conference on Computer Vision and Pattern Recognition. New York: IEEE, 2006: 885-894.
6 LIN C, ZHANG T, LV C, et al. Improved explicit shape regression face alignment algorithm[C] // 2018 Chinese Control and Decision Conference. Shenyang: IEEE, 2018: 1166-1169.
7 DANTONE M, GALL J, FANELLI G, et al. Realtime facial feature detection using conditional regression forests [C] // IEEE Conference on Computer Vision and Pattern Recognition. Providence: IEEE, 2012: 2578-2585.
8 LI Q, SUN Z, HE R. Fast multi-view face alignment via multi-task auto-encoders [C] // 2017 IEEE International Joint Conference on Biometrics. Denver: IEEE, 2017: 538-545.
9 TZIMIROPOULOS G. Project-out cascaded regression with an application to face alignment [C] // IEEE Conference on Computer Vision and Pattern Recognition. Boston: IEEE, 2015: 3659-3667.
10 XIONG X, TORRE F D L. Supervised descent method and its applications to face alignment [C] // IEEE Conference on Computer Vision and Pattern Recognition. Portland: IEEE, 2013: 532-539.
11 LIU X Y An empirical study of boosting methods on severely imbalanced data[J]. Applied Mechanics and Materials, 2014, 513 (2): 2510- 2513
12 DING C, SONG B, AKSHAY M, et al Collaborative sensing in a distributed PTZ camera network[J]. IEEE Transactions on Image Processing, 2012, 21 (7): 3282- 3295
doi: 10.1109/TIP.2012.2188806
13 BERNARDIN K, CAMP F V D, STIEFELHAGEN R. Automatic person detection and tracking using fuzzy controlled active cameras [C] // IEEE Conference on Computer Vision and Pattern Recognition. Minneapolis: IEEE, 2007: 1-8.
14 SCOTTI G, MARCENARO L, COELHO C, et al Dual camera intelligent sensor for high definition 360 degrees surveillance[J]. IEE Proceedings of Vision Image and Signal Processing, 2005, 152 (2): 250- 257
doi: 10.1049/ip-vis:20041302
15 KIM S, CHOI J. Face recognition method based on fixed and PTZ camera control for moving humans [C] // 2019 11th International Conference on Ubiquitous and Future Networks. Zagreb: IEEE, 2019: 561-563.
16 张文靖. 人脸-摄像机主动位姿协同[D]. 昆明: 昆明理工大学, 2019.
ZHANG Wen-jing. Face-camera active pose cooperation [D]. Kunming: Kunming University of Science and Technology, 2019.
17 曾慧, 穆志纯, 袁立 基于三维模型的人脸姿态估计方法[J]. 计算机工程, 2011, 37 (9): 1- 3
ZENG Hui, MU Zhi-chun, YUAN li Face pose estimation method based on 3D model[J]. Computer Engineering, 2011, 37 (9): 1- 3
18 LEE D, YANG M H, OH S. Fast and accurate head pose estimation via random projection forests [C] // IEEE Proceedings of the International Conference on Computer Vision. Santiago: IEEE, 2015: 1958-1966.
19 MURPHY C E, TRIVEDI M M Head pose estimation in computer: a survey[J]. IEEE Transactions on Pattern Analysis and Machine Intelligence, 2009, 31 (4): 607- 626
doi: 10.1109/TPAMI.2008.106
20 王世宙. 双目立体测量系统中数控平台的设计与标定 [D]. 南京: 南京航空航天大学, 2011.
WANG Shi-zhou. The design and calibration of the NC platform in binocular stereo measurement [D]. Nanjing: Nanjing University of Aeronautics and Astronautics, 2011.
21 邵杰, 张爱武, 王书民, 等 交替误差补偿的无人飞艇位姿估计[J]. 计算机辅助设计与图形学学报, 2013, 25 (9): 1361- 1367
SHAO Jie, ZHANG Ai-wu, WANG Shu-min, et al Pose estimation of unmanned airship of alternating error compensation[J]. Journal of Computer-Aided Design and Computer Graphics, 2013, 25 (9): 1361- 1367
22 龚健雅, 季顺平 从摄影测量到计算机视觉[J]. 武汉大学学报: 信息科学版, 2017, 42 (11): 1518- 1522
GONG Jian-ya, JI Shun-ping Form photogrammetry to computer vision[J]. Geomatics and Information Science of Wuhan University, 2017, 42 (11): 1518- 1522
23 龚健雅, 季顺平 摄影测量与深度学习[J]. 测绘学报, 2018, 47 (6): 693- 704
GONG Jian-ya, JI Shun-ping Photogrammetry and deep learning[J]. Acta Geodaetica et Cartographica Sinica, 2018, 47 (6): 693- 704
24 DROUARD V, HORAUD R, DELEFORGE A, et al Robust head-pose estimation based on partially-latent mixture of linear regressions[J]. IEEE Transactions on Image Processing, 2017, 26 (3): 1428- 1440
doi: 10.1109/TIP.2017.2654165
25 李成龙, 钟凡, 马昕, 等 基于卡尔曼滤波和随机回归森林的实时头部姿态估计[J]. 计算机辅助设计与图形学学报, 2017, 29 (12): 2309- 2316
LI Cheng-long, ZHONG Fan, MA Xin, et al Real-time head pose estimation based on Kalman and random regression forest[J]. Journal of Computer-Aided Design and Computer Graphics, 2017, 29 (12): 2309- 2316
doi: 10.3724/SP.J.1089.2017.16521
[1] 王琪,谢坤,马严,丛群. 基于日志统计特征的DNS隧道检测[J]. 浙江大学学报(工学版), 2020, 54(9): 1753-1760.
[2] 曾煌尧,李丹丹,马严,丛群. 园区网风险账号评估方法[J]. 浙江大学学报(工学版), 2020, 54(9): 1761-1767.
[3] 华幸成,刘鹏. 基于动态任务迁移的近数据处理方法[J]. 浙江大学学报(工学版), 2019, 53(12): 2348-2356.
[4] 吕娜,刘创,陈柯帆,曹芳波. 考虑控制器故障的软件定义机载网络选举算法[J]. 浙江大学学报(工学版), 2019, 53(4): 785-793.
[5] 季长清,余胜,王宝凤,陶帅,汪祖民,王润方. 移动云计算环境下的双色反近邻查询算法[J]. 浙江大学学报(工学版), 2016, 50(7): 1330-1337.
[6] 于绩洋,刘鹏,华幸成,马骧,杨建义. 片上光电互连的多核系统仿真方法[J]. 浙江大学学报(工学版), 2015, 49(11): 2214-2222.
[7] 叶霞,辛愿,刘勇,刘鹏. 基于媒体数字信号处理器的流预取机制[J]. J4, 2014, 48(2): 268-278.
[8] 苏程, 俞伟斌, 倪广翼, 黄智才, 陶春辉, 章孝灿. 深水多波束测深侧扫声纳显控系统研究[J]. J4, 2013, 47(6): 934-943.
[9] 全励, 程爱莲, 潘赟, 丁勇, 严晓浪. 基于旁路通道的片上网络差别型服务实现方法[J]. J4, 2013, 47(6): 957-968.
[10] 张振, 李善平. 变频感知的处理器服务时间估算方法[J]. J4, 2012, 46(4): 725-733.
[11] 傅朝阳, 高济, 周尤明. 基于承诺的agent组织描述工具[J]. J4, 2011, 45(4): 627-636.
[12] 曹晓阳, 潘赟, 严晓浪, 宦若虹. 低面积-时间复杂度的离散余弦变换脉动结构[J]. J4, 2011, 45(4): 656-659.
[13] 黄雪维,张培勇,吕冬明,郑丹丹,严晓浪. 基于时延搜索的SRAM建立时间快速提取方法[J]. J4, 2011, 45(3): 445-450.
[14] 徐鸿明,孟建熠,严晓浪,葛海通. 基于高速缓存资源共享的TLB设计方法[J]. J4, 2011, 45(3): 462-466.
[15] 龚帅帅,吴晓波,孟建熠,丁永林. 基于历史链接关系的指令高速缓存低功耗方法[J]. J4, 2011, 45(3): 467-471.