Please wait a minute...
浙江大学学报(工学版)
浙江大学学报(工学版)
计算机技术、电子通信技术     
光流模值估计的微表情捕捉
姜波, 解仑, 刘欣, 韩晶, 王志良
北京科技大学 计算机与通信工程学院,北京 100083
Micro-expression spotting using optical flow magnitude estimation
JIANG Bo, XIE Lun, LIU Xin, HAN Jing, WANG Zhi-liang
School of Computer and Communication Engineering, University of Science and Technology Beijing, Beijing 100083, China
 全文: PDF(3965 KB)   HTML
摘要：

采用力的加速度参量展开描述人脸表情的变化过程,直接反映变化速度,从而有效捕捉表情序列中由不完全肌肉运动所引起的微表情关键帧.利用Horn-Schunck (H-S)光流法对连续运动的人脸图像序列提取运动目标的运动特征；通过光学应变张量算法,结合运动特征中的光流速度估计,推导出加速度参量；利用全局阈值算法对加速度模值和速度与张量模值作分类、比较,实现微表情图像序列关键帧的提取.采用Oulu大学SMIC微表情数据库中16个实验对象的88个微表情片段作为实验样本,平均正确识别率可达80.7%,比仅利用光学张量算法的正确识别率高12.5%.实验结果表明,所提出的加速度参量对微表情提取更具有效性.
Abstract:
Facial expression change process could be described by the acceleration parameter of force to reflect the speed of their change more directly, which could effectively capture the micro-expression key frame caused by incomplete muscle movements from the facial image sequence. The motion features of moving target were extracted by Horn-Schunck (H-S) optical flow from continuous facial image sequences. Then the acceleration parameter was deduced by the optical strain tensor and the optical flow velocity estimation of motion features. The optical flow magnitude from acceleration, speed and tensor was analyzed by the global threshold algorithm to achieve the key frame extraction in micro-expressions image sequence. The testing was computed on dataset SMIC from Oulu University including 16 subjects and 88 sequences of micro-expressions. The average accuracy of spotting is 80.7%, which is 12.5% higher than that of only using tensor algorithm. Experiment results show that the proposed acceleration parameter is more effective for micro-expression spotting.
出版日期: 2017-03-01
CLC:  TP 751.1  
基金资助:

国家自然科学基金资助项目(61672093,61432004)；国家重点研发计划课题(2016YEB1001404)；北京市自然科学基金青年资助项目（4164091）
通讯作者: 解仑,男,教授,博导.ORCID: 0000-0001-8064-6159.     E-mail: xielun@ustb.edu.cn
作者简介: 姜波（1990—）,女,硕士生,从事服务机器人、图像处理研究. ORCID:0000-0002-8555-7283. E-mail:364792077@qq.com
服务　　
把本文推荐给朋友
加入引用管理器
E-mail Alert
作者相关文章　　

引用本文:

姜波, 解仑, 刘欣, 韩晶, 王志良. 光流模值估计的微表情捕捉[J]. 浙江大学学报(工学版), 10.3785/j.issn.1008-973X.2017.03.020.

JIANG Bo, XIE Lun, LIU Xin, HAN Jing, WANG Zhi-liang. Micro-expression spotting using optical flow magnitude estimation. JOURNAL OF ZHEJIANG UNIVERSITY (ENGINEERING SCIENCE), 10.3785/j.issn.1008-973X.2017.03.020.

［1］ EKMAN P, FRIESEN W V. Nonverbal leakage and clues to deception ［J］. Psychiatry-interpersonal and Biological Processes, 1969, 32(1): 88-106.
［2］ SONG Y, MORENCY L P, DAVIS R. Learning a sparse codebook of facial and body microexpressions for emotion recognition ［C］ ∥ Proceedings of ICMI 2013. New York: ACM, 2013: 237-244.
［3］ LIONG S T, SEE J, PHAN C W, et al. Subtle Expression recognition using optical strain weighted features ［C］ ∥ Computer VisionACCV 2014 Workshops. Singapore: Springer, 2014: 644-657.
［4］ POLISOVSKY S, KAMEDA Y, OHTA Y. Facial micro-expressions recognition using high speed camera and 3D-gradients descriptor ［C］ ∥ Proceedings of 3rd International Conference on ICDP 2009. London: IEEE, 2009: 1-6.
［5］ 吴奇,申寻兵,傅小兰.微表情研究及其应用［J］.心理科学进展,2010, 18(9): 1359-1368.
WU Qi, SHEN Xun-bing, FU Xiao-lan, Micro-expression and its applications ［J］. Advances in Psychological Science, 2010, 18(9): 1359-1368.
［6］ WANG S, YAN W J, ZHAO G, et al. Micro-expression recognition using robust principal component analysis and local spatiotemporal directional features ［C］ ∥ Computer Vision-ECCV 2014 Workshops. Switzerland: Springer, 2014: 325-338.
［7］ GUO Y, TIAN Y, GAO X, et al. Micro-expression recognition based on local binary patterns from three orthogonal planes and nearest neighbor method ［C］ ∥ 2014 International Joint Conference on Neural Networks (IJCNN). Beijing: IEEE, 2014: 3473-3479.
［8］ 梁静,颜文靖,吴奇,等.微表情研究的进展与展望［J］.中国科学基金,2013, 27(2): 75-78.
LIANG Jing, YAN Wen-jing, WU Qi, et al. Recent advances and future trends in microexpresion research ［J］. Science Foundation in China, 2013, 27(2): 75-78.
［9］ FAN X J, TJAHJADI T. A spatial-temporal framework based on histogram of gradients and optical flow for facial expression recognition in video sequences ［J］. Pattern Recognition, 2015, 48(11): 3407-3416.
［10］ SHREVE M, BRIZZI J, FEFILATYEV S, et al. Automatic expression spotting in videos ［J］. Image and Vision Computing, 2014, 32(8): 476-486.
［11］ SHREVE M, GODAVARTHY S, MANOHAR V, et al. Towards macro-and micro-expression spotting in video using strain patterns ［C］ ∥ Workshop on WACV 2009. Snowbird: IEEE, 2009: 1-6.
［12］ SHREVE M, GODAVARTHY S, GOLDGOF D, et al. Macro-and micro-expression spotting in long videos using spatio-temporal strain ［C］ ∥ IEEE International Conference and Workshops on FG 2011. Santa Barbara: IEEE, 2011: 51-56.
［13］ LI X, PFISTER T, HUANG X, et al. A spontaneous micro-expression database: inducement, collection and baseline ［C］ ∥ 2013 10th IEEE International Conference and Workshops on FG 2013. Shanghai: IEEE, 2013: 1-6.
［14］ PFISTER T, LI X, ZHAO G, et al. Recognising spontaneous facial micro-expressions ［C］ ∥ IEEE International Conference on ICCV 2011. Barcelona: IEEE, 2011: 1449-1456.
［15］ BARRON J L, FLEET D J, BEAUCHEMIN S S. Performance of optical flow techniques ［J］. International Journal of Computer Vision, 1994, 12(1): 43-77.
［16］ CHEN J S, YANG C, DENG Y, et al. Exploring facial asymmetry using optical flow ［J］. IEEE Signal Processing Letters, 2014, 21(7): 792-795.
［17］ HORN B K, SCHUNCK B G. Determining optical flow ［J］. Artificial Intelligence, 1981, 17(1-3): 185-203.
［18］ KOMORKIEWICZ M, KRYJAK T, GORGON M. Efficient hardware implementation of the horn-schunck algorithm for high-resolution real-time dense optical flow sensor ［J］. Sensors, 2014, 14(2): 2860-2891.
[1] 余煇,柴登峰. 基于长方形点过程的遥感图像汽车提取[J]. 浙江大学学报(工学版), 2019, 53(9): 1741-1748.
[2] 赫贵然,李奇,冯华君,徐之海,陈跃庭. 基于CNN特征提取的双焦相机连续数字变焦[J]. 浙江大学学报(工学版), 2019, 53(6): 1182-1189.
[3] 厉小润, 朱洁尔, 王晶, 赵辽英. 组合核支持向量机高光谱图像分类[J]. J4, 2013, 47(8): 1403-1410.
[4] 马利刚, 张乐平, 邓劲松, 汪雅婕, 王珂. 资源一号“02C”遥感影像土地利用分类[J]. J4, 2013, 47(8): 1508-1516.
[5] 赖小波,朱世强,方纯洁. 一种复杂背景图像三维重建算法及其医学应用[J]. J4, 2012, 46(11): 2061-2067.
[6] 车红昆, 吕福在, 项占琴. 基于顺序向前浮动搜索时频优选特征的缺陷识别[J]. J4, 2011, 45(12): 2235-2239.
[7] 韩凝, 张秀英, 王小明, 陈利苏, 王珂. 高分辨率影像香榧树分布信息提取[J]. J4, 2010, 44(3): 420-425.