| Computer Technology, Electronic Communications Technologies |
|
|
|
|
| Micro-expression spotting using optical flow magnitude estimation |
| JIANG Bo, XIE Lun, LIU Xin, HAN Jing, WANG Zhi-liang |
| School of Computer and Communication Engineering, University of Science and Technology Beijing, Beijing 100083, China |
|
|
|
Abstract Facial expression change process could be described by the acceleration parameter of force to reflect the speed of their change more directly, which could effectively capture the micro-expression key frame caused by incomplete muscle movements from the facial image sequence. The motion features of moving target were extracted by Horn-Schunck (H-S) optical flow from continuous facial image sequences. Then the acceleration parameter was deduced by the optical strain tensor and the optical flow velocity estimation of motion features. The optical flow magnitude from acceleration, speed and tensor was analyzed by the global threshold algorithm to achieve the key frame extraction in micro-expressions image sequence. The testing was computed on dataset SMIC from Oulu University including 16 subjects and 88 sequences of micro-expressions. The average accuracy of spotting is 80.7%, which is 12.5% higher than that of only using tensor algorithm. Experiment results show that the proposed acceleration parameter is more effective for micro-expression spotting.
|
|
Published: 01 March 2017
|
|
|
|
Cite this article:
JIANG Bo, XIE Lun, LIU Xin, HAN Jing, WANG Zhi-liang. Micro-expression spotting using optical flow magnitude estimation. JOURNAL OF ZHEJIANG UNIVERSITY (ENGINEERING SCIENCE), 2017, 51(3): 577-583.
|
光流模值估计的微表情捕捉
采用力的加速度参量展开描述人脸表情的变化过程,直接反映变化速度,从而有效捕捉表情序列中由不完全肌肉运动所引起的微表情关键帧.利用Horn-Schunck (H-S)光流法对连续运动的人脸图像序列提取运动目标的运动特征；通过光学应变张量算法,结合运动特征中的光流速度估计,推导出加速度参量；利用全局阈值算法对加速度模值和速度与张量模值作分类、比较,实现微表情图像序列关键帧的提取.采用Oulu大学SMIC微表情数据库中16个实验对象的88个微表情片段作为实验样本,平均正确识别率可达80.7%,比仅利用光学张量算法的正确识别率高12.5%.实验结果表明,所提出的加速度参量对微表情提取更具有效性.
|
|
| ［1］ EKMAN P, FRIESEN W V. Nonverbal leakage and clues to deception ［J］. Psychiatry-interpersonal and Biological Processes, 1969, 32(1): 88-106.
［2］ SONG Y, MORENCY L P, DAVIS R. Learning a sparse codebook of facial and body microexpressions for emotion recognition ［C］ ∥ Proceedings of ICMI 2013. New York: ACM, 2013: 237-244.
［3］ LIONG S T, SEE J, PHAN C W, et al. Subtle Expression recognition using optical strain weighted features ［C］ ∥ Computer VisionACCV 2014 Workshops. Singapore: Springer, 2014: 644-657.
［4］ POLISOVSKY S, KAMEDA Y, OHTA Y. Facial micro-expressions recognition using high speed camera and 3D-gradients descriptor ［C］ ∥ Proceedings of 3rd International Conference on ICDP 2009. London: IEEE, 2009: 1-6.
［5］ 吴奇,申寻兵,傅小兰.微表情研究及其应用［J］.心理科学进展,2010, 18(9): 1359-1368.
WU Qi, SHEN Xun-bing, FU Xiao-lan, Micro-expression and its applications ［J］. Advances in Psychological Science, 2010, 18(9): 1359-1368.
［6］ WANG S, YAN W J, ZHAO G, et al. Micro-expression recognition using robust principal component analysis and local spatiotemporal directional features ［C］ ∥ Computer Vision-ECCV 2014 Workshops. Switzerland: Springer, 2014: 325-338.
［7］ GUO Y, TIAN Y, GAO X, et al. Micro-expression recognition based on local binary patterns from three orthogonal planes and nearest neighbor method ［C］ ∥ 2014 International Joint Conference on Neural Networks (IJCNN). Beijing: IEEE, 2014: 3473-3479.
［8］ 梁静,颜文靖,吴奇,等.微表情研究的进展与展望［J］.中国科学基金,2013, 27(2): 75-78.
LIANG Jing, YAN Wen-jing, WU Qi, et al. Recent advances and future trends in microexpresion research ［J］. Science Foundation in China, 2013, 27(2): 75-78.
［9］ FAN X J, TJAHJADI T. A spatial-temporal framework based on histogram of gradients and optical flow for facial expression recognition in video sequences ［J］. Pattern Recognition, 2015, 48(11): 3407-3416.
［10］ SHREVE M, BRIZZI J, FEFILATYEV S, et al. Automatic expression spotting in videos ［J］. Image and Vision Computing, 2014, 32(8): 476-486.
［11］ SHREVE M, GODAVARTHY S, MANOHAR V, et al. Towards macro-and micro-expression spotting in video using strain patterns ［C］ ∥ Workshop on WACV 2009. Snowbird: IEEE, 2009: 1-6.
［12］ SHREVE M, GODAVARTHY S, GOLDGOF D, et al. Macro-and micro-expression spotting in long videos using spatio-temporal strain ［C］ ∥ IEEE International Conference and Workshops on FG 2011. Santa Barbara: IEEE, 2011: 51-56.
［13］ LI X, PFISTER T, HUANG X, et al. A spontaneous micro-expression database: inducement, collection and baseline ［C］ ∥ 2013 10th IEEE International Conference and Workshops on FG 2013. Shanghai: IEEE, 2013: 1-6.
［14］ PFISTER T, LI X, ZHAO G, et al. Recognising spontaneous facial micro-expressions ［C］ ∥ IEEE International Conference on ICCV 2011. Barcelona: IEEE, 2011: 1449-1456.
［15］ BARRON J L, FLEET D J, BEAUCHEMIN S S. Performance of optical flow techniques ［J］. International Journal of Computer Vision, 1994, 12(1): 43-77.
［16］ CHEN J S, YANG C, DENG Y, et al. Exploring facial asymmetry using optical flow ［J］. IEEE Signal Processing Letters, 2014, 21(7): 792-795.
［17］ HORN B K, SCHUNCK B G. Determining optical flow ［J］. Artificial Intelligence, 1981, 17(1-3): 185-203.
［18］ KOMORKIEWICZ M, KRYJAK T, GORGON M. Efficient hardware implementation of the horn-schunck algorithm for high-resolution real-time dense optical flow sensor ［J］. Sensors, 2014, 14(2): 2860-2891.
|
|
Viewed |
|
|
|
Full text
|
|
|
|
|
Abstract
|
|
|
|
|
Cited |
|
|
|
|
| |
Shared |
|
|
|
|
| |
Discussed |
|
|
|
|
