Please wait a minute...
浙江大学学报(工学版)
JOURNAL OF ZHEJIANG UNIVERSITY (ENGINEERING SCIENCE)
Pervasive Computing and Computer Human Interaction     
Mobile device based eye tracking technology
CHENG Shi wei, LU Yu hua, CAI Hong gang
School of Computer Science and Technology, Zhejiang University of Technology, Hangzhou 310023, China
Download:   PDF(1886KB) HTML
Export: BibTeX | EndNote (RIS)      

Abstract  

A hierarchical processing framework and the related detect approaches were proposed, treating images from face to pupil areas in order to develop high efficient and accurate eye tracking technique on mobile device. Firstly, local binary pattern based cascaded classifier was applied to classify face and nonface areas of the image. Then Haar feature based cascaded classifier was used to detect eye areas, and image templatematching method was applied to detect pupil position within eye areas. Finally, an eye tracking reading assistant system was developed, which could detect the change of pupil position and locate the text line where users currently read. The system could help users find the text line to continue reading when they were interrupted. The results of users’ test show that the system has eye tracking accuracy at 1.17° of visual angle in average, and can locate the text lines accurately. The system can help users achieve the average reading speed at 12.42 words per second. The effectiveness of the eye tracking technique was verified.

Published: 01 June 2016
CLC:     
  TP 391  
Service
E-mail this article
Add to my bookshelf
Add to citation manager
E-mail Alert
RSS
Articles by authors
Cite this article:

CHENG Shi wei, LU Yu hua, CAI Hong gang. Mobile device based eye tracking technology. JOURNAL OF ZHEJIANG UNIVERSITY (ENGINEERING SCIENCE), 2016, 50(6): 1160-1166.

URL:

http://www.zjujournals.com/eng/10.3785/j.issn.1008973X.2016.06.021     OR     http://www.zjujournals.com/eng/Y2016/V50/I6/1160


移动设备眼动跟踪技术

为了在移动设备上开发计算效率和精度较高的眼动跟踪技术,提出从人脸到瞳孔逐级推进的图像处理框架和具体检测方法.采用基于局部二值模式特征的级联分类器,对人脸和非人脸区域进行分类检测.在人脸区域内应用 Haar 特征级联分类器检测人眼区域;并使用模板匹配法在人眼区域内检测瞳孔位置.在此基础上开发基于眼动跟踪的移动设备阅读辅助系统, 根据瞳孔位置的变化定位用户当前阅读的文本行,帮助用户在阅读中断后快速找到阅读起点. 测试结果表明：该系统对视线角度的平均检测精度达到1.17°,能精确定位阅读文本,将用户的平均阅读速度提高到每秒12.42个字,验证了所提出的眼动跟踪技术的实用性.

［1］ GARCIA D, SINTOS I. EyeDROID: Gaze tracking component for Android SPCLAutumn 2014［J/OL］. ［20150513］.http:∥www.itu.dk/～tped/teaching/pervasive/SPCLE2014/draft01handins/02_EyeDROID%20Gaze%20tracking%20component%20for%20Android.pdf.
［2］ NAGAMATSU T, YAMAMOTO M, SATO H. MobiGaze: development of a gaze interface for handheld mobile devices ［C］∥ CHI′10 Extended Abstracts on Human Factors in Computing Systems. New York: ACM, 2010: 3349-3354.
［3］ LUKANDER K. Measuring gaze point on handheld mobile devices ［C］∥ CHI′04 Extended Abstracts on Human Factors in Computing Systems. New York: ACM, 2004: 1556-1556.
［4］ DEUBEL H, BRIDGEMAN B. Fourth Purkinje image signals reveal eyelens deviations and retinal image distortions during saccades ［J］. Vision Research, 1995,35(4): 529-538.
［5］ 程时伟,孙志强.用于移动设备人机交互的眼动跟踪方法［J］.计算机辅助设计与图形学学报, 2014, 26(8): 1354-1361.
CHENG Shiwei, SUN Zhiqiang. An approach to eye tracking for mobile device based interaction ［J］. Journal of ComputerAided Design and Computer Graphics, 2014, 26(8): 1354-1361.
［6］ MARIAKAKIS A, GOEL M, AUMI M T I, et al. SwitchBack: using focus and saccade tracking to guide users’ attention for mobile task resumption［C］∥ Proceedings of the 33rd Annual ACM Conference on Human Factors in Computing Systems (CHI). New York: ACM, 2015: 2953-2962.
［7］ KAO C W, CHEN Y W, YANG C W, et al. Eye gaze tracking based on pattern voting scheme for mobile device ［C］∥ Proceedings of International Conference on Instrumentation, Measurement, Computer, Communication and Control. New York: IEEE, 2011: 337-340.
［8］ MILUZZO E, WANG T, CAMPBELL A T. EyePhone: activating mobile phones with your eyes ［C］∥ Proceedings of the Second ACM SIGCOMM Workshop on Networking, Systems, and Applications on Mobile Handhelds. New York: ACM, 2010: 1520.
［9］ PINO C, KAVASIDIS I. Improving mobile device interaction by eye tracking analysis ［C］∥ Proceedings of Federated Conference on Computer Science and Information Systems (FedCSIS). New York: IEEE, 2012: 1199-1202.
［10］ WOOD E, BULLING A. Eyetab: modelbased gaze estimation on unmodified tablet computers ［C］∥ Proceedings of the Symposium on Eye Tracking Research and Applications (ETRA). New York: ACM, 2014: 207-210.
［11］ OJALA T, PIETIKAINEN M, MAENPAA T. Multiresolution grayscale and rotation invariant texture classification with local binary patterns ［J］. IEEE Transactions on Pattern Analysis and Machine Intelligence, 2002, 24(7): 971-987.
［12］ LI W, FU P, ZHOU L. Face recognition method based on dynamic threshold local binary pattern ［C］∥ Proceedings of the 4th International Conference on Internet Multimedia Computing and Service. New York: ACM, 2012: 20-24.
［13］ SONG E, PAN N, HUNG C C, et al. Reflection invariant local binary patterns for image texture classification ［C］∥ Proceedings of the 2015 Conference on Research in Adaptive and Convergent Systems. New York: ACM, 2015: 210-215.
［14］ EBADI T, KUKENYS I, BROWNE W N, et al. Humaninterpretable feature pattern classification system using learning classifier systems ［J］. Evolutionary Computation, 2014, 22(4): 629-650.
［15］ VIOLA P, JONES M. Rapid object detection using a boosted cascade of simple features ［C］∥ Proceedings of the 2001 IEEE Computer Society Conference on Computer Vision and Pattern Recognition. New York: IEEE, 2001: 1511-1518.
［16］ FREUND Y. Boosting a weak learning algorithm by majority ［J］. Information and Computation, 1995,121(2): 256-285.
［17］ MAHALINGAM G, KAMBHAMETTU C. Face verification with aging using AdaBoost and local binary patterns ［C］∥ Proceedings of the Seventh Indian Conference on Computer Vision, Graphics and Image Processing. New York: ACM, 2010: 101-108.
［18］ LIENHART R, MAYDT J. An extended set of Haarlike features for rapid object detection［C］∥ Proceedings of 2002 International Conference on Image Processing. New York: IEEE, 2002: 1900-1903.
［19］ REZAEI M, KLETTE R. Adaptive Haarlike classifier for eye status detection under nonideal lighting conditions ［C］∥ Proceedings of the 27th Conference on Image and Vision Computing New Zealand. New York: ACM, 2012: 521-526.
［20］ TENG Z, ZHANG B, LIU F. Railway region detection based on Haarlike features［C］∥ Proceedings of International Conference on Internet Multimedia Computing and Service. New York: ACM, 2014: 121.
［21］ 刘宝生, 闫莉萍, 周东华. 几种经典相似性度量的比较研究［J］.计算机应用研究, 2006, 23(11): 13.
LIU Baosheng, YAN Liping, ZHOU Donghua. Comparison of some classical similarity measures ［J］. Application Research of Computers, 2006, 23(11): 13.
[1] DONG Kai, LAI Jun ying, QIAN Xiao qian, ZHAN Shu lin, RUAN Fang. Energy efficiency of residential buildings with horizontal external shading in hot summer and cold winter zone[J]. JOURNAL OF ZHEJIANG UNIVERSITY (ENGINEERING SCIENCE), 2016, 50(8): 1431-1437.
[2] LI Jia qi, FAN Li wu, YU Zi tao. Boiling heat transfer characteristics during quench cooling on superhydrophilic surface[J]. JOURNAL OF ZHEJIANG UNIVERSITY (ENGINEERING SCIENCE), 2016, 50(8): 1493-1498.
[3] CHIANG Yen ming, ZHANG Jian quan, MING Yan. Flood forecasting by ensemble neural networks[J]. JOURNAL OF ZHEJIANG UNIVERSITY (ENGINEERING SCIENCE), 2016, 50(8): 1471-1478.
[4] ZHONG Wei, PENG Liang, ZHOU Yong gang, XU Jian, CONG Fei yun. Slagging diagnosis of boiler based on wavelet packet analysis and support vector machine[J]. JOURNAL OF ZHEJIANG UNIVERSITY (ENGINEERING SCIENCE), 2016, 50(8): 1499-1506.
[5] XIA Yu feng, REN Li, YE Cai hong, WANG Li. Multi-objective optimization of locators layout of reinforced panel based on RSM[J]. JOURNAL OF ZHEJIANG UNIVERSITY (ENGINEERING SCIENCE), 2016, 50(8): 1600-1607.
[6] LI Lin yu, WU Zhang hua, YU Guo yao, DAI Wei, LUO Er cang.
Experimental investigation on electroacoustic conversion characteristic of linear compressor
[J]. JOURNAL OF ZHEJIANG UNIVERSITY (ENGINEERING SCIENCE), 2016, 50(8): 1529-1536.
[7] QU Wei wei, TANG Wei, BI Yun bo, LI Shao bo, LUO Shui jun. Pre-joining processes plan to avoid forced assemblies and improve efficiency[J]. JOURNAL OF ZHEJIANG UNIVERSITY (ENGINEERING SCIENCE), 2016, 50(8): 1561-1569.
[8] HU Xiao dong, GU Lin yi, ZHANG Fan meng. High-speed on/off valves applied in digital displacement motor[J]. JOURNAL OF ZHEJIANG UNIVERSITY (ENGINEERING SCIENCE), 2016, 50(8): 1551-1560.
[9] YANG Shu, LIU Guo ping, QI Chang, WANG Da zhi. Simulation and optimization for anti-shock performances of graded metal hollow sphere foam structure[J]. JOURNAL OF ZHEJIANG UNIVERSITY (ENGINEERING SCIENCE), 2016, 50(8): 1593-1599.
[10] YANG Zhang, TONG Gen shu, ZHANG Lei. Effective Rigidity of two one-side stiffeners arranged symmetrically[J]. JOURNAL OF ZHEJIANG UNIVERSITY (ENGINEERING SCIENCE), 2016, 50(8): 1446-1455.
[11] JIANG Xiang, TONG Gen shu, ZHANG Lei. Experiments on fire-resistance performance of fire-resistant steel-concrete composite beams[J]. JOURNAL OF ZHEJIANG UNIVERSITY (ENGINEERING SCIENCE), 2016, 50(8): 1463-1470.
[12] SHAN Hua feng, XIA Tang dai, YU Feng, HU Jun hua,PAN Jin long. Buckling stability analysis on critical load of underpinning pile for excavation beneath existing building[J]. JOURNAL OF ZHEJIANG UNIVERSITY (ENGINEERING SCIENCE), 2016, 50(8): 1425-1430.
[13] GU Tian lai, ZHANG Shuai, ZHENG Yao. Back pressure characteristics of jaws inlet with constant-area isolator[J]. JOURNAL OF ZHEJIANG UNIVERSITY (ENGINEERING SCIENCE), 2016, 50(7): 1418-1424.
[14] ZHENG Cheng zhi, GAO Jin liang, HE Wen jie. Leakage discharge analysis model based on FastICA algorithm[J]. JOURNAL OF ZHEJIANG UNIVERSITY (ENGINEERING SCIENCE), 2016, 50(6): 1031-1039.
[15] HUANG Jia hui, FENG Dong qin. Method for vulnerability evaluation of Cyberphysical system based on generalized profit[J]. JOURNAL OF ZHEJIANG UNIVERSITY (ENGINEERING SCIENCE), 2016, 50(6): 1119-1125.