Please wait a minute...
浙江大学学报(工学版)
JOURNAL OF ZHEJIANG UNIVERSITY (ENGINEERING SCIENCE)
Pervasive Computing and Computer Human Interaction     
Natural interaction model and algorithm based on virtual interface
LIU Hui, FENG Zhi quan, LIANG Li wei, XU Zhi peng
1. Department of Information Science and Engineering, University Jinan, Jinan 250022, China
2. Shandong Provincial Key Laboratory of Network Based Intelligent Computing, Jinan 250022, China
Download:   PDF(1806KB) HTML
Export: BibTeX | EndNote (RIS)      

Abstract  

The concept of virtual interface was proposed aiming at the “Midas Touch Problem” that occured in gesture interaction interfaces based vision. The statistical analysis method was used to establish the cognitive basis of virtual interface, and virtual interface construction algorithm was proposed based on cognitive behavioral model. Then a demonstration application based on virtual interface was implemented. Two methods including the static and dynamic ways were applied to construct the virtual interface. The virtual interface was divided into several regions averagely and each region was endowed with different functions. Therefore, multiple functions could be realized by only one kind of gesture, which reduced the memory load of users. The function of realtime refresh of virtual interface was achieved, which was the same effect as the human body. The effective perception method of virtual interface was put forward, and the interactive ability and positive effect of solving “Midas Touch Problem” were verified. The experimental results show that the proposed virtual interface not only solves the “Midas Touch Problem” efficiently, but also has a good user experience.

Published: 01 June 2016
CLC:  TP 391  
Service
E-mail this article
Add to my bookshelf
Add to citation manager
E-mail Alert
RSS
Articles by authors
Cite this article:

LIU Hui, FENG Zhi quan, LIANG Li wei, XU Zhi peng. Natural interaction model and algorithm based on virtual interface. JOURNAL OF ZHEJIANG UNIVERSITY (ENGINEERING SCIENCE), 2016, 50(6): 1167-1175.

URL:

http://www.zjujournals.com/eng/10.3785/j.issn.1008973X.2016.06.022     OR     http://www.zjujournals.com/eng/Y2016/V50/I6/1167


虚拟界面自然交互模型与算法

针对基于视觉的手势交互界面中的“点石成金问题（Midas Touch Problem）”,提出虚拟界面的概念.采用统计分析的方法建立虚拟界面的认知基础,提出基于认知行为模型的虚拟界面构建算法,实现基于虚拟界面的示范应用.采用静态和动态2种方式构建虚拟界面,对已构建的虚拟界面进行分区,使得每个分区具有不同的功能,从而达到一种手势实现多个功能,降低用户记忆负荷的目的.实现虚拟界面的实时刷新功能,达到与人体亦步亦趋的效果.提出有效的虚拟界面感知方法,验证其交互能力及其解决“Midas Touch Problem”的积极效果.实验结果表明：虚拟界面不仅能够高效解决“Midas Touch Problem”, 还具有良好的用户体验.

［1］ 武霞,张崎,许艳旭.手势识别研究发展现状综述［J］.电子科学,2013,26(6)： 171-174.
WU Xia, ZHANG Qi, XU Yanxu. An overview of hand gesture recognition ［J］. Electronic Science and Technology, 2013, 26(6): 171-174.
［2］ 冯志全,蒋彦.手势识别研究综述［J］.济南大学学报：自然科学版,2013,27(4)： 336-341.
FENG Zhiquan, JIANG Yan. A survey of hand gesture recognition ［J］. Journal of University of Jinan： Science and Technology, 2013, 27(4): 336-341.
［3］ 邬大鹏.基于视觉的手势识别及人机交互研究［D］.南京：南京航空航天大学,2010.
WU Dapeng. Research on visionbased gesture recog nition and humancompurt interaction ［D］. Nanjing: Na njing University of Aeronautics and Astronautics, 2010.
［4］ 方旭亮. 基于计算机视觉的手势交互系统研究与设计 ［D］. 杭州：浙江大学,2008.
FANG Xuliang. The research and design of gesture interactive system based on computer vision ［D］. Hangzhou: Zhejiang University, 2008.
［5］ 陈子毫.基于深度信息的手势检测与跟踪［D］. 广州：华南理工大学,2012.
CHEN Zihao. Hand detection and tracking based on depth maps ［D］. Guangzhou: South China University of Technology, 2012.
［6］ 王辉.基于视觉的实时手势跟踪与识别及其在人机交互中的应用［D］.杭州：浙江大学, 2008.
WANG Hui. Vision based realtime gesture tracking and recognition and its application in humancomputer interaction techniques ［D］. Hangzhou: Zhejiang University, 2008.
［7］ 武汇岳,张凤军,刘玉进,等.基于视觉的互动游戏手势界面工具箱［J］.软件学报,2011,22（5）: 1067-1081.
WU Yuehui, ZHANG Fengjun, LIU Yujin, et al. Visionbased gesture interface toolkint for interactive games ［J］. Journal of Software, 2011,22（5）: 10671081.
［8］ 武汇岳,张凤军,刘玉进,等.基于视觉的手势界面关键技术研究［J］.计算机学报,2009,32（10）: 2030-2041.
WU Yuehui, ZHANG Fengjun, LIU Yujin, et al. Research on key issue of visionbased gesture interfaces ［J］. Chinese Journal of Computers, 2009, 32（10）: 20302041.
［9］ JACOB R J K. Eye movementbased humancomputer interaction techniques: toward noncommand interfaces ［J］. In Advances in HumanComputer Interaction. 1993,4: 15119.
［10］ KATO H, BILLINGHURST M, POUPYREV I, et al. Virtual object manipulation on a tabletop AR environment ［C］∥ IEEE and ACM International Symposium on Augmented Reality, 2000. (ISAR 2000) Proceedings, Munich: IEEE, 2000: 111-119.
［11］ KJELDSEN R, LEVAS A, PINHANEZ C. Dynamically reconfigurable visionbased user interfaces ［J］. Machine Vision and Applications, 2004, 16(1): 612.
［12］ 张清华,周玉兰,滕海涛.基于粒计算的认知模型［J］.重庆邮电大学学报：自然科学版,2009,21(4): 494-501.
ZHANG Qinghua, ZHOU Yulan, TENG Haitao. Cognition model based on granular computing ［J］. Journal of Chong qing University of Posts and Telecommunications: Natural Science Edition. 2009, 21(4): 494-501.
［13］ 林燕.基于多模型融合的人手三维跟踪研究［D］.济南：济南大学,2012.
LIN Yan. Research on 3D hand tracking based on multimodel fusion ［D］. Jinan: University Jinan,2012.
［14］ 刘松涛,殷福亮.基于图割的图像分割方法及其新进展［J］.自动化学报,2012,38(6): 911-922.
LIU Songtao, YIN Fuliang. The basic principle and its new advance of image segmentation methods based on graph cuts ［J］. Acta Automatica sinica, 2012,38(6): 911-922.
［15］ 许新征,丁世飞,史忠植,等.图像分割的新理论和新方法［J］.电子学报,2010,38(增1): 76-82.
XU Xinzheng, DING Shifei, SHI Zhongzhi, et al. New theories and methods of image segmentation ［J］. ACTA Electronica Sinica, 2010, 38(Suppl.1): 76-82.
［16］ 莫舒. 基于视觉的手势分割算法的研究［D］. 广州：华南理工大学,2012.
MO Shu. Hand gesture segmentation algorithm based on vision ［D］. Guangzhou: South China University of Technology, 2012.
［17］ 闵凡文.基于认知行为模型库和Kinect平台的实时手势跟踪算法研究［D］.济南：济南大学,2014.
MIN Fanwen. Research on realtime hand tracking algorithm based on cognitivebehavioral model library and Kinect ［D］. Jinan: University of Jinan, 2014.
［18］ 周航.基于计算机视觉的手势识别系统研究［D］. 北京：北京交通大学,2008.
ZHOU Hang. Studies on the gesture recognition system based on computer vision \[D\]. Beijing: Beijing Jiaotong University, 2008.
［19］ 杨学文,冯志全,黄忠柱,等. 结合手势主方向和类Hausdorff距离的手势识别［J］. 计算机辅助设计与图形学学报. 2016,28(1): 75-81.
YANG Xuewen, FENG Zhiquan, HUANG Zhongzhu, et al. Gesture gecognition based on combining main direction of gesture and hausdorfflike distance ［J］. Journal of ComputerAided Design and Computer Graphics, 2016, 28(1): 75-81.
［20］ 王湃.人体工程学及其未来［J］.中国环境管理干部学院学报,2003, 13(2): 71-73.
WANG Pai. Human engineering science and its future ［J］. JOURNAL OF EMCC, 2003, 13(2): 71-73.
［21］ 杨春青.人体工程学的应用研究［J］.艺术与设计：理论,2009(7): 154-155.
YANG Chunqing. The study of application of ergonomics ［J］. Art and Design: Theory, 2009(7): 154-155.
[1] HE Xue-jun, WANG Jin, LU Guo-dong, LIU Zhen-yu, CHEN Li, JIN Jing. 3D head portrait sculpture by industrial robot based on triangular mesh slicing and collision detection[J]. JOURNAL OF ZHEJIANG UNIVERSITY (ENGINEERING SCIENCE), 2017, 51(6): 1104-1110.
[2] WANG Hua, HAN Tong-yang, ZHOU Ke. KeyGraph-based community detection algorithm for public security intelligence[J]. JOURNAL OF ZHEJIANG UNIVERSITY (ENGINEERING SCIENCE), 2017, 51(6): 1173-1180.
[3] YOU Hai-hui, MA Zeng-yi, TANG Yi-jun, WANG Yue-lan, ZHENG Lin, YU Zhong, JI Cheng-jun. Soft measurement of heating value of burning municipal solid waste for circulating fluidized bed[J]. JOURNAL OF ZHEJIANG UNIVERSITY (ENGINEERING SCIENCE), 2017, 51(6): 1163-1172.
[4] BI Xiao-jun, WANG Jia-hui. Teaching-learning-based optimization algorithm with hybrid learning strategy[J]. JOURNAL OF ZHEJIANG UNIVERSITY (ENGINEERING SCIENCE), 2017, 51(5): 1024-1031.
[5] JIANG Xin-long, CHEN Yi-qiang, LIU Jun-fa, HU Li-sha, SHEN Jian-fei. Wearable system to support proximity awareness for people with autism[J]. JOURNAL OF ZHEJIANG UNIVERSITY (ENGINEERING SCIENCE), 2017, 51(4): 637-647.
[6] WANG Liang, YU Zhi-wen, GUO Bin. Moving trajectory prediction model based on double layer multi-granularity knowledge discovery[J]. JOURNAL OF ZHEJIANG UNIVERSITY (ENGINEERING SCIENCE), 2017, 51(4): 669-674.
[7] LIAO Miao, ZHAO Yu-qian, ZENG Ye-zhan, HUANG Zhong-chao, ZHANG Bing-kui, ZOU Bei-ji. Automatic segmentation for cell images based on support vector machine and ellipse fitting[J]. JOURNAL OF ZHEJIANG UNIVERSITY (ENGINEERING SCIENCE), 2017, 51(4): 722-728.
[8] MU Jing-jing, ZHAO Xin-yue, HE Zai-xing, ZHANG Shu-you. Contour reconstruction of overlapped bubbles based on concave-convex transformation and circle fitting[J]. JOURNAL OF ZHEJIANG UNIVERSITY (ENGINEERING SCIENCE), 2017, 51(4): 714-721.
[9] HUANG Zheng-yu, JIANG Xin-long, LIU Jun-fa, CHEN Yi-qiang, GU Yang. Fusion feature based semi-supervised manifold localization method[J]. JOURNAL OF ZHEJIANG UNIVERSITY (ENGINEERING SCIENCE), 2017, 51(4): 655-662.
[10] LIU Lei, YANG Peng, LIU Zuo-jun. Locomotion-Mode recognition using multiple kernel relevance vector machine[J]. JOURNAL OF ZHEJIANG UNIVERSITY (ENGINEERING SCIENCE), 2017, 51(3): 562-571.
[11] GUO Meng-li, DA Fei-peng, DENG Xing, GAI Shao-yan. 3D face recognition based on keypoints and local feature[J]. JOURNAL OF ZHEJIANG UNIVERSITY (ENGINEERING SCIENCE), 2017, 51(3): 584-589.
[12] DAI Cai-yan, CHEN Ling, LI Bin, CHEN Bo-lun. Sampling-based link prediction in complex networks[J]. JOURNAL OF ZHEJIANG UNIVERSITY (ENGINEERING SCIENCE), 2017, 51(3): 554-561.
[13] WANG Hai jun, GE Hong juan, ZHANG Sheng yan. Fast object tracking algorithm via kernel collaborative presentation[J]. JOURNAL OF ZHEJIANG UNIVERSITY (ENGINEERING SCIENCE), 2017, 51(2): 399-407.
[14] ZHANG Ya nan, CHEN De yun, WANG Ying jie, LIU Yu peng. Incremental graph pattern matching based dynamic recommendation method for cold-start user[J]. JOURNAL OF ZHEJIANG UNIVERSITY (ENGINEERING SCIENCE), 2017, 51(2): 408-415.
[15] LIU Yu peng, QIAO Xiu ming, ZHAO Shi lei, MA Chun guang. Deep combination of large-scale features in statistical machine translation[J]. JOURNAL OF ZHEJIANG UNIVERSITY (ENGINEERING SCIENCE), 2017, 51(1): 46-56.