Please wait a minute...
浙江大学学报(工学版)
Journal of ZheJiang University (Engineering Science)  2021, Vol. 55 Issue (4): 608-614    DOI: 10.3785/j.issn.1008-973X.2021.04.002
    
User profiling based on activity sensing
Ming-hui YOU(),Ya-feng YIN*(),Lei XIE,Sang-lu LU
State Key Laboratory for Novel Software Technology, Nanjing University, Nanjing 210023, China
Download: HTML     PDF(873KB) HTML
Export: BibTeX | EndNote (RIS)      

Abstract  

A user profiling method based on activity sensing was proposed in order to build user profile while protecting their privacy. The built-in inertial sensors of the smart phone were used to sense the user activities (e.g., browsing, typing, calling etc.). Then the recognized activities were used to mine user characteristics, such as social personality and stress level, in order to build a preliminary user profile. The experimental results show that the method can recognize user activities and build user profiles. The accuracy of activity recognition was 87.2%, while the accuracy of predictions for the three user characteristics in gender, social personality, and stress level was 81.8%, 72.7%, 72.7%, respectively.

Key words： smartphone      inertial sensor      activity sensing      user characteristic mining      user profiling     
Received: 22 January 2021      Published: 07 May 2021
CLC:  TP 399  
Fund:  国家自然科学基金资助项目（61802169，61872174，61832008，61902175，61906085）；江苏省自然科学基金资助项目（BK20180325，BK20190293）；江苏省重点研发基金资助项目（BE2018116）；软件新技术与产业化协同创新中心资助项目
Corresponding Authors: Ya-feng YIN     E-mail: mf20330111@smail.nju.edu.cn;yafeng@nju.edu.cn
Service
E-mail this article
Add to my bookshelf
Add to citation manager
E-mail Alert
RSS
Articles by authors
Ming-hui YOU
Ya-feng YIN
Lei XIE
Sang-lu LU
Cite this article:

Ming-hui YOU,Ya-feng YIN,Lei XIE,Sang-lu LU. User profiling based on activity sensing. Journal of ZheJiang University (Engineering Science), 2021, 55(4): 608-614.

URL:

http://www.zjujournals.com/eng/10.3785/j.issn.1008-973X.2021.04.002     OR     http://www.zjujournals.com/eng/Y2021/V55/I4/608


基于行为感知的用户画像技术

为了在保护用户隐私的同时构建用户画像，提出基于行为感知的用户画像技术. 采用智能手机内置的惯性传感器，可以感知识别用户使用手机的行为（如浏览阅读、打字聊天、通话等）. 通过识别的用户行为挖掘用户特性，如性别、社交性格、压力状态等，构建初步的用户画像. 实验结果表明，利用该方法能够较好地识别用户行为并构建用户画像，其中行为识别的准确率为87.2%，针对性别、社交性格、压力状态3个具体用户特性的预测准确率分别为81.8%、72.7%、72.7%.


关键词： 智能手机,  惯性传感器,  行为感知,  用户特性挖掘,  用户画像 
Fig.1 User profiling based on activity sensing model
Fig.2 Noise-reducing with Kalman filtering
Fig.3 Data splitting on time series data
时域特征 计算公式
均值 $\mu =\dfrac{1}{N}{\displaystyle\sum }_{i=1}^{{N} }{C}_{i}$
标准差 $\sigma =\sqrt{\dfrac{1}{N}{\displaystyle\sum }_{i=1}^{N}{\left({C}_{i}-\mu\right)}^{2} }$
最大值 $f_{{\rm{max}}}={\rm{max} }\;\left\{ {C}_{i}|i\in \left(1, N\right)\right\}$
最小值 $f_{{\rm{min}}}={\rm{min} }\;\left\{ {C}_{i}|i\in \left(1, N\right)\right\}$
过均值率 $f_{{\rm{above}}}=\dfrac{ {\rm{num} }\;\left\{ {C}_{i}|i\in \left(1, N\right),{C}_{i} > \mu\right\}}{N}$
Tab.1 Time domain features used in training
频域特征 计算公式
均值 ${\mu }_{{\rm{amp}}}=\dfrac{1}{N}{\displaystyle\sum }_{i=1}^{{N} }{G}_{i}$
标准差 ${\sigma }_{{\rm{amp}}}=\sqrt{\dfrac{1}{N}{\displaystyle\sum }_{i=1}^{N}{\left({G}_{i}-{\mu }_{{\rm{amp}}}\right)}^{2} }$
峰度 ${\gamma }_{{\rm{amp}}}=\dfrac{1}{N}{\displaystyle\sum }_{i=1}^{N}{\left[\dfrac{ {G}_{i}-{\mu }_{{\rm{amp}}} }{ {\sigma }_{{\rm{amp}}} }\right]}^{4}- 3$
Tab.2 Frequency domain features used in training
Fig.4 Structure of feature vector
Fig.5 User profile and user tags
行为类别 浏览阅读 游戏 打字聊天 通话 视频
浏览阅读 77.6% 0.0% 16.5% 2.0% 3.9%
游戏 1.5% 89.9% 2.0% 0.3% 6.3%
打字聊天 5.2% 4.0% 87.9% 2.4% 0.5%
通话 4.8% 0.0% 5.1% 88.7% 1.4%
视频 3.7% 3.5% 0.2% 0.2% 92.4%
Tab.3 Accuracy confusion matrix of activity recognition
Fig.6 Accuracy of activity recognition on different users
用户特性 Acc /%
性别 81.8
社交性格 72.7
压力状态 72.7
Tab.4 Classification accuracy of user characteristics
Fig.7 User characteristic classification accuracy of different data collection time
[1]   ZHAO S, LI S, RAMOS J, et al User profiling from their use of smartphone applications: a survey[J]. Pervasive and Mobile Computing, 2019, 59: 101052
doi: 10.1016/j.pmcj.2019.101052
[2]   赵莎. 基于大规模手机感知数据的用户特性挖掘[D]. 杭州: 浙江大学, 2017.
ZHAO Sha. User understanding based on large-scale smartphone-sensed data [D]. Hangzhou: Zhejiang University, 2017.
[3]   CHOUDHURY T, CONSOLVO S, HARRISON B, et al The mobile sensing platform: an embedded activity recognition system[J]. IEEE Pervasive Computing, 2008, 7 (2): 32- 41
doi: 10.1109/MPRV.2008.39
[4]   何卫华. 人体行为识别关键技术研究[D]. 重庆: 重庆大学, 2012.
HE Wei-hua. Research on key technologies of human activity sensing [D]. Chongqing: Chongqing University, 2012.
[5]   LANE N D, MILUZZO E, LU H, et al A survey of mobile phone sensing[J]. IEEE Communications Magazine, 2010, 48 (9): 140- 150
doi: 10.1109/MCOM.2010.5560598
[6]   PL?TZ T, HAMMERLA N Y, OLIVIER P. Feature learning for activity recognition in ubiquitous computing [C]// International Joint Conference on Artificial Intelligence. Barcelona: [s.n.], 2011.
[7]   PREECE S J, GOULERMAS J Y, KENNEY L P J, et al A comparison of feature extraction methods for the classification of dynamic activities from accelerometer data[J]. IEEE Transactions on Biomedical Engineering, 2009, 56 (3): 871- 879
doi: 10.1109/TBME.2008.2006190
[8]   RAVI N, DANDEKAR N, MYSORE P, et al. Activity recognition from accelerometer data [C]// Proceedings of the 20th National Conference on Artificial Intelligence and the 17th Innovative Applications of Artificial Intelligence Conference. Pittsburgh: AAAI, 2005.
[9]   CASALE P, PUJOL O, RADEVA P. Human activity recognition from accelerometer data using a wearable device [C]// Iberian Conference on Pattern Recognition and Image Analysis. Berlin: Springer, 2011: 289-296.
[10]   SENEVIRATNE S, SENEVIRATNE A, MOHAPATRA P, et al Predicting user traits from a snapshot of apps installed on a smartphone[J]. ACM Sigmobile Mobile Computing and Communications Review, 2014, 18 (2): 1- 8
doi: 10.1145/2636242.2636244
[11]   ZHAO S, RAMOS J, TAO J, et al. Discovering different kinds of smartphone users through their application usage behaviors [C]// Proceedings of the 2016 ACM International Joint Conference on Pervasive and Ubiquitous Computing. New York: ACM, 498–509.
[12]   O′DONOGHUE J, HERBERT J. Profile based sensor data acquisition in a ubiquitous medical environment [C]// 4th Annual IEEE International Conference on Pervasive Computing and Communications Workshops. Pisa: IEEE, 2006.
[13]   FARSEEV A, CHUA T S. Tweetfit: fusing multiple social media and sensor data for wellness profile learning [C]// Proceedings of the 31st AAAI Conference on Artificial Intelligence. San Francisco: AAAI, 2017: 95–101.
[14]   RAHMAN M A, EL SADDIK A, GUEAIEB W Augmenting context awareness by combining body sensor networks and social networks[J]. IEEE Transactions on Instrumentation and Measurement, 2010, 60 (2): 345- 353
[15]   SPARACINO F The museum wearable: real-time sensor-driven understanding of visitors' interests for personalized visually-augmented museum experiences[J]. Institution Archives and Museum Informatics, 2002, 17: 41
[16]   SZTYLER T, V?LKER J, CARMONA J, et al. Discovery of personal processes from labeled sensor data: an application of process mining to personalized health care [C]// Proceedings of the International Workshop on Algorithms and Theories for the Analysis of Event Data: Brussels: [s. n.], 2015: 31-46.
[17]   KALMAN R E A new approach to linear filtering and prediction problems[J]. Journal of Basic Engineering, 1960, 82D: 35- 45
[18]   张宪超, 武继刚, 蒋增荣, 等 离散傅里叶变换的算术傅里叶变换算法[J]. 电子学报, 2000, 28 (5): 105- 107
ZHANG Xian-chao, WU Ji-gang, JIANG Zeng-rong, et al An algorithm for computing DFT using arithmetic fourier transform[J]. Acta Electronica Sinica, 2000, 28 (5): 105- 107
[19]   OPUSZKO M, BERGER G, RUHLAND J. The impact of public scandals on social media: a sentiment analysis on Youtube to detect the influence on reputation [C]// 2019 6th European Conference on Social Media. Belgium: ACI, 2019: 36.
[1] Qi lei, JIN Wen-guang, GENG Wei-dong. Human motion capture using wireless inertial sensors[J]. Journal of ZheJiang University (Engineering Science), 2012, 46(2): 280-285.