| Image Analysis and 3D Reconstruction |
|
|
|
|
| Home fitness monitoring system based on monocular camera |
| YU Peng, LIU Lan, CAI Yun, HE Yu, ZHANG Songhai |
| Department of Computer Science and Technology, Tsinghua University, Beijing 100084, China |
|
|
|
Abstract With the increasing richness of online fitness resources, autonomous fitness has become a new sporting trend. However, due to the lack of action guidance and correction by professional fitness coaches, autonomous fitness usually cannot guarantee the fitness effect and is easy to cause sports injuries, so real-time monitoring of fitness actions is required. Existing fitness monitoring equipment usually relies on professional hardware such as big screens, stereo camera and other sensors. As a result, they fail to satisfy common needs for virtual fitness due to high cost, complexity of installation and limited application scenario. With gradual maturing of human pose estimation technique, identification of human face and movements of limbs can be realized through easily accessible cell phone camera with high accuracy and speed. Low cost, multi-scenario virtual fitness monitoring on mobile terminal thus it made possible. Based on the background above, this work designs a fitness action evaluation algorithm based on angle thresholds in 3D scenes which relies on cell phone monocular camera and 3D human key point detection technology. The algorithm can detect whether the user's fitness actions are standard in real time and give corresponding responses through voice. The work has implemented a prototype system on an Android phone. This work verifies the usability and real-time performance of the algorithm and the system through a series of user experiments. It also verifies the accuracy of the action evaluation algorithm through comparative experiments with relevant work in recent years. Results shows that the algorithm and functions of the prototype system were greatly recognized by the users with high accuracy, reasonable responding speed for real time usage.
|
|
Received: 09 December 2020
Published: 15 September 2021
|
|
|
基于单目摄像头的自主健身监测系统
随着在线健身资源的日益丰富，自主健身已成为新的运动趋势。然而由于缺少专业健身教练的动作指导与纠正，自主健身通常无法保障健身效果且容易造成运动损伤，因此需要对健身动作准确性进行实时监测。现有的健身监测设备往往依托大屏幕、深度摄像头或传感器等硬件，存在设备昂贵、安装不便、使用场景受限等问题，较难满足大众健身的需求。随着人体关键点检测技术的不断成熟，通过手机单目摄像头即可实现对人体姿态的识别，且有较高的准确度和速度，使得在手机端实现低成本、多场景的健身监测成为可能。基于以上背景，设计了三维场景下基于角度阈值的健身动作评估算法，依托于手机单目摄像头和3D人体关键点检测技术，实时检测用户健身动作是否标准并通过语音给出相应提示。同时，在安卓手机上实现了原型系统，通过一系列用户实验验证了系统的可用性与实时性，通过与近年相关工作的对比实验，验证了动作评估方法的准确性。结果表明，本文方法与系统被用户所认可，健身动作评估准确率较高、响应速度满足实时性要求。
关键词：
自主健身,
人体关键点检测,
动作监测,
虚拟健身教练
|
|
1 刁在箴，马更娣，张莹，等.中国体育健身俱乐部发展概况之研究［J］.北京体育大学学报，2002（6）：744-745，750. DOI： 10.3969/j.issn.1007-3612.2002. 06.009 DIAO Z Z， MA G D， ZHANG Y， et al. Research on general situation of clubs for body-building in China［J］. Journal of Beijing University of Physical Education， 2002（6）：744-745，750. DOI： 10.3969/j.issn.1007-3612.2002.06.009
2 XIA S H， GAO L， LAI Y K， et al. A survey on human performance capture and animation［J］. Journal of Computer Science and Technology， 2017， 32： 536-554. DOI： 10.1007/s11390-017-1742-y
3 VELLOSO E， BULLING A， GELLERSEN H， et al. Qualitative activity recognition of weight lifting exercises［C］//Proceedings of 4th Augmented Human International Conference. New York： Association for Computing Machinery， 2013： 116-123. DOI： 10.1145/2459236.2459256
4 CHANG K H， CHEN M Y， CANNY J. Tracking free-weight exercises［C］//Proceedings of the 9th International Conference on Ubiquitous Computing. Berlin/Heidelberg： Springer-Verlag， 2007： 19-37. DOI： 10.5555/1771592.1771594
5 CREMA C， DEPARI A， FLAMMINI A， et al. IMU-based solution for automatic detection and classification of exercises in the fitness scenario［C］//2017 IEEE Sensors Applications Symposium （SAS）. Piscataway： IEEE， 2017： 1-6. DOI： 10. 1109/SAS.2017.7894068
6 DING H， SHANGGUAN L F， YANG Z， et al. FEMO： A platform for free-weight exercise monitoring with RFIDS［C］//Proceedings of the 13th ACM Conference on Embedded Networked Sensor Systems. New York： Association for Computing Machinery， 2015： 141-154. DOI： 10.1145/2809695. 2809708
7 ZHOU B， SUNDHOLM M， CHENG J Y， et al. Never skip leg day： A novel wearable approach to monitoring gym leg exercises［C］//2016 IEEE International Conference on Pervasive Computing and Communications. Piscataway： IEEE， 2016： 1-9. DOI： 10.1109/PERCOM.2016. 7456520
8 HAO T， XING G L， ZHOU G. RunBuddy： A smartphone system for running rhythm monitoring［C］// Proceedings of the 2015 ACM International Joint Conference on Pervasive and Ubiquitous Computing. New York： Association for Computing Machinery， 2015： 133-144. DOI： 10.1145/2750858. 2804293
9 JIN X， YAO Y， JIANG Q L， et al. Virtual personal trainer via the Kinect sensor［C］//2015 IEEE 16th International Conference on Communication Technology （ICCT）. Piscataway： IEEE， 2015： 460-463. DOI：10.1109/icct.2015.7399879
10 KUMAR P， SAINI R， YADAVA M， et al. Virtual trainer with real-time feedback using kinect sensor［C］//2017 IEEE Region 10 Symposium （TENSYMP）. Piscataway： IEEE， 2017： 1-5. DOI： 10.1109/TENCONSpring.2017.8070063
11 QIAO S， WANG Y L， LI J. Real-time human gesture grading based on OpenPose［C］//2017 10th International Congress on Image and Signal Processing， BioMedical Engineering and Informatics （CISP-BMEI）. Piscataway： IEEE， 2017： 1-6. DOI： 10.1109/CISP-BMEI.2017.8301910
12 DEB S， SHARAN A， CHATURVEDI S， et al. Interactive dance lessons through human body pose estimation and skeletal topographies matching［J］. International Journal of Computational Intelligence & IoT， 2018， 2（4）：711-716.
13 WEI S E， RAMAKRISHNA V， KANADE T， et al. Convolutional pose machines［C］//2016 IEEE conference on Computer Vision and Pattern Recognition. Piscataway： IEEE， 2016： 4724-4732. DOI： 10.1109/CVPR.2016.511.
14 NEWELL A， YANG K Y， DENG J. Stacked hourglass networks for human pose estimation［C］// European Conference on Computer Vision （ECCV）. Cham： Springer， 2016： 483-499. DOI： 10.1007/978-3-319-46484-8_29
15 HE K， GKIOXARI G， DOLLÁR P， et al. Mask RCNN［C］// 2017 IEEE International Conference on Computer Vision. Piscataway： IEEE， 2017： 2961-2969. DOI： 10.1109/tpami.2018.2844175
16 CHEN Y L， WANG Z C， PENG Y X， et al. Cascaded pyramid network for multi-person pose estimation［C］// 2018 IEEE Conference on Computer Vision and Pattern Recognition. Piscataway： IEEE， 2018： 7103-7112. DOI： 10.1109/CVPR.2018.00742.
17 LI W B， WANG Z C， YIN B Y， et al. Rethinking on multi-stage networks for human pose estimation［Z/OL］. （2019-01-01）.https：//arXiv.org/abs/1901. 00148. doi:10.1609/aaai.v34i07.6797
18 FANG H S， XIE S Q， TAI Y W， et al. RMPE： Regional multi-person pose estimation［C］// 2017 IEEE International Conference on Computer Vision. Piscataway： IEEE， 2017： 2353-2362. DOI： 10.1109/ICCV.2017.256.
19 SUN K， XIAO B， LIU D， et al. Deep high-resolution representation learning for human pose estimation［C］//2019 IEEE Conference on Computer Vision and Pattern Recognition. Piscataway： IEEE， 2019： 5686-5696. DOI： 10.1109/CVPR.2019.00584.
20 CAO Z， SIMON T， WEI S E， et al. Realtime multi-person 2D pose estimation using part affinity fields［C］//2017 IEEE Conference on Computer Vision and Pattern Recognition （CVPR）. Piscataway： IEEE， 2017： 1302-1310. DOI： 10.1109/CVPR. 2017.143.
21 NEWELL A， HUANG Z， DENG J. Associative embedding： End-to-end learning for joint detection and grouping［C］// 31st International Conference on Neural Information Processing Systems. New York： Curran Associates Inc，2017： 2274-2284.
22 XIA F T， WANG P， CHEN X J， et al. Joint multi-person pose estimation and semantic part segmentation［C］//2017 IEEE Conference on Computer Vision and Pattern Recognition （CVPR）. Piscataway： IEEE， 2017： 6780-6789. DOI： 10.1109/CVPR. 2017.644.
23 SHOTTON J， FITZGIBBON A， COOK M， et al. Real-time human pose recognition in parts from single depth images［C］//2011 IEEE Conference on Computer Vision and Pattern Recognition （CVPR）. Piscataway： IEEE， 2011： 1297-1304. DOI： 10.1109/CVPR.2011.5995316.
24 HAQUE A， PENG B Y， LUO Z L， et al. Towards viewpoint invariant 3D human pose estimation［C］//European Conference on Computer Vision. Cham： Springer， 2016： 160-177. DOI： 10.1007/978-3-319-46448-0_10
25 CHEN C H， RAMANAN D. 3D human pose estimation = 2D pose estimation + matching［C］// 2017 IEEE Conference on Computer Vision and Pattern Recognition （CVPR）. Piscataway： IEEE， 2017： 5759-5767. DOI： 10.1109/CVPR.2017.610.
26 MARTINEZ J， HOSSAIN R， ROMERO J， et al. A simple yet effective baseline for 3D human pose estimation［C］// 2017 IEEE International Conference on Computer Vision （ICCV）. Piscataway： IEEE， 2017： 2659-2668. DOI： 10.1109/ICCV.2017.288.
27 GÜLER R A，NEVEROVA N，KDKKINOS I.DensePose：Dense human pose estimation in the wild［C］//IEEE/CVF Corference on Computer Vision and Pattern Recognition.Salt Lake City：IEEE，2018：7297-7306. doi:10.1109/cvpr.2018.00762
28 PAVLLO D， FEICHTENHOFER C， GRANGIER D， et al. 3D human pose estimation in video with temporal convolutions and semi-supervised training［C］//2019 IEEE/CVF Conference on Computer Vision and Pattern Recognition （CVPR）. Piscataway： IEEE， 2019： 7745-7754. DOI： 10.1109/CVPR.2019.00794
29 IONESCU C， PAPAVA D， OLARU V， et al. Human 3.6m： Large scale datasets and predictive methods for 3D human sensing in natural environments［J］. IEEE Transactions on Pattern Analysis and Machine Intelligence， 2013， 36（7）： 1325-1339. DOI： 10.1109/TPAMI.2013.248 |
|
Viewed |
|
|
|
Full text
|
|
|
|
|
Abstract
|
|
|
|
|
Cited |
|
|
|
|
| |
Shared |
|
|
|
|
| |
Discussed |
|
|
|
|
