| Automatic Technology, Computer Technology |
|
|
|
|
| Self-learning framework for attribution relationship of people carrying objects under family environment |
Hao WU1( ),Wen-jing LI1,Guo-hui TIAN1,*(),Zhao-wei CHEN1,Yong YANG2 |
1. School of Control Science and Engineering, Shandong University, Jinan 250061, China
2. Jinan Haoyuan Automatic Control System Engineering Limited Company, Jinan 250061, China |
|
|
|
Abstract It is necessary for service robots to have the ability to independently obtain the attribution relationship between people and their carrying objects in order to satisfy the requirements of robot personalized service and enable robots to select exclusive objects to perform inference and planning according to different service individual. A self-learning framework for the attribution relationship between people and their carrying objects was proposed aiming at the problem of the attribution relationship between people carrying objects and people in the family environment. The method of detecting and locating people carrying objects was used based on the object detection model SSD and the human posture estimation model OpenPose in order to realize the detection of human carrying objects. The face detection and recognition model MTCNN were used to complete the service individual identification by extracting the objects features by convolutional neural network based on migration learning and using the backend classifier to complete the object instance attribute identification. The self-learning of the attribution relationship was completed through the self-learning strategy. The experimental results show that the proposed self-learning framework for attribution relationship of people carrying objects can accurately and efficiently complete the learning of attribution relationship, effectively eliminating the influence of environmental interference factors on attribution learning. The proposed framework has good accuracy and robustness.
|
|
Received: 31 October 2018
Published: 25 June 2019
|
|
|
|
Corresponding Authors:
Guo-hui TIAN
E-mail: wh911@sdu.edu.cn;g.h.tian@sdu.edu.cn
|
室内环境中人穿携物品归属关系自主学习框架
为了满足机器人个性化服务需求，使机器人可以根据不同的服务对象来选择专属物品进行任务的推理和规划，需要服务机器人具备自主获取人穿携物品与人的归属关系的能力. 针对家庭环境下人穿携物品与人的归属关系获取问题，提出人穿携物品归属关系自主学习框架. 采用基于物品检测模型SSD与人体姿态估计模型OpenPose相结合的人穿携物品检测定位方法，实现人穿携物品检测. 利用基于迁移学习的卷积神经网络提取物品特征，通过后端分类器完成物品实例属性识别，使用人脸检测与识别模型MTCNN完成服务对象身份识别. 通过归属关系自主学习策略，完成归属关系的自主学习. 实验结果表明，提出的人穿携物品归属关系自主学习框架能够准确、高效地完成归属关系的学习，有效排除环境干扰因素对归属关系学习的影响，具有良好的准确性和鲁棒性.
关键词：
服务机器人,
个性化服务,
深度学习,
穿携物品,
物品归属关系
|
|
| [1] |
HSIEH J W, CHENG J C, CHEN L C, et al Handheld object detection and its related event analysis using ratio histogram and mixture of HMMs[J]. Journal of Visual Communication and Image Representation, 2014, 25 (6): 1399- 1415
doi: 10.1016/j.jvcir.2014.05.009
|
|
|
| [2] |
RIVERA-RUBIO J, IDREES S, ALEXIOU I, et al. A dataset for hand-held object recognition [C] // IEEE International Conference on Image Processing. Paris: IEEE, 2015: 5881-5885.
|
|
|
| [3] |
LV X, JIANG S Q, HERRANZ L, et al RGB-D hand-held object recognition based on heterogeneous feature fusion[J]. Computer Science and Technology, 2015, 30 (2): 340- 352
doi: 10.1007/s11390-015-1527-0
|
|
|
| [4] |
LI X, JIANG S Q, LV X, et al. Learning to recognize hand-held objects from scratch [C] // Advances in Multimedia Information Processing. Berlin: Springer, 2016: 527–539.
|
|
|
| [5] |
YAMAGUCHI K, KIAPOUR M H, ORTIZ L E, et al. Parsing clothing in fashion photographs [C] // Computer Vision and Pattern Recognition. Phode Island: IEEE, 2012: 3570-3577.
|
|
|
| [6] |
LIANG X D, XU C Y, SHEN X H, et al. Human parsing with contextualized convolutional neural network [C] // IEEE International Conference on Computer Vision. Santiago: IEEE, 2015: 1386-1394.
|
|
|
| [7] |
GONG K, LIANG X D, ZHANG D Y, et al. Look into person: self-supervised structure-sensitive learning and a new benchmark for human parsing [C] // IEEE Conference on Computer Vision and Pattern Recognition. Honolulu: IEEE, 2017: 932-940.
|
|
|
| [8] |
CHEN X J, MOTTAGHI R, LIU X B, et al. Detect what you can: detecting and representing objects using holistic models and body parts [C] // Computer Vision and Pattern Recognition. Columbus: IEEE, 2014: 1979-1986.
|
|
|
| [9] |
LI J S, ZHAO J, WEI Y C, et al. Towards real world human parsing: multiple-human parsing in the wild [C] // Computer Vision and Pattern Recognition. Honolulu: IEEE, 2017.
|
|
|
| [10] |
LIU W, ANGUELOV D, ERHAN D, et al. SSD: single shot multibox detector [C] // European Conference on Computer Vision. Berlin: Springer, 2016: 21-37.
|
|
|
| [11] |
LIN T Y, GOYAL P, GIRSHICK R, et al. Focal loss for dense object detection [C] // Proceedings of the IEEE International Conference on Computer Vision. Venice: IEEE, 2017: 2999-3007.
|
|
|
| [12] |
IOFFE S, SZEGEDY C. Batch normalization: accelerating deep network training by reducing internal covariate shift [C] // Proceedings of the 32nd International Conference on Machine Learning. Coimbra: ACM, 2015: 448-456.
|
|
|
| [13] |
SARAFIANOS N, BOTEANU B, IONESCU B, et al 3D human pose estimation: a review of the literature and analysis of covariates[J]. Computer Vision and Image Understanding, 2016, 152: 1- 20
doi: 10.1016/j.cviu.2016.09.002
|
|
|
| [14] |
CAO Z, SIMON T, WEI S E, et al. Realtime multi-person 2D pose estimation using part affinity fields [C] // Computer Vision and Pattern Recognition. Las Vegas: IEEE, 2016: 1302-1310.
|
|
|
| [15] |
CHENG M M. Saliency and similarity detection for image scene analysis [D]. Tianjin: NanKai University, 2012: 33-56.
|
|
|
| [16] |
ZHAO Z Q, HUANG D S, SUN B Y Human face recognition based on multi-features using neural networks committee[J]. Pattern Recognition Letters, 2004, 25 (12): 1351- 1358
doi: 10.1016/j.patrec.2004.05.008
|
|
|
| [17] |
GEETHA K P, VADIVELU S S, SINGH N A. Human face recognition using neural networks [C] // Radio Science Conference. Cairo: IEEE, 2012: 260-263.
|
|
|
| [18] |
YANG S, LUO P, CHEN C L, et al. WIDER FACE: a face detection benchmark [C] // Computer Vision and Pattern Recognition. Las Vegas: IEEE, 2016: 5525-5533.
|
|
|
| [19] |
LIU Z, LUO P, WANG X, et al. Deep learning face attributes in the wild [C] // Proceedings of the IEEE International Conference on Computer Vision. Boston: IEEE, 2015: 3730-3738.
|
|
|
| [20] |
WANG M, DENG W. Deep face recognition: a survey [C] // Computer Vision and Pattern Recognition. Salt Lake City: IEEE, 2018: 471-478.
|
|
|
| [21] |
ZHANG K, ZHANG Z, LI Z, et al Joint face detection and alignment using multitask cascaded convolutional networks[J]. IEEE Signal Processing Letters, 2016, 23 (10): 1499- 1503
doi: 10.1109/LSP.2016.2603342
|
|
|
|
Viewed |
|
|
|
Full text
|
|
|
|
|
Abstract
|
|
|
|
|
Cited |
|
|
|
|
| |
Shared |
|
|
|
|
| |
Discussed |
|
|
|
|
