Please wait a minute...
浙江大学学报(工学版)
Journal of ZheJiang University (Engineering Science)  2022, Vol. 56 Issue (4): 702-710    DOI: 10.3785/j.issn.1008-973X.2022.04.009
    
Robot tactile recognition system based on piezoelectric film sensor
Yun-hao WANG1(),Ming-hui SUN1,*(),Yi XIN2,Bo-xuan ZHANG3()
1. College of Computer Science and Technology, Jilin University, Changchun 130012, China
2. College of Instrument Science and Electrical Engineering, Jilin University, Changchun 130061, China
3. College of Engineering and Computer Science, Portland State University, Portland 97201, USA
Download: HTML     PDF(1558KB) HTML
Export: BibTeX | EndNote (RIS)      

Abstract  

A robot tactile recognition system based on tactile sensor and convolutional neural network was designed and developed according to the piezoelectric effect of polyvinylidene fluoride (PVDF) material in order to make the robot sense the external environment information through touch and make up for the lack of audio-visual interaction information. The material type can be identified according to the collected tactile signal. A tactile recognition algorithm was proposed based on the progressive cascade convolutional neural network. The signal features of the robot sensor were extracted based on the convolutional neural network, including the short-time Fourier transform of the tactile data spectrum and the time domain characteristics of the signal representation period. The classification process was divided into coarse and fine levels by K-Medoids clustering algorithm and dynamic time warping (DTW) distance measure algorithm in order to solve the confusion problem of specific material recognition. The progressive classification model was constructed. The experimental results showed that the average recognition accuracy of the tactile sensor was about 97%. The robot can successfully recognize the touched real material, laying a foundation for the next exploration and interaction tasks.

Key words： robot haptics      polyvinylidene fluoride (PVDF)      piezoelectric film      convolutional neural network      sensor      clustering algorithm     
Received: 24 October 2021      Published: 24 April 2022
CLC:  TP 39  
  TP 212  
  TM 282  
Fund:  国家自然科学基金资助项目（61872164）
Corresponding Authors: Ming-hui SUN     E-mail: yunhaow20@mails.jlu.edu.cn;smh@jlu.edu.cn;pedanbox@gmail.com
Service
E-mail this article
Add to my bookshelf
Add to citation manager
E-mail Alert
RSS
Articles by authors
Yun-hao WANG
Ming-hui SUN
Yi XIN
Bo-xuan ZHANG
Cite this article:

Yun-hao WANG,Ming-hui SUN,Yi XIN,Bo-xuan ZHANG. Robot tactile recognition system based on piezoelectric film sensor. Journal of ZheJiang University (Engineering Science), 2022, 56(4): 702-710.

URL:

https://www.zjujournals.com/eng/10.3785/j.issn.1008-973X.2022.04.009     OR     https://www.zjujournals.com/eng/Y2022/V56/I4/702


基于压电薄膜传感器的机器人触觉识别系统

为了使机器人通过触觉感知外部环境信息，弥补视听交互信息缺失的不足，根据聚偏氟乙烯（PVDF）材料的压电效应设计开发基于触觉传感器和卷积神经网络的机器人触觉识别系统，能够根据所采集的触觉信号识别出材质类型. 提出基于渐进式级联卷积神经网络的触觉识别算法. 该算法基于卷积神经网络提取机器人传感器的信号特征，包括经过短时傅里叶变换的触觉数据频谱图和信号表征周期内的时域特征. 为了解决特定材质识别混淆的问题，利用K-Medoids聚类算法和动态时间规整（DTW）距离度量算法将分类过程区分为粗、细2个层次，构建渐进式分类模型. 实验表明，设计的触觉传感器对物体材质的平均识别正确率约为97%，机器人能够成功识别触摸到的真实材质，为下一步的探索交互任务奠定基础.


关键词： 机器人触觉,  聚偏氟乙烯(PVDF),  压电薄膜,  卷积神经网络,  传感器,  聚类算法 
Fig.1 Diagram of piezoelectric effect of PVDF material
Fig.2 Diagram of PVDF sensor
Fig.3 Evaluation of psychophysical dimensions of materials
Fig.4 Robotic arm used to identify object materials
Fig.5 Form of data obtained and processing process
Fig.6 Transformation of information from time domain to frequency domain
Fig.7 Structure of 1-D CNN
模块 参数 参数值
卷积层 卷积核维度 3×1×1
卷积层 卷积核步长 2
卷积层 卷积核个数 16
卷积层 卷积层激活函数 tanh
池化层 池化方式 最大池化
池化层 卷积核维度 3×1×1
池化层 卷积核步长 1
Tab.1 Parameter settings of one-dimensional CNN
Fig.8 Confusion matrix of classification results of 1-D CNN
Fig.9 Structure of 2-D CNN
Fig.10 Confusion matrix of classification results of 2-D CNN
Fig.11 Comparison of Euclidean distance and DTW distance measurement methods
Fig.12 Transfer of time planning curve in cost matrix
Fig.13 Comparison of clustering results between DTW and Euclidean distance
数据 Acc/%
人工分类（四分类） 36.67
人工分类（五分类） 57.74
算法分类（四分类） 87.33
算法分类（五分类） 99.44
Tab.2 Comparison of data classification accuracy based on different clustering methods
Fig.14 Signal classification model architecture of cascaded neural networks
Fig.15 Confusion matrix of final classification
Fig.16 Quantitative analysis of similar material recognition results
材质类型 Acc/%
1-D CNN 2-D CNN 渐进式模型
百洁布 95.25 100 100
栅格文件袋 70.32 100 100
硬质纸板 97.54 90.02 90.03
泡沫塑料 35.26 57.44 71.44
无痕布 88.44 100 100
纹理布 82.47 100 93.66
硅胶片 95.52 100 100
塑料片 100 100 100
塑料栅格 38.33 62.15 100
粗糙胶带 68.10 100 100
柔软布 72.05 100 100
软牛仔裤 78.66 100 100
硬质布 68.38 100 100
硬牛仔裤 85.54 57.06 100
毛衣 72.80 100 100
平均准确率 76.57 91.11 97.01
Tab.3 Classification accuracy of different algorithms for entire dataset
[1]   ALATISE M B, HANCKE G P A review on challenges of autonomous mobile robot and sensor fusion methods[J]. IEEE Access, 2020, 8 (1): 39830- 39846
[2]   LIU L, XU Y, ZHU J, et al A flexible thermal sensor based on PVDF film for robot finger skin[J]. Integrated Ferroelectrics, 2019, 201 (1): 23- 31
doi: 10.1080/10584587.2019.1668687
[3]   EGUíLUZ A G, RA?ó I, COLEMAN S A, et al Multimodal material identification through recursive tactile sensing[J]. Robotics and Autonomous Systems, 2018, 106: 130- 139
doi: 10.1016/j.robot.2018.05.003
[4]   YANG Y J, CHENG M Y, SHIH S C, et al A 32×32 temperature and tactile sensing array using PI-copper films[J]. The International Journal of Advanced Manufacturing Technology, 2010, 46 (9-12): 945- 956
doi: 10.1007/s00170-009-1940-z
[5]   HERSHEY S, CHAUDHURI S, ELLIS D P W, et al. CNN architectures for large-scale audio classification [C]// 2017 IEEE International Conference on Acoustics, Speech and Signal Processing. New Orleans: IEEE, 2017: 131-135.
[6]   WU H, CHEN J, LIU X, et al One-dimensional CNN-based intelligent recognition of vibrations in pipeline monitoring with DAS[J]. Journal of Lightwave Technology, 2019, 37 (17): 4359- 4366
doi: 10.1109/JLT.2019.2923839
[7]   JANA G C, SHARMA R, AGRAWAL A A 1D-CNN-spectrogram based approach for seizure detection from EEG signal[J]. Procedia Computer Science, 2020, 167 (1): 403- 412
[8]   INCE T, KIRANYAZ S, EREN L, et al Real-time motor fault detection by 1-D convolutional neural networks[J]. IEEE Transactions on Industrial Electronics, 2016, 63 (11): 7067- 7075
doi: 10.1109/TIE.2016.2582729
[9]   ACQUARELLI J, VAN LAARHOVEN T, GERRETZEN J, et al Convolutional neural networks for vibrational spectroscopic data analysis[J]. Analytica Chimica Acta, 2017, 954 (1): 22- 31
[10]   VIJAYA ARJUNAN R. ECG signal classification based on statistical features with SVM classification [J]. International Journal of Advances in Signal and Image Sciences, 2016, 2(1): 5-10.
[11]   RAMÓN M M, ATWOOD T, BARBIN S, et al. Signal classification with an SVM-FFT approach for feature extraction in cognitive radio [C]// IEEE MTT-S International Microwave and Optoelectronics Conference. Belem: IEEE, 2009: 286-289.
[12]   SOMAN S High performance EEG signal classification using classifiability and the Twin SVM[J]. Applied Soft Computing, 2015, 30 (1): 305- 318
[13]   RICHHARIYA B, TANVEER M EEG signal classification using universum support vector machine[J]. Expert Systems with Applications, 2018, 106 (1): 169- 182
[14]   LI Y, WEN P P Clustering technique-based least square support vector machine for EEG signal classification[J]. Computer Methods and Programs in Biomedicine, 2011, 104 (3): 358- 372
doi: 10.1016/j.cmpb.2010.11.014
[15]   YANG Z, ZHAO Q, LIU W Neural signal classification using a simplified feature set with nonparametric clustering[J]. Neurocomputing, 2009, 73 (1-3): 412- 422
doi: 10.1016/j.neucom.2009.07.013
[16]   LECUN Y, BOTTOU L, BENGIO Y, et al Gradient-based learning applied to document recognition[J]. Proceedings of the IEEE, 1998, 86 (11): 2278- 2324
doi: 10.1109/5.726791
[17]   KRIZHEVSKY A, SUTSKEVER I, HINTON G E ImageNet classification with deep convolutional neural networks[J]. Advances in Neural Information Processing Systems, 2012, 25 (1): 1097- 1105
[18]   GIRSHICK R, DONAHUE J, DARRELL T, et al. Rich feature hierarchies for accurate object detection and semantic segmentation [C]// Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition. Columbus: IEEE, 2014: 580-587.
[19]   OKAMOTO S, NAGANO H, YAMADA Y Psychophysical dimensions of tactile perception of textures[J]. IEEE Transactions on Haptics, 2012, 6 (1): 81- 93
[20]   MALEK S, MELGANI F, BAZI Y One-dimensional convolutional neural networks for spectroscopic signal regression[J]. Journal of Chemometrics, 2018, 32 (5): e2977
doi: 10.1002/cem.2977
[21]   ULLAH A, ANWAR S M, BILAL M, et al Classification of arrhythmia by using deep learning with 2-D ECG spectral image representation[J]. Remote Sensing, 2020, 12 (10): 1685
doi: 10.3390/rs12101685
[22]   LIAO T W Clustering of time series data: a survey[J]. Pattern Recognition, 2005, 38 (11): 1857- 1874
doi: 10.1016/j.patcog.2005.01.025
[23]   LOTFI A, LANGENSIEPEN C, MAHMOUD S M, et al Smart homes for the elderly dementia sufferers: identification and prediction of abnormal behavior[J]. Journal of Ambient Intelligence and Humanized Computing, 2012, 3 (3): 205- 218
doi: 10.1007/s12652-010-0043-x
[24]   SCH?FER P The BOSS is concerned with time series classification in the presence of noise[J]. Data Mining and Knowledge Discovery, 2015, 29 (6): 1505- 1530
doi: 10.1007/s10618-014-0377-7
[25]   LI T, WU X, ZHANG J Time series clustering model based on DTW for classifying car parks[J]. Algorithms, 2020, 13 (3): 57
doi: 10.3390/a13030057
[26]   KESKIN C, CEMGIL A T, AKARUN L. DTW based clustering to improve hand gesture recognition [C]// International Workshop on Human Behavior Understanding. Heidelberg: Springer, 2011: 72-81.
[27]   MA R, ANGRYK R. Distance and density clustering for time series data [C]// 2017 IEEE International Conference on Data Mining Workshops. New Orleans: IEEE, 2017: 25-32.
[28]   DE BOER P T, KROESE D P, MANNOR S, et al A tutorial on the cross-entropy method[J]. Annals of Operations Research, 2005, 134 (1): 19- 67
doi: 10.1007/s10479-005-5724-z
[1] Pei-zhi WEN,Jun-mou CHEN,Yan-nan XIAO,Ya-yuan WEN,Wen-ming HUANG. Underwater image enhancement algorithm based on GAN and multi-level wavelet CNN[J]. Journal of ZheJiang University (Engineering Science), 2022, 56(2): 213-224.
[2] Tong CHEN,Jian-feng GUO,Xin-zhong HAN,Xue-li XIE,Jian-xiang XI. Visible and infrared image matching method based on generative adversarial model[J]. Journal of ZheJiang University (Engineering Science), 2022, 56(1): 63-74.
[3] Song REN,Qian-wen ZHU,Xin-yue TU,Chao DENG,Xiao-shu WANG. Lining disease identification of highway tunnel based on deep learning[J]. Journal of ZheJiang University (Engineering Science), 2022, 56(1): 92-99.
[4] Hao-ran MA,Ya-bin DING. Calibration method of laser displacement sensor based on binocular vision[J]. Journal of ZheJiang University (Engineering Science), 2021, 55(9): 1634-1642.
[5] Ming-hui YOU,Ya-feng YIN,Lei XIE,Sang-lu LU. User profiling based on activity sensing[J]. Journal of ZheJiang University (Engineering Science), 2021, 55(4): 608-614.
[6] Teng ZHANG,Xin-long JIANG,Yi-qiang CHEN,Qian CHEN,Tao-mian MI,Piu CHAN. Wrist attitude-based Parkinson's disease ON/OFF state assessment after medication[J]. Journal of ZheJiang University (Engineering Science), 2021, 55(4): 639-647.
[7] Ying-jie ZHENG,Song-rong WU,Ruo-yu WEI,Zhen-wei TU,Jin LIAO,Dong LIU. Metro location point matching and false alarm elimination based on FCM algorithm of target image[J]. Journal of ZheJiang University (Engineering Science), 2021, 55(3): 586-593.
[8] Jin-zhen LIU,Fang-fang YE,Hui XIONG. Recognition of multi-class motor imagery EEG signals based on convolutional neural network[J]. Journal of ZheJiang University (Engineering Science), 2021, 55(11): 2054-2066.
[9] Fei-hong LIN,Ji ZHOU,Jian-pei ZHANG,Guo-hua YANG,Qing YANG. Reflective miniature optical electric field sensor[J]. Journal of ZheJiang University (Engineering Science), 2021, 55(11): 2207-2214.
[10] Hua-jian DENG,Hao WANG,Ben-dong WANG,Zhong-he JIN. High-precision compensation and calibration method for four-quadrant analog sun sensor[J]. Journal of ZheJiang University (Engineering Science), 2021, 55(10): 1993-2001.
[11] Qiao-hong CHEN,YI CHEN,Wen-shu Li,Yu-bo JIA. Clothing image classification based on multi-scale SE-Xception[J]. Journal of ZheJiang University (Engineering Science), 2020, 54(9): 1727-1735.
[12] Deng-wen ZHOU,Jin-yue TIAN,Lu-yao MA,Xiu-xiu SUN. Lightweight image semantic segmentation based on multi-level feature cascaded network[J]. Journal of ZheJiang University (Engineering Science), 2020, 54(8): 1516-1524.
[13] Yan-nan ZHANG,Xiao-hong HUANG,Yan MA,Qun CONG. Method with recording text classification based on deep learning[J]. Journal of ZheJiang University (Engineering Science), 2020, 54(7): 1264-1271.
[14] Hai-jin WANG,Zong-yu YIN,Zhen-zheng KE,Ying-jie GUO,Hui-yue DONG. Wear monitoring of helical milling tool based on one-dimensional convolutional neural network[J]. Journal of ZheJiang University (Engineering Science), 2020, 54(5): 931-939.
[15] Ping YANG,Dan WANG,Zi-jian KAGN,Tong LI,Li-hua FU,Yue-ren YU. Prediction model of paroxysmal atrial fibrillation based on pattern recognition and ensemble CNN-LSTM[J]. Journal of ZheJiang University (Engineering Science), 2020, 54(5): 1039-1048.