Please wait a minute...
浙江大学学报(工学版)
浙江大学学报(工学版)  2022, Vol. 56 Issue (4): 702-710    DOI: 10.3785/j.issn.1008-973X.2022.04.009
计算机技术、信息工程     
基于压电薄膜传感器的机器人触觉识别系统
王云灏1(),孙铭会1,*(),辛毅2,张博宣3()
1. 吉林大学 计算机科学与技术学院，吉林 长春 130012
2. 吉林大学 仪器科学与电气工程学院，吉林 长春 130061
3. 美国波特兰州立大学 工程与计算机科学学院，俄勒冈州 波特兰 97201
Robot tactile recognition system based on piezoelectric film sensor
Yun-hao WANG1(),Ming-hui SUN1,*(),Yi XIN2,Bo-xuan ZHANG3()
1. College of Computer Science and Technology, Jilin University, Changchun 130012, China
2. College of Instrument Science and Electrical Engineering, Jilin University, Changchun 130061, China
3. College of Engineering and Computer Science, Portland State University, Portland 97201, USA
 全文: PDF(1558 KB)   HTML
摘要：

为了使机器人通过触觉感知外部环境信息，弥补视听交互信息缺失的不足，根据聚偏氟乙烯（PVDF）材料的压电效应设计开发基于触觉传感器和卷积神经网络的机器人触觉识别系统，能够根据所采集的触觉信号识别出材质类型. 提出基于渐进式级联卷积神经网络的触觉识别算法. 该算法基于卷积神经网络提取机器人传感器的信号特征，包括经过短时傅里叶变换的触觉数据频谱图和信号表征周期内的时域特征. 为了解决特定材质识别混淆的问题，利用K-Medoids聚类算法和动态时间规整（DTW）距离度量算法将分类过程区分为粗、细2个层次，构建渐进式分类模型. 实验表明，设计的触觉传感器对物体材质的平均识别正确率约为97%，机器人能够成功识别触摸到的真实材质，为下一步的探索交互任务奠定基础.
关键词： 机器人触觉;  聚偏氟乙烯(PVDF);  压电薄膜;  卷积神经网络;  传感器;  聚类算法    
Abstract:

A robot tactile recognition system based on tactile sensor and convolutional neural network was designed and developed according to the piezoelectric effect of polyvinylidene fluoride (PVDF) material in order to make the robot sense the external environment information through touch and make up for the lack of audio-visual interaction information. The material type can be identified according to the collected tactile signal. A tactile recognition algorithm was proposed based on the progressive cascade convolutional neural network. The signal features of the robot sensor were extracted based on the convolutional neural network, including the short-time Fourier transform of the tactile data spectrum and the time domain characteristics of the signal representation period. The classification process was divided into coarse and fine levels by K-Medoids clustering algorithm and dynamic time warping (DTW) distance measure algorithm in order to solve the confusion problem of specific material recognition. The progressive classification model was constructed. The experimental results showed that the average recognition accuracy of the tactile sensor was about 97%. The robot can successfully recognize the touched real material, laying a foundation for the next exploration and interaction tasks.
Key words: robot haptics    polyvinylidene fluoride (PVDF)    piezoelectric film    convolutional neural network    sensor    clustering algorithm
收稿日期: 2021-10-24 出版日期: 2022-04-24
CLC:  TP 39  
基金资助: 国家自然科学基金资助项目（61872164）
通讯作者: 孙铭会     E-mail: yunhaow20@mails.jlu.edu.cn;smh@jlu.edu.cn;pedanbox@gmail.com
作者简介: 王云灏 （1998—），男，硕士生，从事智能人机交互、大数据可视化、虚拟现实与增强现实的研究. orcid.org/0000-0002-9569-1830. E-mail： yunhaow20@mails.jlu.edu.cn
服务　　
把本文推荐给朋友
加入引用管理器
E-mail Alert
作者相关文章　　
王云灏
孙铭会
辛毅
张博宣

引用本文:

王云灏,孙铭会,辛毅,张博宣. 基于压电薄膜传感器的机器人触觉识别系统[J]. 浙江大学学报(工学版), 2022, 56(4): 702-710.

Yun-hao WANG,Ming-hui SUN,Yi XIN,Bo-xuan ZHANG. Robot tactile recognition system based on piezoelectric film sensor. Journal of ZheJiang University (Engineering Science), 2022, 56(4): 702-710.

链接本文:

https://www.zjujournals.com/eng/CN/10.3785/j.issn.1008-973X.2022.04.009    或    https://www.zjujournals.com/eng/CN/Y2022/V56/I4/702

图 1  PVDF材料压电效应的示意图
图 2  PVDF传感器的示意图
图 3  材质的心理物理维度评估
图 4  用于识别物体材质的机器人手臂
图 5  获取到的数据形式及处理过程
图 6  信息从时域到频域的转化
图 7  1-D CNN的结构
模块 参数 参数值
卷积层 卷积核维度 3×1×1
卷积层 卷积核步长 2
卷积层 卷积核个数 16
卷积层 卷积层激活函数 tanh
池化层 池化方式 最大池化
池化层 卷积核维度 3×1×1
池化层 卷积核步长 1
表 1  一维卷积神经网络参数设置
图 8  1-D CNN的分类结果混淆矩阵
图 9  2-D CNN的结构
图 10  2-D CNN的分类结果混淆矩阵
图 11  欧氏距离和DTW距离度量方法的对比
图 12  时间规划曲线在代价矩阵中的传递
图 13  DTW 与欧氏距离的聚类结果比较
数据 Acc/%
人工分类（四分类） 36.67
人工分类（五分类） 57.74
算法分类（四分类） 87.33
算法分类（五分类） 99.44
表 2  基于不同聚类方式的数据分类准确率比较
图 14  级联神经网络的信号分类模型架构
图 15  最终分类的混淆矩阵
图 16  相似材质识别结果的量化分析
材质类型 Acc/%
1-D CNN 2-D CNN 渐进式模型
百洁布 95.25 100 100
栅格文件袋 70.32 100 100
硬质纸板 97.54 90.02 90.03
泡沫塑料 35.26 57.44 71.44
无痕布 88.44 100 100
纹理布 82.47 100 93.66
硅胶片 95.52 100 100
塑料片 100 100 100
塑料栅格 38.33 62.15 100
粗糙胶带 68.10 100 100
柔软布 72.05 100 100
软牛仔裤 78.66 100 100
硬质布 68.38 100 100
硬牛仔裤 85.54 57.06 100
毛衣 72.80 100 100
平均准确率 76.57 91.11 97.01
表 3  不同算法对于整体数据的分类准确率
1 ALATISE M B, HANCKE G P A review on challenges of autonomous mobile robot and sensor fusion methods[J]. IEEE Access, 2020, 8 (1): 39830- 39846
2 LIU L, XU Y, ZHU J, et al A flexible thermal sensor based on PVDF film for robot finger skin[J]. Integrated Ferroelectrics, 2019, 201 (1): 23- 31
doi: 10.1080/10584587.2019.1668687
3 EGUíLUZ A G, RA?ó I, COLEMAN S A, et al Multimodal material identification through recursive tactile sensing[J]. Robotics and Autonomous Systems, 2018, 106: 130- 139
doi: 10.1016/j.robot.2018.05.003
4 YANG Y J, CHENG M Y, SHIH S C, et al A 32×32 temperature and tactile sensing array using PI-copper films[J]. The International Journal of Advanced Manufacturing Technology, 2010, 46 (9-12): 945- 956
doi: 10.1007/s00170-009-1940-z
5 HERSHEY S, CHAUDHURI S, ELLIS D P W, et al. CNN architectures for large-scale audio classification [C]// 2017 IEEE International Conference on Acoustics, Speech and Signal Processing. New Orleans: IEEE, 2017: 131-135.
6 WU H, CHEN J, LIU X, et al One-dimensional CNN-based intelligent recognition of vibrations in pipeline monitoring with DAS[J]. Journal of Lightwave Technology, 2019, 37 (17): 4359- 4366
doi: 10.1109/JLT.2019.2923839
7 JANA G C, SHARMA R, AGRAWAL A A 1D-CNN-spectrogram based approach for seizure detection from EEG signal[J]. Procedia Computer Science, 2020, 167 (1): 403- 412
8 INCE T, KIRANYAZ S, EREN L, et al Real-time motor fault detection by 1-D convolutional neural networks[J]. IEEE Transactions on Industrial Electronics, 2016, 63 (11): 7067- 7075
doi: 10.1109/TIE.2016.2582729
9 ACQUARELLI J, VAN LAARHOVEN T, GERRETZEN J, et al Convolutional neural networks for vibrational spectroscopic data analysis[J]. Analytica Chimica Acta, 2017, 954 (1): 22- 31
10 VIJAYA ARJUNAN R. ECG signal classification based on statistical features with SVM classification [J]. International Journal of Advances in Signal and Image Sciences, 2016, 2(1): 5-10.
11 RAMÓN M M, ATWOOD T, BARBIN S, et al. Signal classification with an SVM-FFT approach for feature extraction in cognitive radio [C]// IEEE MTT-S International Microwave and Optoelectronics Conference. Belem: IEEE, 2009: 286-289.
12 SOMAN S High performance EEG signal classification using classifiability and the Twin SVM[J]. Applied Soft Computing, 2015, 30 (1): 305- 318
13 RICHHARIYA B, TANVEER M EEG signal classification using universum support vector machine[J]. Expert Systems with Applications, 2018, 106 (1): 169- 182
14 LI Y, WEN P P Clustering technique-based least square support vector machine for EEG signal classification[J]. Computer Methods and Programs in Biomedicine, 2011, 104 (3): 358- 372
doi: 10.1016/j.cmpb.2010.11.014
15 YANG Z, ZHAO Q, LIU W Neural signal classification using a simplified feature set with nonparametric clustering[J]. Neurocomputing, 2009, 73 (1-3): 412- 422
doi: 10.1016/j.neucom.2009.07.013
16 LECUN Y, BOTTOU L, BENGIO Y, et al Gradient-based learning applied to document recognition[J]. Proceedings of the IEEE, 1998, 86 (11): 2278- 2324
doi: 10.1109/5.726791
17 KRIZHEVSKY A, SUTSKEVER I, HINTON G E ImageNet classification with deep convolutional neural networks[J]. Advances in Neural Information Processing Systems, 2012, 25 (1): 1097- 1105
18 GIRSHICK R, DONAHUE J, DARRELL T, et al. Rich feature hierarchies for accurate object detection and semantic segmentation [C]// Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition. Columbus: IEEE, 2014: 580-587.
19 OKAMOTO S, NAGANO H, YAMADA Y Psychophysical dimensions of tactile perception of textures[J]. IEEE Transactions on Haptics, 2012, 6 (1): 81- 93
20 MALEK S, MELGANI F, BAZI Y One-dimensional convolutional neural networks for spectroscopic signal regression[J]. Journal of Chemometrics, 2018, 32 (5): e2977
doi: 10.1002/cem.2977
21 ULLAH A, ANWAR S M, BILAL M, et al Classification of arrhythmia by using deep learning with 2-D ECG spectral image representation[J]. Remote Sensing, 2020, 12 (10): 1685
doi: 10.3390/rs12101685
22 LIAO T W Clustering of time series data: a survey[J]. Pattern Recognition, 2005, 38 (11): 1857- 1874
doi: 10.1016/j.patcog.2005.01.025
23 LOTFI A, LANGENSIEPEN C, MAHMOUD S M, et al Smart homes for the elderly dementia sufferers: identification and prediction of abnormal behavior[J]. Journal of Ambient Intelligence and Humanized Computing, 2012, 3 (3): 205- 218
doi: 10.1007/s12652-010-0043-x
24 SCH?FER P The BOSS is concerned with time series classification in the presence of noise[J]. Data Mining and Knowledge Discovery, 2015, 29 (6): 1505- 1530
doi: 10.1007/s10618-014-0377-7
25 LI T, WU X, ZHANG J Time series clustering model based on DTW for classifying car parks[J]. Algorithms, 2020, 13 (3): 57
doi: 10.3390/a13030057
26 KESKIN C, CEMGIL A T, AKARUN L. DTW based clustering to improve hand gesture recognition [C]// International Workshop on Human Behavior Understanding. Heidelberg: Springer, 2011: 72-81.
27 MA R, ANGRYK R. Distance and density clustering for time series data [C]// 2017 IEEE International Conference on Data Mining Workshops. New Orleans: IEEE, 2017: 25-32.
28 DE BOER P T, KROESE D P, MANNOR S, et al A tutorial on the cross-entropy method[J]. Annals of Operations Research, 2005, 134 (1): 19- 67
doi: 10.1007/s10479-005-5724-z
[1] 温佩芝,陈君谋,肖雁南,温雅媛,黄文明. 基于生成式对抗网络和多级小波包卷积网络的水下图像增强算法[J]. 浙江大学学报(工学版), 2022, 56(2): 213-224.
[2] 任松,朱倩雯,涂歆玥,邓超,王小书. 基于深度学习的公路隧道衬砌病害识别方法[J]. 浙江大学学报(工学版), 2022, 56(1): 92-99.
[3] 马浩然,丁雅斌. 基于双目视觉的激光位移传感器标定方法[J]. 浙江大学学报(工学版), 2021, 55(9): 1634-1642.
[4] 尤明辉,殷亚凤,谢磊,陆桑璐. 基于行为感知的用户画像技术[J]. 浙江大学学报(工学版), 2021, 55(4): 608-614.
[5] 张腾,蒋鑫龙,陈益强,陈前,米涛免,陈彪. 基于腕部姿态的帕金森病用药后开-关期检测[J]. 浙江大学学报(工学版), 2021, 55(4): 639-647.
[6] 张琦,陈红,周继彪,张敏,郭璘,杨仁法. 道路开口对临近交叉口交通安全的影响[J]. 浙江大学学报(工学版), 2021, 55(4): 720-726.
[7] 郑英杰,吴松荣,韦若禹,涂振威,廖进,刘东. 基于目标图像FCM算法的地铁定位点匹配及误报排除方法[J]. 浙江大学学报(工学版), 2021, 55(3): 586-593.
[8] 刘近贞,叶方方,熊慧. 基于卷积神经网络的多类运动想象脑电信号识别[J]. 浙江大学学报(工学版), 2021, 55(11): 2054-2066.
[9] 林飞宏,周吉,张建培,杨国华,杨青. 反射式小型光学电场传感器[J]. 浙江大学学报(工学版), 2021, 55(11): 2207-2214.
[10] 陈巧红,陈翊,李文书,贾宇波. 多尺度SE-Xception服装图像分类[J]. 浙江大学学报(工学版), 2020, 54(9): 1727-1735.
[11] 周登文,田金月,马路遥,孙秀秀. 基于多级特征并联的轻量级图像语义分割[J]. 浙江大学学报(工学版), 2020, 54(8): 1516-1524.
[12] 张彦楠,黄小红,马严,丛群. 基于深度学习的录音文本分类方法[J]. 浙江大学学报(工学版), 2020, 54(7): 1264-1271.
[13] 闫旭,范晓亮,郑传潘,臧彧,王程,程明,陈龙彪. 基于图卷积神经网络的城市交通态势预测算法[J]. 浙江大学学报(工学版), 2020, 54(6): 1147-1155.
[14] 沈宗礼,余建波. 基于迁移学习与深度森林的晶圆图缺陷识别[J]. 浙江大学学报(工学版), 2020, 54(6): 1228-1239.
[15] 汪海晋,尹宗宇,柯臻铮,郭英杰,董辉跃. 基于一维卷积神经网络的螺旋铣刀具磨损监测[J]. 浙江大学学报(工学版), 2020, 54(5): 931-939.