Please wait a minute...
浙江大学学报(工学版)
Journal of ZheJiang University (Engineering Science)  2020, Vol. 54 Issue (7): 1272-1280    DOI: 10.3785/j.issn.1008-973X.2020.07.004
    
Automatic garbage classification system based on machine vision
Zhuang KANG(),Jie YANG*(),Hao-qi GUO
School of Electrical Engineering and Automation, Jiangxi University of Science and Technology, Ganzhou 341000, China
Download: HTML     PDF(1333KB) HTML
Export: BibTeX | EndNote (RIS)      

Abstract  

An automatic garbage classification system was designed based on machine vision in order to improve the efficiency of front-end collection in garbage classification process. The hardware device of the garbage classification system was designed and manufactured, which mainly included two boxes, the recyclable box and the non-recyclable box. A method of garbage type recognition was proposed based on Inception v3 feature extraction network structure and migration learning aiming at the data lacking problem caused by small garbage data sets. The method was trained and tested on the constructed garbage data set. The test results show that the method can accurately identify garbage types with an average accuracy of 0.99. The trained model was deployed on the raspberry pi 3B+, and tested on the real garbage bin. When the whole system was running stably, the average time for the system to complete the classification of one garbage was 0.95 second. The experimental results show that the automatic garbage classification system can effectively identify the types of garbage and complete the classification and recycling of the garbage.

Key words： artificial intelligence      Inception v3      machine vision      image classification      intelligent trash bin      transfer learning     
Received: 09 February 2020      Published: 05 July 2020
CLC:  TP 399  
Corresponding Authors: Jie YANG     E-mail: zhuangkangxxy@163.com;yangjie@jxust.edu.cn
Service
E-mail this article
Add to my bookshelf
Add to citation manager
E-mail Alert
RSS
Articles by authors
Zhuang KANG
Jie YANG
Hao-qi GUO
Cite this article:

Zhuang KANG,Jie YANG,Hao-qi GUO. Automatic garbage classification system based on machine vision. Journal of ZheJiang University (Engineering Science), 2020, 54(7): 1272-1280.

URL:

http://www.zjujournals.com/eng/10.3785/j.issn.1008-973X.2020.07.004     OR     http://www.zjujournals.com/eng/Y2020/V54/I7/1272


基于机器视觉的垃圾自动分类系统设计

为了提高垃圾分类过程中前端收集的工作效率，基于机器视觉技术设计垃圾自动分类系统. 设计制作垃圾分类的硬件设备，主要包括可回收和不可回收2个箱体；针对垃圾数据集较少的问题，提出基于Inception v3网络特征提取模型和迁移学习相结合的垃圾种类识别方法，在自建的垃圾数据集上进行训练和测试. 结果显示，利用该方法可以准确地对垃圾种类进行识别，平均准确率达到0.99；将训练好的模型部署在树莓派3B+上，在制作的实物垃圾桶上进行测试，系统稳定后，平均完成一次分类回收的时间为0.95 s. 实验表明，该系统能够有效地进行垃圾种类的识别和完成垃圾的分类回收.


关键词： 人工智能,  Inception v3,  机器视觉,  图像分类,  智能垃圾桶,  迁移学习 
Fig.1 Structure of trash bin
Fig.2 Production of trash bin
Fig.3 Servo control circuit
Fig.4 Development flow chart of information monitoring APP for garbage bin
可回收数据集 不可回收数据集
分类 数量 分类 数量
书 217 塑料袋 425
水瓶 417 灯泡 206
毛巾 340 打包盒 296
纸盒 205 电池 276
金属 278 香蕉皮 343
碎玻璃 376 树叶 209
纸团 286 橘子皮 294
Tab.1 Garbage dataset
Fig.5 Network structure of Inception v3
类型 Kernel尺寸/步长 输入尺寸
卷积 3×3/2 299×299×3
卷积 3×3/1 149×149×32
卷积 3×3/1 147×147×32
池化 3×3/2 147×147×64
卷积 3×3/1 73×73×64
卷积 3×3/2 71×71×80
卷积 3×3/1 35×35×192
Inception模块组 3个Inception 模块 35×35×288
Inception模块组 5个Inception 模块 17×17×768
Inception模块组 3个Inception 模块 8×8×1280
池化 8×8 8×8×2048
线性 Logits 1×1×2048
Softmax 分类输出 1×1×1000
Tab.2 Inception v3 model structure
Fig.6 Accuracy and loss of five models
组别 ALeNet AAlexNet AVgg AResNet AInv3
0~10 0.508 0.632 0.655 0.683 0.689
10~20 0.569 0.747 0.785 0.813 0.839
20~30 0.589 0.784 0.820 0.848 0.900
30~40 0.603 0.806 0.841 0.869 0.926
40~50 0.614 0.821 0.853 0.881 0.939
50~60 0.622 0.830 0.860 0.888 0.945
60~70 0.628 0.841 0.868 0.896 0.950
70~80 0.633 0.847 0.874 0.902 0.953
80~90 0.637 0.854 0.877 0.905 0.955
90~100 0.640 0.859 0.879 0.907 0.957
Tab.3 Comparison of average accuracy of five models
组别 LLeNet LAlexNet LVgg LResNet LInv3
0~10 1.400 1.046 0.971 0.911 0.874
10~20 1.234 0.723 0.615 0.555 0.456
20~30 1.176 0.616 0.509 0.449 0.283
30~40 1.137 0.551 0.449 0.389 0.212
40~50 1.108 0.507 0.414 0.354 0.180
50~60 1.086 0.473 0.392 0.332 0.169
60~70 1.069 0.447 0.370 0.310 0.160
70~80 1.053 0.424 0.355 0.295 0.155
80~90 1.041 0.406 0.343 0.283 0.153
90~100 1.031 0.389 0.338 0.278 0.154
Tab.4 Comparison of average loss of five models
Fig.7 Train results of improved Inception v3
Fig.8 Comparison of training accuracy of two models
模型 Pall Ptr Pnt Rtr/%
Inception v3 23 915 310 23 880 878 34 432 99.86
迁移学习 23 915 310 2 112 526 21 802 784 8.83
Tab.5 Training parameters of two models
可回收数据集 不可回收数据集
预测值 真实值 预测值 真实值
书 书 塑料袋 塑料袋
水瓶 水瓶 灯泡 灯泡
毛巾 毛巾 打包盒 打包盒
纸盒 纸盒 电池 电池
金属 金属 香蕉皮 香蕉皮
碎玻璃 碎玻璃 树叶 树叶
纸团 纸团 橘子皮 橘子皮
Tab.6 Test results of improved Inception v3
Fig.9 Identification speed when the system works stably
Fig.10 Lighting equipment test result
Fig.11 Screenshot of APP
Fig.12 Recognition and test results of garbage pictures
种类 识别分类结果 半圆式挡板 定轴拨板 分类箱体
Book 可回收 不动作 动作 可回收
Plastic 不可回收 动作 不动作 不可回收
Bottle 可回收 不动作 动作 可回收
Bulb 不可回收 动作 不动作 不可回收
Towel 可回收 不动作 动作 可回收
Packing bag 不可回收 动作 不动作 不可回收
Carton 可回收 不动作 动作 可回收
Battery 不可回收 动作 不动作 不可回收
Metal 可回收 不动作 动作 可回收
Banana peel 不可回收 动作 不动作 不可回收
Broken glass 可回收 不动作 动作 可回收
Leaf 不可回收 动作 不动作 不可回收
Paper ball 可回收 不动作 动作 可回收
Orange peel 不可回收 动作 不动作 不可回收
Tab.7 Test results of garbage classfication system
[1]   王佳佳, 赵娜娜, 李金惠 中国海洋微塑料污染现状与防治建议[J]. 中国环境科学, 2019, 39 (7): 3056- 3063
WANG Jia-jia, ZHAO Na-na, LI Jin-hui Present status and prevention of marine microplastics pollution in China[J]. China Environmental Science, 2019, 39 (7): 3056- 3063
doi: 10.3969/j.issn.1000-6923.2019.07.043
[2]   李文瀚, 马增益, 杨恩权, 等 循环流化床垃圾焚烧系统电除尘飞灰和布袋飞灰特性研究[J]. 中国电机工程学报, 2019, 39 (5): 1397- 1405
LI Wen-han, MA Zeng-yi, YANG En-quan, et al Study on characteristics of electric dust removal fly ash and bag fly ash in circulating fluidized bed waste incineration system[J]. Chinese Journal of Electrical Engineering, 2019, 39 (5): 1397- 1405
[3]   任越, 杨俊杰, 曹洪军, 等 农村生活垃圾全自动全组分分类处理技术与应用[J]. 农业工程学报, 2019, 35 (4): 248- 254
REN Yue, YANG Jun-jie, CAO Hong-jun, et al All components resourcing system of rural garbage based on post-gather automatic sorting and disposal technology and its application[J]. Transactions of the Chinese Society of Agricultural Engineering, 2019, 35 (4): 248- 254
doi: 10.11975/j.issn.1002-6819.2019.04.031
[4]   赵冬娥, 吴瑞, 赵宝国, 等 高光谱成像的垃圾分类识别研究[J]. 光谱学与光谱分析, 2019, 39 (3): 921- 926
ZHAO Dong-e, WU Rui, ZHAO Bao-guo, et al Research on garbage classification and recognition of hyperspectral imaging[J]. Spectroscopy and Spectral Analysis, 2019, 39 (3): 921- 926
[5]   潘鹏, 胡运峰, 秦金志, 等 一种基于NB-IOT的智能垃圾桶[J]. 科技风, 2019, (1): 11
PAN Peng, HU Yun-feng, QIN Jin-zhi, et al A smart trash can based on NB-IOT[J]. Science and Technology, 2019, (1): 11
[6]   陈景翼, 李春燕, 盛宇彪, 等 基于GPRS通讯的智能垃圾桶硬件电路设计[J]. 电工技术, 2019, (2): 18- 20
CHEN Jing-yi, LI Chun-yan, SHENG Yu-biao, et al Intelligent hardware design of intelligent trash can based on GPRS communication[J]. Electric Engineering, 2019, (2): 18- 20
doi: 10.3969/j.issn.1002-1388.2019.02.008
[7]   白桂峰 基于STC89C52的家用智能垃圾桶设计[J]. 自动化与仪表, 2019, 34 (3): 97- 100
BAI Gui-feng Design of household intelligent wastebin based on STC89C52[J]. Automation and Instrumentation, 2019, 34 (3): 97- 100
[8]   SALIMI I, DEWANTARA B S B, WIBOWO I K. Visual-based trash detection and classification system for smart trash bin robot [C]//2018 International Electronics Symposium on Knowledge Creation and Intelligent Computing. Bali: IEEE, 2018: 378-383.
[9]   SUDHA S, VIDHYALAKSHMI M, PAVITHRA K, et al. An automatic classification method for environment: friendly waste segregation using deep learning [C]//2016 IEEE Technological Innovations in ICT for Agriculture and Rural Development. Chennai: IEEE, 2016: 65-70.
[10]   SETIAWAN W, WAHYUDIN A, WIDIANTO G R. The use of scale invariant feature transform (SIFT) algorithms to identification garbage images based on product label [C]//2017 3rd International Conference on Science in Information Technology. Bandung: IEEE, 2017: 336-341.
[11]   PAPALITSAS C, ANDRONIKOS T Unconventional GVNS for solving the garbage collection problem with time windows[J]. Technologies, 2019, 7 (3): 61
doi: 10.3390/technologies7030061
[12]   JAID JIM A A, KADIR R, MAMUN M A A, et al A noble proposal for Internet of garbage bins (IoGB)[J]. Smart Citie, 2019, 2 (2): 214- 229
doi: 10.3390/smartcities2020014
[13]   党宇, 张继贤, 邓喀中, 等 基于深度学习AlexNet的遥感影像地表覆盖分类评价研究[J]. 地球信息科学学报, 2017, 19 (11): 1530- 1537
DANG Yu, ZHANG Ji-xian, DENG Ka-zhong, et al Study on the evaluation of land cover classification using remote sensing images based on AlexNet[J]. Journal of Geo Information Science, 2017, 19 (11): 1530- 1537
[14]   李余兴, 李亚安, 陈晓, 等 基于VMD和SVM的舰船辐射噪声特征提取及分类识别[J]. 国防科技大学学报, 2019, 41 (1): 89- 94
LI Yu-xing, LI Ya-an, CHEN Xiao, et al Feature extraction and classification identification of ship radiated noise based on VMD and SVM[J]. Journal of National University of Defense Technology, 2019, 41 (1): 89- 94
doi: 10.11887/j.cn.201901013
[15]   周玉阳, 秦科. 改进的VGG网络可提升年龄与性别预测准确率[J/OL]. 计算机工程与应用, 2019, 55(18): 173-179. [2019-08-08]. http://kns.cnki.net/kcms/detail/11.2127.TP.20181108.1712.009.html.
ZHOU Yu-yang, QIN Ke. Improved VGG network can improve the accuracy of age and gender predicttion [J/OL]. Computer Engineering and Application, 2019, 55(18): 173-179. [2019-08-08]. http://kns.cnki.net/kcms/detail/11.2127.TP.20181108.1712.009.html.
[16]   ZENG D, ZHANG S, CHEN F, et al Multi-scale CNN based garbage detection of airborne hyperspectral data[J]. IEEE Access, 2019, (7): 104514- 104527
[17]   RABANO S L, CABATUAN M K, SYBINGCO E, et al. Common garbage classification using mobilenet [C]//2018 IEEE 10th International Conference on Humanoid, Nanotechnology, Information Technology, Communication and Control, Environment and Management. Baguio City, Philippines: IEEE, 2018: 1-4.
[18]   XIONG J, CUI W, ZHANG W, et al. YOLOv3-darknet with adaptive clustering anchor box for intelligent dry and wet garbage identification and classification [C]//2019 11th International Conference on Intelligent Human-Machine Systems and Cybernetics. Hangzhou: IEEE, 2019: 80-84.
[19]   向伟, 史晋芳, 刘桂华, 等 改进CaffeNet模型在水面垃圾识别中的应用[J]. 传感器与微系统, 2019, 38 (8): 150- 152
XIANG Wei, SHI Jin-fang, LIU Gui-hua, et al Application of improved CaffeNet model in water surface garbage identification[J]. Transducer and Microsystem Technologies, 2019, 38 (8): 150- 152
[20]   廖祥文, 吴晓静, 桂林, 等 结合表示学习和迁移学习的跨领域情感分类[J]. 北京大学学报: 自然科学版, 2019, 55 (1): 37- 46
LIAO Xiang-wen, WU Xiao-jing, GUI Lin, et al Cross-domain sentiment classification combined with learning and migration learning[J]. Journal of Peking University: Natural Science, 2019, 55 (1): 37- 46
[21]   张野, 李明超, 韩帅 基于岩石图像深度学习的岩性自动识别与分类方法[J]. 岩石学报, 2018, 34 (2): 333- 342
ZHANG Ye, LI Ming-chao, HAN Shuai Automatic identification and classification in lithology based on deep learning in rock images[J]. Acta Petrologica Sinica, 2018, 34 (2): 333- 342
[22]   李冠东, 张春菊, 王铭恺 卷积神经网络迁移的高分影像场景分类学习[J]. 测绘科学, 2019, 44 (4): 116- 123
LI Guan-dong, ZHANG Chun-ju, WANG Ming-kai Classification learning of high score image scenes based on convolutional neural network migration[J]. Science and Mapping Science, 2019, 44 (4): 116- 123
[23]   GUO W, LI W, YANG B, et al A novel noninvasive and cost-effective handheld detector on soluble solids content of fruits[J]. Journal of Food Engineering, 2019, 257: 1- 9
doi: 10.1016/j.jfoodeng.2019.03.022
[24]   BEASON R D, RIESCH R, KORICHEVA J AURITA: an affo-rdable, autonomous recording device for acoustic monitoring of audible and ultrasonic frequencies[J]. Bioacoustics, 2018, 28: 381- 396
[1] Xun CHENG,Jian-bo YU. Monitoring method for machining tool wear based on machine vision[J]. Journal of ZheJiang University (Engineering Science), 2021, 55(5): 896-904.
[2] Qiao-hong CHEN,YI CHEN,Wen-shu Li,Yu-bo JIA. Clothing image classification based on multi-scale SE-Xception[J]. Journal of ZheJiang University (Engineering Science), 2020, 54(9): 1727-1735.
[3] Jun DU,Chen MA,Zheng-ying WEI. Dynamic response of surface morphology of aluminum (Al) deposited layers in wire and arc additive manufacturing based on visual sensing[J]. Journal of ZheJiang University (Engineering Science), 2020, 54(8): 1481-1489.
[4] Qi SHEN,Yan ZHAO,Xiao-wei ZHOU,Xiao-ran YUAN. Image Hashing algorithm based on structure and gradient[J]. Journal of ZheJiang University (Engineering Science), 2020, 54(8): 1525-1533.
[5] Tao MING,Dan WANG,Ji-chang GUO,Qiang LI. Breast cancer histopathological image classification using multi-scale channel squeeze-and-excitation model[J]. Journal of ZheJiang University (Engineering Science), 2020, 54(7): 1289-1297.
[6] Zong-li SHEN,Jian-bo YU. Wafer map defect recognition based on transfer learning and deep forest[J]. Journal of ZheJiang University (Engineering Science), 2020, 54(6): 1228-1239.
[7] Hui-fang WANG,Chen-yu ZHANG. Prediction of voltage stability margin in power system based on extreme gradient boosting algorithm[J]. Journal of ZheJiang University (Engineering Science), 2020, 54(3): 606-613.
[8] Xiao-feng FU,Li NIU,Zhuo-qun HU,Jian-jun LI,Qing WU. Deep micro-expression spotting network training based on concept of transition frame[J]. Journal of ZheJiang University (Engineering Science), 2020, 54(11): 2128-2137.
[9] Hong-xia WANG,Jia-qi ZHOU,Cheng-hao GU,Hong LIN. Design of activation function in CNN for image classification[J]. Journal of ZheJiang University (Engineering Science), 2019, 53(7): 1363-1373.
[10] BI Yun bo, XU Chao, Fan Xin tian, YAN Wei miao. Method of countersink perpendicularity detection using vision measurement[J]. Journal of ZheJiang University (Engineering Science), 2017, 51(2): 312-318.
[11] ZOU Bei-ji, GUO Jian-jing, ZHU Cheng-zhang, YANG Wen-Jun, WU Hui, HE Qi. Image classification based on BOW-HOG feature[J]. Journal of ZheJiang University (Engineering Science), 2017, 51(12): 2311-2319.
[12] LIU Ya nan, NI He peng, ZHANG Cheng rui WANG Yun fei; SUN Hao chun. PC-based open control platform design of integration of machine vision and motion control[J]. Journal of ZheJiang University (Engineering Science), 2016, 50(7): 1381-1386.
[13] YE Xiao-wei, ZHANG Xiao-ming, NI Yi-qing, WONG Kai-yuen, FAN Ke-qing. Bridge deflection measurement method based on machine vision technology[J]. Journal of ZheJiang University (Engineering Science), 2014, 48(5): 813-819.
[14] XU Song,SUN Xiu-xia,HE Yan. Iterative method of camera distortion calibration utilizing lines-imaging characteristics[J]. Journal of ZheJiang University (Engineering Science), 2014, 48(3): 404-413.
[15] LI Xiao-run, ZHU Jie-er, WANG Jing, ZHAO Liao-ying. Hyperspectral image classification based on compsite kernels support vector machine[J]. Journal of ZheJiang University (Engineering Science), 2013, 47(8): 1403-1410.