An automatic garbage classification system was designed based on machine vision in order to improve the efficiency of front-end collection in garbage classification process. The hardware device of the garbage classification system was designed and manufactured, which mainly included two boxes, the recyclable box and the non-recyclable box. A method of garbage type recognition was proposed based on Inception v3 feature extraction network structure and migration learning aiming at the data lacking problem caused by small garbage data sets. The method was trained and tested on the constructed garbage data set. The test results show that the method can accurately identify garbage types with an average accuracy of 0.99. The trained model was deployed on the raspberry pi 3B+, and tested on the real garbage bin. When the whole system was running stably, the average time for the system to complete the classification of one garbage was 0.95 second. The experimental results show that the automatic garbage classification system can effectively identify the types of garbage and complete the classification and recycling of the garbage.
Zhuang KANG,Jie YANG,Hao-qi GUO. Automatic garbage classification system based on machine vision. Journal of ZheJiang University (Engineering Science), 2020, 54(7): 1272-1280.
为了提高垃圾分类过程中前端收集的工作效率，基于机器视觉技术设计垃圾自动分类系统. 设计制作垃圾分类的硬件设备，主要包括可回收和不可回收2个箱体；针对垃圾数据集较少的问题，提出基于Inception v3网络特征提取模型和迁移学习相结合的垃圾种类识别方法，在自建的垃圾数据集上进行训练和测试. 结果显示，利用该方法可以准确地对垃圾种类进行识别，平均准确率达到0.99；将训练好的模型部署在树莓派3B+上，在制作的实物垃圾桶上进行测试，系统稳定后，平均完成一次分类回收的时间为0.95 s. 实验表明，该系统能够有效地进行垃圾种类的识别和完成垃圾的分类回收.
关键词：
人工智能,
Inception v3,
机器视觉,
图像分类,
智能垃圾桶,
迁移学习
Fig.1Structure of trash bin
Fig.2Production of trash bin
Fig.3Servo control circuit
Fig.4Development flow chart of information monitoring APP for garbage bin
可回收数据集
不可回收数据集
分类
数量
分类
数量
书
217
塑料袋
425
水瓶
417
灯泡
206
毛巾
340
打包盒
296
纸盒
205
电池
276
金属
278
香蕉皮
343
碎玻璃
376
树叶
209
纸团
286
橘子皮
294
Tab.1Garbage dataset
Fig.5Network structure of Inception v3
类型
Kernel尺寸/步长
输入尺寸
卷积
3×3/2
299×299×3
卷积
3×3/1
149×149×32
卷积
3×3/1
147×147×32
池化
3×3/2
147×147×64
卷积
3×3/1
73×73×64
卷积
3×3/2
71×71×80
卷积
3×3/1
35×35×192
Inception模块组
3个Inception 模块
35×35×288
Inception模块组
5个Inception 模块
17×17×768
Inception模块组
3个Inception 模块
8×8×1280
池化
8×8
8×8×2048
线性
Logits
1×1×2048
Softmax
分类输出
1×1×1000
Tab.2Inception v3 model structure
Fig.6Accuracy and loss of five models
组别
ALeNet
AAlexNet
AVgg
AResNet
AInv3
0~10
0.508
0.632
0.655
0.683
0.689
10~20
0.569
0.747
0.785
0.813
0.839
20~30
0.589
0.784
0.820
0.848
0.900
30~40
0.603
0.806
0.841
0.869
0.926
40~50
0.614
0.821
0.853
0.881
0.939
50~60
0.622
0.830
0.860
0.888
0.945
60~70
0.628
0.841
0.868
0.896
0.950
70~80
0.633
0.847
0.874
0.902
0.953
80~90
0.637
0.854
0.877
0.905
0.955
90~100
0.640
0.859
0.879
0.907
0.957
Tab.3Comparison of average accuracy of five models
组别
LLeNet
LAlexNet
LVgg
LResNet
LInv3
0~10
1.400
1.046
0.971
0.911
0.874
10~20
1.234
0.723
0.615
0.555
0.456
20~30
1.176
0.616
0.509
0.449
0.283
30~40
1.137
0.551
0.449
0.389
0.212
40~50
1.108
0.507
0.414
0.354
0.180
50~60
1.086
0.473
0.392
0.332
0.169
60~70
1.069
0.447
0.370
0.310
0.160
70~80
1.053
0.424
0.355
0.295
0.155
80~90
1.041
0.406
0.343
0.283
0.153
90~100
1.031
0.389
0.338
0.278
0.154
Tab.4Comparison of average loss of five models
Fig.7Train results of improved Inception v3
Fig.8Comparison of training accuracy of two models
模型
Pall
Ptr
Pnt
Rtr/%
Inception v3
23 915 310
23 880 878
34 432
99.86
迁移学习
23 915 310
2 112 526
21 802 784
8.83
Tab.5Training parameters of two models
可回收数据集
不可回收数据集
预测值
真实值
预测值
真实值
书
书
塑料袋
塑料袋
水瓶
水瓶
灯泡
灯泡
毛巾
毛巾
打包盒
打包盒
纸盒
纸盒
电池
电池
金属
金属
香蕉皮
香蕉皮
碎玻璃
碎玻璃
树叶
树叶
纸团
纸团
橘子皮
橘子皮
Tab.6Test results of improved Inception v3
Fig.9Identification speed when the system works stably
Fig.10Lighting equipment test result
Fig.11Screenshot of APP
Fig.12Recognition and test results of garbage pictures
种类
识别分类结果
半圆式挡板
定轴拨板
分类箱体
Book
可回收
不动作
动作
可回收
Plastic
不可回收
动作
不动作
不可回收
Bottle
可回收
不动作
动作
可回收
Bulb
不可回收
动作
不动作
不可回收
Towel
可回收
不动作
动作
可回收
Packing bag
不可回收
动作
不动作
不可回收
Carton
可回收
不动作
动作
可回收
Battery
不可回收
动作
不动作
不可回收
Metal
可回收
不动作
动作
可回收
Banana peel
不可回收
动作
不动作
不可回收
Broken glass
可回收
不动作
动作
可回收
Leaf
不可回收
动作
不动作
不可回收
Paper ball
可回收
不动作
动作
可回收
Orange peel
不可回收
动作
不动作
不可回收
Tab.7Test results of garbage classfication system
[1]
王佳佳, 赵娜娜, 李金惠 中国海洋微塑料污染现状与防治建议[J]. 中国环境科学, 2019, 39 (7): 3056- 3063 WANG Jia-jia, ZHAO Na-na, LI Jin-hui Present status and prevention of marine microplastics pollution in China[J]. China Environmental Science, 2019, 39 (7): 3056- 3063
doi: 10.3969/j.issn.1000-6923.2019.07.043
[2]
李文瀚, 马增益, 杨恩权, 等 循环流化床垃圾焚烧系统电除尘飞灰和布袋飞灰特性研究[J]. 中国电机工程学报, 2019, 39 (5): 1397- 1405 LI Wen-han, MA Zeng-yi, YANG En-quan, et al Study on characteristics of electric dust removal fly ash and bag fly ash in circulating fluidized bed waste incineration system[J]. Chinese Journal of Electrical Engineering, 2019, 39 (5): 1397- 1405
[3]
任越, 杨俊杰, 曹洪军, 等 农村生活垃圾全自动全组分分类处理技术与应用[J]. 农业工程学报, 2019, 35 (4): 248- 254 REN Yue, YANG Jun-jie, CAO Hong-jun, et al All components resourcing system of rural garbage based on post-gather automatic sorting and disposal technology and its application[J]. Transactions of the Chinese Society of Agricultural Engineering, 2019, 35 (4): 248- 254
doi: 10.11975/j.issn.1002-6819.2019.04.031
[4]
赵冬娥, 吴瑞, 赵宝国, 等 高光谱成像的垃圾分类识别研究[J]. 光谱学与光谱分析, 2019, 39 (3): 921- 926 ZHAO Dong-e, WU Rui, ZHAO Bao-guo, et al Research on garbage classification and recognition of hyperspectral imaging[J]. Spectroscopy and Spectral Analysis, 2019, 39 (3): 921- 926
[5]
潘鹏, 胡运峰, 秦金志, 等 一种基于NB-IOT的智能垃圾桶[J]. 科技风, 2019, (1): 11 PAN Peng, HU Yun-feng, QIN Jin-zhi, et al A smart trash can based on NB-IOT[J]. Science and Technology, 2019, (1): 11
[6]
陈景翼, 李春燕, 盛宇彪, 等 基于GPRS通讯的智能垃圾桶硬件电路设计[J]. 电工技术, 2019, (2): 18- 20 CHEN Jing-yi, LI Chun-yan, SHENG Yu-biao, et al Intelligent hardware design of intelligent trash can based on GPRS communication[J]. Electric Engineering, 2019, (2): 18- 20
doi: 10.3969/j.issn.1002-1388.2019.02.008
[7]
白桂峰 基于STC89C52的家用智能垃圾桶设计[J]. 自动化与仪表, 2019, 34 (3): 97- 100 BAI Gui-feng Design of household intelligent wastebin based on STC89C52[J]. Automation and Instrumentation, 2019, 34 (3): 97- 100
[8]
SALIMI I, DEWANTARA B S B, WIBOWO I K. Visual-based trash detection and classification system for smart trash bin robot [C]//2018 International Electronics Symposium on Knowledge Creation and Intelligent Computing. Bali: IEEE, 2018: 378-383.
[9]
SUDHA S, VIDHYALAKSHMI M, PAVITHRA K, et al. An automatic classification method for environment: friendly waste segregation using deep learning [C]//2016 IEEE Technological Innovations in ICT for Agriculture and Rural Development. Chennai: IEEE, 2016: 65-70.
[10]
SETIAWAN W, WAHYUDIN A, WIDIANTO G R. The use of scale invariant feature transform (SIFT) algorithms to identification garbage images based on product label [C]//2017 3rd International Conference on Science in Information Technology. Bandung: IEEE, 2017: 336-341.
[11]
PAPALITSAS C, ANDRONIKOS T Unconventional GVNS for solving the garbage collection problem with time windows[J]. Technologies, 2019, 7 (3): 61
doi: 10.3390/technologies7030061
[12]
JAID JIM A A, KADIR R, MAMUN M A A, et al A noble proposal for Internet of garbage bins (IoGB)[J]. Smart Citie, 2019, 2 (2): 214- 229
doi: 10.3390/smartcities2020014
[13]
党宇, 张继贤, 邓喀中, 等 基于深度学习AlexNet的遥感影像地表覆盖分类评价研究[J]. 地球信息科学学报, 2017, 19 (11): 1530- 1537 DANG Yu, ZHANG Ji-xian, DENG Ka-zhong, et al Study on the evaluation of land cover classification using remote sensing images based on AlexNet[J]. Journal of Geo Information Science, 2017, 19 (11): 1530- 1537
[14]
李余兴, 李亚安, 陈晓, 等 基于VMD和SVM的舰船辐射噪声特征提取及分类识别[J]. 国防科技大学学报, 2019, 41 (1): 89- 94 LI Yu-xing, LI Ya-an, CHEN Xiao, et al Feature extraction and classification identification of ship radiated noise based on VMD and SVM[J]. Journal of National University of Defense Technology, 2019, 41 (1): 89- 94
doi: 10.11887/j.cn.201901013
[15]
周玉阳, 秦科. 改进的VGG网络可提升年龄与性别预测准确率[J/OL]. 计算机工程与应用, 2019, 55(18): 173-179. [2019-08-08]. http://kns.cnki.net/kcms/detail/11.2127.TP.20181108.1712.009.html. ZHOU Yu-yang, QIN Ke. Improved VGG network can improve the accuracy of age and gender predicttion [J/OL]. Computer Engineering and Application, 2019, 55(18): 173-179. [2019-08-08]. http://kns.cnki.net/kcms/detail/11.2127.TP.20181108.1712.009.html.
[16]
ZENG D, ZHANG S, CHEN F, et al Multi-scale CNN based garbage detection of airborne hyperspectral data[J]. IEEE Access, 2019, (7): 104514- 104527
[17]
RABANO S L, CABATUAN M K, SYBINGCO E, et al. Common garbage classification using mobilenet [C]//2018 IEEE 10th International Conference on Humanoid, Nanotechnology, Information Technology, Communication and Control, Environment and Management. Baguio City, Philippines: IEEE, 2018: 1-4.
[18]
XIONG J, CUI W, ZHANG W, et al. YOLOv3-darknet with adaptive clustering anchor box for intelligent dry and wet garbage identification and classification [C]//2019 11th International Conference on Intelligent Human-Machine Systems and Cybernetics. Hangzhou: IEEE, 2019: 80-84.
[19]
向伟, 史晋芳, 刘桂华, 等 改进CaffeNet模型在水面垃圾识别中的应用[J]. 传感器与微系统, 2019, 38 (8): 150- 152 XIANG Wei, SHI Jin-fang, LIU Gui-hua, et al Application of improved CaffeNet model in water surface garbage identification[J]. Transducer and Microsystem Technologies, 2019, 38 (8): 150- 152
[20]
廖祥文, 吴晓静, 桂林, 等 结合表示学习和迁移学习的跨领域情感分类[J]. 北京大学学报: 自然科学版, 2019, 55 (1): 37- 46 LIAO Xiang-wen, WU Xiao-jing, GUI Lin, et al Cross-domain sentiment classification combined with learning and migration learning[J]. Journal of Peking University: Natural Science, 2019, 55 (1): 37- 46
[21]
张野, 李明超, 韩帅 基于岩石图像深度学习的岩性自动识别与分类方法[J]. 岩石学报, 2018, 34 (2): 333- 342 ZHANG Ye, LI Ming-chao, HAN Shuai Automatic identification and classification in lithology based on deep learning in rock images[J]. Acta Petrologica Sinica, 2018, 34 (2): 333- 342
[22]
李冠东, 张春菊, 王铭恺 卷积神经网络迁移的高分影像场景分类学习[J]. 测绘科学, 2019, 44 (4): 116- 123 LI Guan-dong, ZHANG Chun-ju, WANG Ming-kai Classification learning of high score image scenes based on convolutional neural network migration[J]. Science and Mapping Science, 2019, 44 (4): 116- 123
[23]
GUO W, LI W, YANG B, et al A novel noninvasive and cost-effective handheld detector on soluble solids content of fruits[J]. Journal of Food Engineering, 2019, 257: 1- 9
doi: 10.1016/j.jfoodeng.2019.03.022
[24]
BEASON R D, RIESCH R, KORICHEVA J AURITA: an affo-rdable, autonomous recording device for acoustic monitoring of audible and ultrasonic frequencies[J]. Bioacoustics, 2018, 28: 381- 396
ZOU Bei-ji, GUO Jian-jing, ZHU Cheng-zhang, YANG Wen-Jun, WU Hui, HE Qi. Image classification based on BOW-HOG feature[J]. Journal of ZheJiang University (Engineering Science), 2017, 51(12): 2311-2319.
