Please wait a minute...
Journal of ZheJiang University (Engineering Science)  2021, Vol. 55 Issue (8): 1490-1499    DOI: 10.3785/j.issn.1008-973X.2021.08.010
    
Garbage image classification algorithm based on improved MobileNet v2
Zhi-chao CHEN1,2(),Hai-ning JIAO1,2,*(),Jie YANG1,2,Hua-fu ZENG1,2
1. School of Electrical Engineering and Automation, Jiangxi University of Science and Technology, Ganzhou 341000, China
2. Jiangxi Provincial Key Laboratory of Maglev Technology, Ganzhou 341000, China
Download: HTML     PDF(1439KB) HTML
Export: BibTeX | EndNote (RIS)      

Abstract  

A garbage image classification method based on improved MobileNet v2 was proposed aiming at the problems of poor real-time performance and low classification accuracy of existing garbage image classification models. A lightweight feature extraction network based on MobileNet v2 was constructed. The parameter numbers of the model were reduced by adjusting its width factor, channel and spatial attention modules were embedded in the model to enhance the network's ability to refine features, a multi-scale feature fusion structure was designed to enhance the adaptability of the network to scale, and transfer learning was used to optimize the model parameters to further improve the model accuracy. Experimental results show that the average accuracy of the algorithm on the self built dataset was 94.6%, which was 2.0%, 3.4%, 3.2%, 2.3% and 1.2% higher than that of MobileNet v2, VGG16, GoogleNet, ResNet50 and ResNet101 models, respectively. The proposed algorithm achieved good performance in two public image classification datasets, CIFAR-100 and tiny-ImageNet. The parameter numbers of the model was only 0.83 M, which was about 2/5 of the basic model. The single inference on edge device JETSON TX2 took 68 ms, which proved the improvement of inference speed and prediction accuracy.



Key wordsgarbage image classification      MobileNet v2      attention mechanism      multi scale feature fusion      transfer learning     
Received: 18 March 2021      Published: 01 September 2021
CLC:  TP 399  
Fund:  国家自然科学基金资助项目(61763016);江西省03专项及5G资助项目(20204ABC03A15)
Corresponding Authors: Hai-ning JIAO     E-mail: chenzhichao_ai@163.com;jiaohaining@yeah.net
Cite this article:

Zhi-chao CHEN,Hai-ning JIAO,Jie YANG,Hua-fu ZENG. Garbage image classification algorithm based on improved MobileNet v2. Journal of ZheJiang University (Engineering Science), 2021, 55(8): 1490-1499.

URL:

https://www.zjujournals.com/eng/10.3785/j.issn.1008-973X.2021.08.010     OR     https://www.zjujournals.com/eng/Y2021/V55/I8/1490


基于改进MobileNet v2的垃圾图像分类算法

针对现有的垃圾图像分类模型实时性能差和分类精度低的问题,提出基于改进MobileNet v2的垃圾图像分类方法,构建以MobileNet v2为核心的轻量级特征提取网络. 通过调整宽度因子降低模型的参数量;在模型中嵌入通道和空间注意力模块,增强网络对特征的细化能力;设计多尺度特征融合结构,增强网络对尺度的适应性;利用迁移学习的方式优化模型参数,进一步提高模型精度. 实验结果表明,算法在自建数据集上的平均准确率为94.6%,分别高于MobileNet v2、VGG16、GoogleNet、ResNet50、ResNet101模型2.0%、3.4%、3.2%、2.3%、1.2%;所提算法在2种公共图像分类数据集CIFAR-100和tiny-ImageNet中均取得不错表现;模型参数量仅为0.83 M,体积约为基础模型的2/5,在边缘设备JETSON TX2上的单次推理耗时68 ms,实现了推理速度和预测准确率的提升.


关键词: 垃圾图像分类,  MobileNet v2,  注意力机制,  多尺度特征融合,  迁移学习 
Fig.1 Overall network framework of improved MobileNet v2
输入尺寸 操作 t c n s
2242×3 卷积 ? 16 ? 2
1122×16 Bottleneck_A 1 8 1 1
1122×8 Bottleneck_I 6 8 1 2
562×8 Bottleneck_A 6 12 1 1
562×12 Bottleneck_I 6 12 1 2
282×12 Bottleneck_A 6 16 2 1
282×16 Bottleneck_I 6 16 1 2
142×16 Bottleneck_A 6 32 3 1
142×32 Bottleneck_A 6 48 3 1
142×48 Bottleneck_I 6 48 1 2
72×48 Bottleneck_A 6 80 2 1
72×80 Bottleneck_A 6 160 1 1
72×160 PW卷积 ? 640 ? 1
72×640 平均池化 ? 640 ? 1
72×640 PW卷积 ? 14 ? 1
Tab.1 Network structure of improved MobileNet v2
Fig.2 Principle of deep separable convolution
Fig.3 Bottleneck structure under two strides
模型 Atop1/% GFL/M P/M
MobileNet(1.00)[14] 70.6 569 4.2
MobileNet(0.75)[14] 68.4 325 2.6
MobileNet(0.50)[14] 63.7 149 1.3
Tab.2 MobileNet performance under different width factors
Fig.4 Channel and spatial attention mechanisms
Fig.5 Reverse residual module with attention mechanism (Bottleneck_A)
Fig.6 Inception v1 module
Fig.7 Multi-scale feature fusion module (Bottleneck_I)
垃圾名称 数目/张 垃圾名称 数目/张
毛巾 302 树叶 341
塑料袋 270 纸箱 260
纸团 264 碎玻璃 283
包装袋 291 瓶子 357
橘子皮 287 298
金属块 365 电池 300
灯泡 321 香蕉皮 317
Tab.3 Proposed sample distribution of dataset
网络 因素 A/% P/M
宽度
因子
CBAM 多尺度
特征融合
迁移
学习
Mobilenet v2 ? ? ? ? 92.6 2.24
0.5 ? ? ? 90.4 0.59
? ? ? 94.3 2.26
? ? ? 93.9 3.09
? ? ? 93.4 2.24
本研究算法 0.5 94.6 0.83
Tab.4 Ablation study results
Fig.8 Training curves of each model in proposed data set
模型 AIM/% ACF/%
本研究算法 68.4 75.4
MobileNet v2 66.2 74.0
ResNet101 72.3 76.5
ResNet50 69.4 74.6
GoogleNet 62.7 63.9
VGG16 64.5 70.5
Tab.5 Test accuracy of each model in common data set
模型 A/% P/M Ts/ms
本研究算法 94.6 0.83 37.97
MobileNet v2 92.6 2.24 44.32
ResNet101 93.4 42.53 204.94
ResNet50 92.3 23.53 124.22
GoogleNet 91.4 10.35 68.64
VGG16 91.2 70.32 197.28
Tab.6 Comparison of training results of each model in proposed data set
Fig.9 Model recognition velocity curve
Fig.10 Garbage image recognition effect
[1]   吕君, 翟晓颖 基于横向视角的垃圾回收处理体系的国际比较研究及启示[J]. 生态经济, 2015, 31 (12): 102- 106
LV Jun, ZHAI Xiao-ying An international comparison study and enlightenment of waste recycling system based on lateral perspective[J]. Ecological Economy, 2015, 31 (12): 102- 106
doi: 10.3969/j.issn.1671-4407.2015.12.022
[2]   WANG Z L, LI H, YANG X T Vision-based robotic system for on-site construction and demolition waste sorting and recycling[J]. Journal of Building Engineering, 2020, 32: 1- 13
[3]   武凌, 王浩, 张晓春, 等 基于深度迁移学习的垃圾分类系统设计与实现[J]. 沈阳大学学报: 自然科学版, 2020, 32 (6): 496- 502
WU Ling, WANG Hao, ZHANG Xiao-chun, et al Design and implementation of garbage classification system based on deep transfer learning[J]. Journal of Shenyang University: Natural Science, 2020, 32 (6): 496- 502
[4]   康庄, 杨杰, 郭濠奇 基于机器视觉的垃圾自动分类系统设计[J]. 浙江大学学报: 工学版, 2020, 54 (7): 1272- 1280
KANG Zhuang, YANG Jie, GUO Hao-qi Automatic garbage classification system based on machine vision[J]. Journal of Zhejiang University: Engineering Science, 2020, 54 (7): 1272- 1280
[5]   王爽 微信小程序在垃圾分类中的应用研究[J]. 信息与电脑, 2019, 31 (22): 66- 68
WANG Shuang Research on WeChat small program in garbage classification application[J]. China Computer and Communication, 2019, 31 (22): 66- 68
[6]   GORLI R Interlinking of IoT, big data, smart mobile app with smart garbage monitoring[J]. International Journal of Computerences and Engineering, 2017, 5 (1): 70- 74
[7]   KRIZHEVSKY A, SUTSKEVER I, HINTON G. ImageNet classification with deep convolutional neural networks[C]// International Conference on Neural Information Processing Systems. Lake Tahoe: Curran Associates Inc, 2012: 1097-1105.
[8]   SZEGEDY C, LIU W, JIA Y, et al. Going deeper with convolutions[C]// Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition. Boston: IEEE, 2015: 1-9.
[9]   SIMONYAN K, ZISSERMAN A. Very deep convolutional networks for large-scale image recognition [EB/OL]. [2021-02-22]. http://arxiv.org/abs/1409.1556.
[10]   HE K, ZHANG X, REN S, et al. Deep residual learning for image recognition[C]// 2016 the IEEE Conference on Computer Vision and Pattern Recognition. Las Vegas: IEEE, 2016: 770-778.
[11]   IANDOLA F N, HAN S, MOSKEWICZ M W, et al. SqueezeNet: AlexNet-level accuracy with 50x fewer parameters and <0.5 MB model size [EB/OL]. [2021-02-22]. https://arxiv.org/pdf/1602.07360.pdf.
[12]   ZHANG X, ZHOU X, LIN M, et al. Shufflenet: an extremely efficient convolutional neural network for mobile devices[C]// Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition. Salt Lake City: IEEE, 2018: 6848-6856.
[13]   CHOLLET F. Xception: deep learning with depthwise separable convolutions[C]// 2017 IEEE Conference on Computer Vision and Pattern Recognition. Hawaii: IEEE, 2017: 1251-1258.
[14]   HOWARD A, ZHU M, CHEN B, et al. MobileNets: efficient convolutional neural networks for mobile vision applications[C]// Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition. Hawaii: IEEE, 2017: 1704-1712.
[15]   YANG Z, LI D WasNet: a neural network-based garbage collection management system[J]. IEEE Access, 2020, 8: 103984- 103993
doi: 10.1109/ACCESS.2020.2999678
[16]   LIU X, WU Z Z, ZOU L, et al. Lightweight neural network based garbage image classification using a deep mutual learning[C]// 11th International Symposium on Parallel Architectures, Algorithms and Programming. Shenzhen: Springer, 2021: 212-223.
[17]   袁建野, 南新元, 蔡鑫, 等 基于轻量级残差网路的垃圾图片分类方法[J]. 环境工程, 2021, 39 (2): 110- 115
YUAN Jian-ye, NAN Xin-yuan, CAI Xin, et al Garbage image classification by lightweight residual network[J]. Environmental Engineering, 2021, 39 (2): 110- 115
[18]   高明, 陈玉涵, 张泽慧, 等 基于新型空间注意力机制和迁移学习的垃圾图像分类算法[J]. 系统工程理论与实践, 2021, 41 (2): 498- 512
GAO Min, CHEN Yu-han, ZHANG Ze-hui, et al Classification algorithm of garbage images based on novel spatial attention mechanism and transfer learning[J]. Systems Engineering Theory and Practice, 2021, 41 (2): 498- 512
doi: 10.12011/SETP2020-1645
[19]   SHI C, XIA R, WANG L A novel multi-branch channel expansion network for garbage image classification[J]. IEEE Access, 2020, 8: 154436- 154452
doi: 10.1109/ACCESS.2020.3016116
[20]   SANDLER M, HOWARD A, ZHU M, et al. MobileNetV2: inverted residuals and linear bottlenecks[C]// Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition. Salt Lake City: IEEE, 2018: 4510-4520.
[21]   WOO S, PARK J, LEE J Y, et al. CBAM: convolutional block attention module[C]// European Conference on Computer Vision. Munich: Springer, 2018: 3-19.
[22]   NASRIN S, BADAWI D, CETIN A E, et al MF-Net: compute-in-memory SRAM for multibit precision inference using memory-immersed data conversion and multiplication-free operators[J]. IEEE Transactions on Circuits and Systems, 2021, 68 (5): 1966- 1978
[1] Yi-cong GAO,Yan-kun WANG,Shao-mei FEI,Qiong LIN. Intelligent proofreading method of engineering drawing based on transfer learning[J]. Journal of ZheJiang University (Engineering Science), 2022, 56(5): 856-863, 889.
[2] Xiao-chen JU,Xin-xin ZHAO,Sheng-sheng QIAN. Self-attention mechanism based bridge bolt detection algorithm[J]. Journal of ZheJiang University (Engineering Science), 2022, 56(5): 901-908.
[3] You-wei WANG,Shuang TONG,Li-zhou FENG,Jian-ming ZHU,Yang LI,Fu CHEN. New inductive microblog rumor detection method based on graph convolutional network[J]. Journal of ZheJiang University (Engineering Science), 2022, 56(5): 956-966.
[4] Xue-qin ZHANG,Tian-ren LI. Breast cancer pathological image classification based on Cycle-GAN and improved DPN network[J]. Journal of ZheJiang University (Engineering Science), 2022, 56(4): 727-735.
[5] Meng XU,Dan WANG,Zhi-yuan LI,Yuan-fang CHEN. IncepA-EEGNet: P300 signal detection method based on fusion of Inception network and attention mechanism[J]. Journal of ZheJiang University (Engineering Science), 2022, 56(4): 745-753, 782.
[6] Chang-yuan LIU,Xian-ping HE,Xiao-jun BI. Efficient network vehicle recognition combined with attention mechanism[J]. Journal of ZheJiang University (Engineering Science), 2022, 56(4): 775-782.
[7] Qiao-hong CHEN,Hao-lei PEI,Qi SUN. Image caption based on relational reasoning and context gate mechanism[J]. Journal of ZheJiang University (Engineering Science), 2022, 56(3): 542-549.
[8] Yuan-jun NONG,Jun-jie WANG,Hong CHEN,Wen-han SUN,Hui GENG,Shu-yue LI. A image caption method of construction scene based on attention mechanism and encoding-decoding architecture[J]. Journal of ZheJiang University (Engineering Science), 2022, 56(2): 236-244.
[9] Ying-li LIU,Rui-gang WU,Chang-hui YAO,Tao SHEN. Construction method of extraction dataset of Al-Si alloy entity relationship[J]. Journal of ZheJiang University (Engineering Science), 2022, 56(2): 245-253.
[10] Xin WANG,Qiao-hong CHEN,Qi SUN,Yu-bo JIA. Visual question answering method based on relational reasoning and gating mechanism[J]. Journal of ZheJiang University (Engineering Science), 2022, 56(1): 36-46.
[11] Zi-ye YONG,Ji-chang GUO,Chong-yi LI. weakly supervised underwater image enhancement algorithm incorporating attention mechanism[J]. Journal of ZheJiang University (Engineering Science), 2021, 55(3): 555-562.
[12] Han-juan CHEN,Fei-peng DA,Shao-yan GAI. Deep 3D point cloud classification network based on competitive attention fusion[J]. Journal of ZheJiang University (Engineering Science), 2021, 55(12): 2342-2351.
[13] Yue-lin CHEN,Wen-jing TIAN,Xiao-dong CAI,Shu-ting ZHENG. Text matching model based on dense connection networkand multi-dimensional feature fusion[J]. Journal of ZheJiang University (Engineering Science), 2021, 55(12): 2352-2358.
[14] Wen-bin XIN,Hui-min HAO,Ming-long BU,Yuan LAN,Jia-hai HUANG,Xiao-yan XIONG. Static gesture real-time recognition method based on ShuffleNetv2-YOLOv3 model[J]. Journal of ZheJiang University (Engineering Science), 2021, 55(10): 1815-1824.
[15] Zhuang KANG,Jie YANG,Hao-qi GUO. Automatic garbage classification system based on machine vision[J]. Journal of ZheJiang University (Engineering Science), 2020, 54(7): 1272-1280.