Please wait a minute...
浙江大学学报(工学版)
浙江大学学报(工学版)  2021, Vol. 55 Issue (8): 1490-1499    DOI: 10.3785/j.issn.1008-973X.2021.08.010
计算机技术     
基于改进MobileNet v2的垃圾图像分类算法
陈智超1,2(),焦海宁1,2,*(),杨杰1,2,曾华福1,2
1. 江西理工大学 电气工程与自动化学院，江西 赣州 341000
2. 江西省磁悬浮技术重点实验室，江西 赣州 341000
Garbage image classification algorithm based on improved MobileNet v2
Zhi-chao CHEN1,2(),Hai-ning JIAO1,2,*(),Jie YANG1,2,Hua-fu ZENG1,2
1. School of Electrical Engineering and Automation, Jiangxi University of Science and Technology, Ganzhou 341000, China
2. Jiangxi Provincial Key Laboratory of Maglev Technology, Ganzhou 341000, China
 全文: PDF(1439 KB)   HTML
摘要：

针对现有的垃圾图像分类模型实时性能差和分类精度低的问题，提出基于改进MobileNet v2的垃圾图像分类方法，构建以MobileNet v2为核心的轻量级特征提取网络. 通过调整宽度因子降低模型的参数量；在模型中嵌入通道和空间注意力模块，增强网络对特征的细化能力；设计多尺度特征融合结构，增强网络对尺度的适应性；利用迁移学习的方式优化模型参数，进一步提高模型精度. 实验结果表明，算法在自建数据集上的平均准确率为94.6%，分别高于MobileNet v2、VGG16、GoogleNet、ResNet50、ResNet101模型2.0%、3.4%、3.2%、2.3%、1.2%；所提算法在2种公共图像分类数据集CIFAR-100和tiny-ImageNet中均取得不错表现；模型参数量仅为0.83 M，体积约为基础模型的2/5，在边缘设备JETSON TX2上的单次推理耗时68 ms，实现了推理速度和预测准确率的提升.
关键词： 垃圾图像分类;  MobileNet v2;  注意力机制;  多尺度特征融合;  迁移学习    
Abstract:

A garbage image classification method based on improved MobileNet v2 was proposed aiming at the problems of poor real-time performance and low classification accuracy of existing garbage image classification models. A lightweight feature extraction network based on MobileNet v2 was constructed. The parameter numbers of the model were reduced by adjusting its width factor, channel and spatial attention modules were embedded in the model to enhance the network's ability to refine features, a multi-scale feature fusion structure was designed to enhance the adaptability of the network to scale, and transfer learning was used to optimize the model parameters to further improve the model accuracy. Experimental results show that the average accuracy of the algorithm on the self built dataset was 94.6%, which was 2.0%, 3.4%, 3.2%, 2.3% and 1.2% higher than that of MobileNet v2, VGG16, GoogleNet, ResNet50 and ResNet101 models, respectively. The proposed algorithm achieved good performance in two public image classification datasets, CIFAR-100 and tiny-ImageNet. The parameter numbers of the model was only 0.83 M, which was about 2/5 of the basic model. The single inference on edge device JETSON TX2 took 68 ms, which proved the improvement of inference speed and prediction accuracy.
Key words: garbage image classification    MobileNet v2    attention mechanism    multi scale feature fusion    transfer learning
收稿日期: 2021-03-18 出版日期: 2021-09-01
CLC:  TP 399  
基金资助: 国家自然科学基金资助项目（61763016）；江西省03专项及5G资助项目（20204ABC03A15）
通讯作者: 焦海宁     E-mail: chenzhichao_ai@163.com;jiaohaining@yeah.net
作者简介: 陈智超（1997—），男，硕士生，从事计算机视觉研究. orcid.org/0000-0002-7150-4914. E-mail： chenzhichao_ai@163.com
服务　　
把本文推荐给朋友
加入引用管理器
E-mail Alert
作者相关文章　　
陈智超
焦海宁
杨杰
曾华福

引用本文:

陈智超,焦海宁,杨杰,曾华福. 基于改进MobileNet v2的垃圾图像分类算法[J]. 浙江大学学报(工学版), 2021, 55(8): 1490-1499.

Zhi-chao CHEN,Hai-ning JIAO,Jie YANG,Hua-fu ZENG. Garbage image classification algorithm based on improved MobileNet v2. Journal of ZheJiang University (Engineering Science), 2021, 55(8): 1490-1499.

链接本文:

https://www.zjujournals.com/eng/CN/10.3785/j.issn.1008-973X.2021.08.010    或    https://www.zjujournals.com/eng/CN/Y2021/V55/I8/1490

图 1  改进MobileNet v2网络整体框图
输入尺寸 操作 t c n s
2242×3 卷积 ? 16 ? 2
1122×16 Bottleneck_A 1 8 1 1
1122×8 Bottleneck_I 6 8 1 2
562×8 Bottleneck_A 6 12 1 1
562×12 Bottleneck_I 6 12 1 2
282×12 Bottleneck_A 6 16 2 1
282×16 Bottleneck_I 6 16 1 2
142×16 Bottleneck_A 6 32 3 1
142×32 Bottleneck_A 6 48 3 1
142×48 Bottleneck_I 6 48 1 2
72×48 Bottleneck_A 6 80 2 1
72×80 Bottleneck_A 6 160 1 1
72×160 PW卷积 ? 640 ? 1
72×640 平均池化 ? 640 ? 1
72×640 PW卷积 ? 14 ? 1
表 1  改进MobileNet v2网络结构表
图 2  深度可分离卷积计算原理
图 3  2种步长下的瓶颈结构
模型 Atop1/% GFL/M P/M
MobileNet（1.00）[14] 70.6 569 4.2
MobileNet（0.75）[14] 68.4 325 2.6
MobileNet（0.50）[14] 63.7 149 1.3
表 2  不同宽度因子下的MobileNet性能表现
图 4  通道和空间注意力机制
图 5  带注意力机制的反向残差模块（Bottleneck_A）
图 6  Inception v1模块
图 7  多尺度特征融合模块（Bottleneck_I）
垃圾名称 数目/张 垃圾名称 数目/张
毛巾 302 树叶 341
塑料袋 270 纸箱 260
纸团 264 碎玻璃 283
包装袋 291 瓶子 357
橘子皮 287 书 298
金属块 365 电池 300
灯泡 321 香蕉皮 317
表 3  本研究所提出的数据集样本分布
网络 因素 A/% P/M
宽度
因子 		CBAM 多尺度
特征融合 		迁移
学习
Mobilenet v2 ? ? ? ? 92.6 2.24
0.5 ? ? ? 90.4 0.59
? √ ? ? 94.3 2.26
? ? √ ? 93.9 3.09
? ? ? √ 93.4 2.24
本研究算法 0.5 √ √ √ 94.6 0.83
表 4  消融实验结果
图 8  各模型在本研究数据集中的训练曲线
模型 AIM/% ACF/%
本研究算法 68.4 75.4
MobileNet v2 66.2 74.0
ResNet101 72.3 76.5
ResNet50 69.4 74.6
GoogleNet 62.7 63.9
VGG16 64.5 70.5
表 5  各模型在公共数据集中的测试准确率
模型 A/% P/M Ts/ms
本研究算法 94.6 0.83 37.97
MobileNet v2 92.6 2.24 44.32
ResNet101 93.4 42.53 204.94
ResNet50 92.3 23.53 124.22
GoogleNet 91.4 10.35 68.64
VGG16 91.2 70.32 197.28
表 6  各模型在本研究自建数据集中的训练结果对比
图 9  模型识别速度曲线
图 10  垃圾图像识别效果
1 吕君, 翟晓颖 基于横向视角的垃圾回收处理体系的国际比较研究及启示[J]. 生态经济, 2015, 31 (12): 102- 106
LV Jun, ZHAI Xiao-ying An international comparison study and enlightenment of waste recycling system based on lateral perspective[J]. Ecological Economy, 2015, 31 (12): 102- 106
doi: 10.3969/j.issn.1671-4407.2015.12.022
2 WANG Z L, LI H, YANG X T Vision-based robotic system for on-site construction and demolition waste sorting and recycling[J]. Journal of Building Engineering, 2020, 32: 1- 13
3 武凌, 王浩, 张晓春, 等 基于深度迁移学习的垃圾分类系统设计与实现[J]. 沈阳大学学报: 自然科学版, 2020, 32 (6): 496- 502
WU Ling, WANG Hao, ZHANG Xiao-chun, et al Design and implementation of garbage classification system based on deep transfer learning[J]. Journal of Shenyang University: Natural Science, 2020, 32 (6): 496- 502
4 康庄, 杨杰, 郭濠奇 基于机器视觉的垃圾自动分类系统设计[J]. 浙江大学学报: 工学版, 2020, 54 (7): 1272- 1280
KANG Zhuang, YANG Jie, GUO Hao-qi Automatic garbage classification system based on machine vision[J]. Journal of Zhejiang University: Engineering Science, 2020, 54 (7): 1272- 1280
5 王爽 微信小程序在垃圾分类中的应用研究[J]. 信息与电脑, 2019, 31 (22): 66- 68
WANG Shuang Research on WeChat small program in garbage classification application[J]. China Computer and Communication, 2019, 31 (22): 66- 68
6 GORLI R Interlinking of IoT, big data, smart mobile app with smart garbage monitoring[J]. International Journal of Computerences and Engineering, 2017, 5 (1): 70- 74
7 KRIZHEVSKY A, SUTSKEVER I, HINTON G. ImageNet classification with deep convolutional neural networks[C]// International Conference on Neural Information Processing Systems. Lake Tahoe: Curran Associates Inc, 2012: 1097-1105.
8 SZEGEDY C, LIU W, JIA Y, et al. Going deeper with convolutions[C]// Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition. Boston: IEEE, 2015: 1-9.
9 SIMONYAN K, ZISSERMAN A. Very deep convolutional networks for large-scale image recognition [EB/OL]. [2021-02-22]. http://arxiv.org/abs/1409.1556.
10 HE K, ZHANG X, REN S, et al. Deep residual learning for image recognition[C]// 2016 the IEEE Conference on Computer Vision and Pattern Recognition. Las Vegas: IEEE, 2016: 770-778.
11 IANDOLA F N, HAN S, MOSKEWICZ M W, et al. SqueezeNet: AlexNet-level accuracy with 50x fewer parameters and <0.5 MB model size [EB/OL]. [2021-02-22]. https://arxiv.org/pdf/1602.07360.pdf.
12 ZHANG X, ZHOU X, LIN M, et al. Shufflenet: an extremely efficient convolutional neural network for mobile devices[C]// Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition. Salt Lake City: IEEE, 2018: 6848-6856.
13 CHOLLET F. Xception: deep learning with depthwise separable convolutions[C]// 2017 IEEE Conference on Computer Vision and Pattern Recognition. Hawaii: IEEE, 2017: 1251-1258.
14 HOWARD A, ZHU M, CHEN B, et al. MobileNets: efficient convolutional neural networks for mobile vision applications[C]// Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition. Hawaii: IEEE, 2017: 1704-1712.
15 YANG Z, LI D WasNet: a neural network-based garbage collection management system[J]. IEEE Access, 2020, 8: 103984- 103993
doi: 10.1109/ACCESS.2020.2999678
16 LIU X, WU Z Z, ZOU L, et al. Lightweight neural network based garbage image classification using a deep mutual learning[C]// 11th International Symposium on Parallel Architectures, Algorithms and Programming. Shenzhen: Springer, 2021: 212-223.
17 袁建野, 南新元, 蔡鑫, 等 基于轻量级残差网路的垃圾图片分类方法[J]. 环境工程, 2021, 39 (2): 110- 115
YUAN Jian-ye, NAN Xin-yuan, CAI Xin, et al Garbage image classification by lightweight residual network[J]. Environmental Engineering, 2021, 39 (2): 110- 115
18 高明, 陈玉涵, 张泽慧, 等 基于新型空间注意力机制和迁移学习的垃圾图像分类算法[J]. 系统工程理论与实践, 2021, 41 (2): 498- 512
GAO Min, CHEN Yu-han, ZHANG Ze-hui, et al Classification algorithm of garbage images based on novel spatial attention mechanism and transfer learning[J]. Systems Engineering Theory and Practice, 2021, 41 (2): 498- 512
doi: 10.12011/SETP2020-1645
19 SHI C, XIA R, WANG L A novel multi-branch channel expansion network for garbage image classification[J]. IEEE Access, 2020, 8: 154436- 154452
doi: 10.1109/ACCESS.2020.3016116
20 SANDLER M, HOWARD A, ZHU M, et al. MobileNetV2: inverted residuals and linear bottlenecks[C]// Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition. Salt Lake City: IEEE, 2018: 4510-4520.
21 WOO S, PARK J, LEE J Y, et al. CBAM: convolutional block attention module[C]// European Conference on Computer Vision. Munich: Springer, 2018: 3-19.
22 NASRIN S, BADAWI D, CETIN A E, et al MF-Net: compute-in-memory SRAM for multibit precision inference using memory-immersed data conversion and multiplication-free operators[J]. IEEE Transactions on Circuits and Systems, 2021, 68 (5): 1966- 1978
[1] 高一聪,王彦坤,费少梅,林琼. 基于迁移学习的机械制图智能评阅方法[J]. 浙江大学学报(工学版), 2022, 56(5): 856-863, 889.
[2] 鞠晓臣,赵欣欣,钱胜胜. 基于自注意力机制的桥梁螺栓检测算法[J]. 浙江大学学报(工学版), 2022, 56(5): 901-908.
[3] 王友卫,童爽,凤丽洲,朱建明,李洋,陈福. 基于图卷积网络的归纳式微博谣言检测新方法[J]. 浙江大学学报(工学版), 2022, 56(5): 956-966.
[4] 张雪芹,李天任. 基于Cycle-GAN和改进DPN网络的乳腺癌病理图像分类[J]. 浙江大学学报(工学版), 2022, 56(4): 727-735.
[5] 许萌,王丹,李致远,陈远方. IncepA-EEGNet: 融合Inception网络和注意力机制的P300信号检测方法[J]. 浙江大学学报(工学版), 2022, 56(4): 745-753, 782.
[6] 柳长源,何先平,毕晓君. 融合注意力机制的高效率网络车型识别[J]. 浙江大学学报(工学版), 2022, 56(4): 775-782.
[7] 陈巧红,裴皓磊,孙麒. 基于视觉关系推理与上下文门控机制的图像描述[J]. 浙江大学学报(工学版), 2022, 56(3): 542-549.
[8] 农元君,王俊杰,陈红,孙文涵,耿慧,李书悦. 基于注意力机制和编码-解码架构的施工场景图像描述方法[J]. 浙江大学学报(工学版), 2022, 56(2): 236-244.
[9] 刘英莉,吴瑞刚,么长慧,沈韬. 铝硅合金实体关系抽取数据集的构建方法[J]. 浙江大学学报(工学版), 2022, 56(2): 245-253.
[10] 董红召,方浩杰,张楠. 旋转框定位的多尺度再生物品目标检测算法[J]. 浙江大学学报(工学版), 2022, 56(1): 16-25.
[11] 王鑫,陈巧红,孙麒,贾宇波. 基于关系推理与门控机制的视觉问答方法[J]. 浙江大学学报(工学版), 2022, 56(1): 36-46.
[12] 雍子叶,郭继昌,李重仪. 融入注意力机制的弱监督水下图像增强算法[J]. 浙江大学学报(工学版), 2021, 55(3): 555-562.
[13] 陈涵娟,达飞鹏,盖绍彦. 基于竞争注意力融合的深度三维点云分类网络[J]. 浙江大学学报(工学版), 2021, 55(12): 2342-2351.
[14] 陈岳林,田文靖,蔡晓东,郑淑婷. 基于密集连接网络和多维特征融合的文本匹配模型[J]. 浙江大学学报(工学版), 2021, 55(12): 2352-2358.
[15] 辛文斌,郝惠敏,卜明龙,兰媛,黄家海,熊晓燕. 基于ShuffleNetv2-YOLOv3模型的静态手势实时识别方法[J]. 浙江大学学报(工学版), 2021, 55(10): 1815-1824.