Please wait a minute...
浙江大学学报(工学版)
浙江大学学报(工学版)  2024, Vol. 58 Issue (9): 1757-1767    DOI: 10.3785/j.issn.1008-973X.2024.09.001
计算机与控制工程     
基于MA-ConvNext网络和分步关系知识蒸馏的苹果叶片病害识别
刘欢1(),李云红1,*(),张蕾涛1,郭越2,苏雪平1,朱耀麟1,侯乐乐1
1. 西安工程大学 电子信息学院，陕西 西安 710048
2. 山西大学 生命科学学院，山西 太原 030031
Identification of apple leaf diseases based on MA-ConvNext network and stepwise relational knowledge distillation
Huan LIU1(),Yunhong LI1,*(),Leitao ZHANG1,Yue GUO2,Xueping SU1,Yaolin ZHU1,Lele HOU1
1. School of Electronics and Information, Xi’an Polytechnic University, Xi’an 710048, China
2. School of Life Science, Shanxi University, Taiyuan 030031, China
 全文: PDF(5637 KB)   HTML
摘要：

针对复杂环境下苹果叶片病害图像背景杂乱、病斑大小不一，以及现有模型参数多、计算量大的问题，提出基于注意力和多尺度特征融合的苹果叶片病害识别网络(MA-ConvNext). 通过引入多尺度空间通道重组块(MSCB)和融合三分支注意力机制的特征提取模块(TAFB)，有效提取苹果叶片病害图像不同尺度的特征，增强模型对叶片病斑的关注. 采用分步关系知识蒸馏方法，将“教师”网络(MA-ConvNext)和“中间”网络(DenseNet121)融合，指导“学生”网络(EfficientNet-B0)训练，实现模型轻量化. 实验结果表明，MA-ConvNext网络识别准确率为99.38%，较ResNet50、MobileNet-V3和EfficientNet-V2网络分别提高了3.98个百分点、7.55个百分点和4.27个百分点. 经过分步关系知识蒸馏后，识别准确率较蒸馏前提高了1.76个百分点，并且具有更小的网络规模和参数量，分别为1.56×107、5.29×106. 所提方法能为后续精准农业的病虫害检测提供新思路和技术支持.
关键词： 苹果叶片病害识别;  注意力;  多尺度特征融合;  分步关系;  知识蒸馏    
Abstract:

The backgrounds are cluttered, the spot sizes of apple leaf disease are varying in complex environments, and the existing models have the problems of multiple parameters and a large amount of calculation. Thus, an apple leaf disease recognition network, ConvNext network based on attention and multiscale feature fusion (MA-ConvNext), was proposed. A multiscale spatial reconstruction and channel reconstruction block (MSCB) and a feature extraction block with triplet attention fusion (TAFB) were utilized to effectively extract the features at different scales and enhance the focus on leaf disease spots. Additionally, a stepwise relational knowledge distillation method was employed to fuse the "teacher" network (MA-ConvNext) with an "intermediate" network (DenseNet121) to guide the training of the "student" network (EfficientNet-B0) and achieve the model lightweighting. Experimental results showed that MA-ConvNext achieved a recognition accuracy of 99.38%, improving by 3.98 percentage points, 7.55 percentage points and 4.27 percentage points compared to ResNet50, MobileNet-V3, and EfficientNet-V2 networks, respectively. After the stepwise relational knowledge distillation, the recognition accuracy further improved by 1.76 percentage points, with a smaller network size and parameters of 1.56×107 and 5.29×106. respectively. The proposed method offers new insights and technical support for the precise detection of pests and diseases in agriculture.
Key words: apple leaf disease identification    attention    multiscale feature fusion    stepwise relationship    knowledge distillation
收稿日期: 2024-05-10 出版日期: 2024-08-30
CLC:  TP 393  
基金资助: 国家自然科学基金资助项目（62203344）；陕西省自然科学基础研究重点资助项目（2022JZ-35）；陕西高校青年创新团队资助项目.
通讯作者: 李云红     E-mail: huanlabc@163.com;hitliyunhong@163.com
作者简介: 刘欢（2000—），女，硕士生，从事图像处理研究. orcid.org/0009-0004-2491-8358. E-mail：huanlabc@163.com
服务　　
把本文推荐给朋友
加入引用管理器
E-mail Alert
作者相关文章　　
刘欢
李云红
张蕾涛
郭越
苏雪平
朱耀麟
侯乐乐

引用本文:

刘欢,李云红,张蕾涛,郭越,苏雪平,朱耀麟,侯乐乐. 基于MA-ConvNext网络和分步关系知识蒸馏的苹果叶片病害识别[J]. 浙江大学学报(工学版), 2024, 58(9): 1757-1767.

Huan LIU,Yunhong LI,Leitao ZHANG,Yue GUO,Xueping SU,Yaolin ZHU,Lele HOU. Identification of apple leaf diseases based on MA-ConvNext network and stepwise relational knowledge distillation. Journal of ZheJiang University (Engineering Science), 2024, 58(9): 1757-1767.

链接本文:

https://www.zjujournals.com/eng/CN/10.3785/j.issn.1008-973X.2024.09.001    或    https://www.zjujournals.com/eng/CN/Y2024/V58/I9/1757

图 1  苹果叶片病害图像
病害类型N
AppleApple-IA
黑腐病8887284
锈病5474276
疮痂病9446384
健康叶片12856168
总数366424112
表 1  苹果叶片病害数据集参数
图 2  MA-ConvNext网络结构图
图 3  ScConv网络结构图
图 4  TAFB模块结构图
图 5  分步关系知识蒸馏网络结构
网络层SISCSSO
Convolution＆
pooling		224×2247×72112×112
112×1123×3 max pool256×56
Dense
Block 1		56×56$ \left[\begin{array}{l}1 \times 1 {\mathrm{c o n v}} \\3 \times 3 {\mathrm{c o n v}}\end{array}\right] \times 6 $156×56
Transition
Layer 1		56×561×1 conv156×56
56×562×2 average pool228×28
Dense
Block 2		28×28$ \left[\begin{array}{l}1 \times 1 {\mathrm{c o n v}} \\3 \times 3 {\mathrm{c o n v}}\end{array}\right] \times 12 $128×28
Transition
Layer 2		28×281×1 conv128×28
28×282×2 average pool214×14
Dense
Block 3		14×14$ \left[\begin{array}{l}1 \times 1 {\mathrm{c o n v}} \\3 \times 3 {\mathrm{c o n v}}\end{array}\right] \times 24 $114×14
Transition
Layer 3		14×141×1 conv114×14
14×142×2 average pool27×7
Dense
Block 4		7×7$ \left[\begin{array}{l}1 \times 1 {\mathrm{c o n v}} \\3 \times 3 {\mathrm{c o n v}}\end{array}\right] \times 16 $17×7
Classification Layer7×7Global average pool—1×1
1×11000
Fully-connected		—1000
表 2  DenseNet121网络结构
图 6  EfficientNet-B0模型复合缩放方法
网络模型FPS /(帧·s?1)W/106改进算法ACC/%F1/%R/%
MSCBTAFB
ConvNext73.510.5——98.7798.0198.20
A-ConvNext71.510.5—√99.1898.8398.83
M-ConvNext74.711.2√—99.2198.7698.93
MA-ConvNext75.212.7√√99.3899.0999.21
表 3  消融实验结果
图 7  各模块对黑腐病苹果叶片的识别热力图
图 8  各模块对锈病苹果叶片的识别热力图
图 9  各模块对疮痂病苹果叶片的识别热力图
网络模型ACC/%
原始数据集扩充后数据集
ConvNext74.4598.77
MA-ConvNext78.3499.38
表 4  数据扩充前、后精度的比较
网络模型S/106F/109P/106ACC/%F1/%R/%
ResNet50[20]90.004.1325.5695.4093.1793.70
Inception-V3[21]83.402.8627.1697.0995.4995.81
DenseNet121[22]27.602.907.9895.9494.0494.43
MobileNet-V3[23]2.280.235.4891.8388.4189.33
EfficientNet-V2[24]77.802.8921.4695.1192.7893.18
ConvNext[25]106.004.4928.5998.7798.0198.20
MA-ConvNext106.004.6027.8699.3899.0999.21
表 5  不同模型在测试集上的比较
图 10  改进前、后网络的混淆矩阵对比
网络模型ACC/%LossS/106F/109P/106
MA-ConvNext99.380.037106.04.6027.86
DenseNet12195.940.01927.62.907.98
EfficientNet-B093.720.06815.60.415.29
表 6  分步关系知识蒸馏中各网络性能
网络模型ACC/%S/106F/106P/106
EfficientNet-B093.7215.6410.25.29
关系知识蒸馏
EfficientNet-B0		94.7515.615.65.29
分步关系知识蒸馏
EfficientNet-B0		95.4815.615.65.29
表 7  “学生”网络不同蒸馏方式的对比
网络模型S/106F/109P/106ACC/%
MA-ConvNext106.04.627.8699.38
DenseNet12127.62.97.9895.94
关系知识蒸馏
DenseNet121		27.62.97.9894.57
表 8  “中间”网络关系知识蒸馏结果
图 11  “中间”网络训练过程中准确率曲线
网络模型ACC/%LossS/106F/109
MA-ConvNext99.380.037106.04.60
关系知识蒸馏
DenseNet121		94.571.41227.62.90
分步关系知识蒸馏
EfficientNet-B0		95.480.82315.60.41
表 9  分步关系知识蒸馏结果
1 翟肇裕, 曹益飞, 徐焕良, 等 农作物病虫害识别关键技术研究综述[J]. 农业机械学报, 2021, 52 (7): 1- 18
ZHAI Zhaoyu, CAO Yifei, XU Huanliang, et al Review of key techniques for crop disease and pest detection[J]. Transactions of the Chinese Society for Agricultural Machinery, 2021, 52 (7): 1- 18
2 孟子恒, 朱海燕, 刘学忠 农业产业集聚对农业经济增长的影响研究: 基于苹果产业的实证分析[J]. 中国农业资源与区划, 2022, 43 (2): 231- 239
MENG Ziheng, ZHU Haiyan, LIU Xuezhong Research on the impact of agricultural industry agglomeration on agricultural economic growth: an empirical analysis based on apple industry[J]. China Agricultural Resources and Zoning, 2022, 43 (2): 231- 239
3 岳喜申 深度学习在农作物病害识别中的研究进展[J]. 安徽农学通报, 2024, 30 (6): 100- 103
YUE Xishen Research progress of deep learning in crop disease identification[J]. Anhui Agricultural Bulletin, 2024, 30 (6): 100- 103
4 盛帅, 段先华, 胡维康, 等. Dynamic-YOLOX: 复杂背景下的苹果叶片病害检测模型[J]. 计算机科学与探索, 2024, 18(8): 2118−2129.
SHENG Shuai, DUAN Xianhua, HU Weikang, et al. Dynamic-YOLOX: apple leaf disease detection model under complex background [J]. Computer Science and Exploration , 2024, 18(8): 2118−2129.
5 李佼隆. 基于LeafGAN的数据增强及木薯叶片病害分类识别系统研究[D]. 雅安: 四川农业大学, 2023.
LI Jiaolong. Research on data enhancement and cassava leaf disease classification and identification system based on LeafGAN [D]. Yaan: Sichuan Agricultural University, 2023.
6 李云红, 张蕾涛, 李丽敏, 等 基于CycleGAN-IA方法和M-ConvNext网络的苹果叶片病害图像识别[J]. 农业机械学报, 2024, 55 (4): 204- 212
LI Yunhong, ZHANG Leitao, LI Limin, et al Apple leaf disease image recognition based on CycleGAN-IA method and M-ConvNet network[J]. Journal of Agricultural Machinery, 2024, 55 (4): 204- 212
7 ZHANG Y, HUANG S, ZHOU G, et al Identification of tomato leaf diseases based on multi-channel automatic orientation recurrent attention network[J]. Computers and Electronics in Agriculture, 2023, 205: 107605
doi: 10.1016/j.compag.2022.107605
8 HAQUE M A, MARWAHA S, ARORA A, et al A lightweight convolutional neural network for recognition of severity stages of maydis leaf blight disease of maize[J]. Frontiers in Plant Science, 2022, 13: 1077568
doi: 10.3389/fpls.2022.1077568
9 MA Z, WANG Y, ZHANG T, et al Maize leaf disease identification using deep transfer convolutional neural networks[J]. International Journal of Agricultural and Biological Engineering, 2022, 15 (5): 187- 195
doi: 10.25165/j.ijabe.20221505.6658
10 YANG L, YU X, ZHANG S, et al GoogLeNet based on residual network and attention mechanism identification of rice leaf diseases[J]. Computers and Electronics in Agriculture, 2023, 204: 107543
doi: 10.1016/j.compag.2022.107543
11 ELFATIMI E, ERVIGIT R, ELFATIMI L Beans leaf diseases classification using mobilenet models[J]. IEEE Access, 2022, 10: 9471- 9482
doi: 10.1109/ACCESS.2022.3142817
12 LEE S H, CHAN C S, WILKIN P, et al. Deep-plant: plant identification with convolutional neural networks [C]// 2015 IEEE International Conference on Image Processing . Quebec City: IEEE, 2015: 452−456.
13 SLADOJEVIC S, ARSENOVIC M, ANDERLA A, et al Deep neural networks based recognition of plant diseases by leaf image classification[J]. Computational Intelligence and Neuroscience, 2016, (1): 1- 11
14 BRAHIMI M, BOUKHALFA K, MOUSSAOUI A Deep learning for tomato diseases: classification and symptoms visualization[J]. Applied Artificial Intelligence, 2017, 31 (4): 299- 315
doi: 10.1080/08839514.2017.1315516
15 FERENTINOS K P Deep learning models for plant disease detection and diagnosis[J]. Computers and Electronics in Agriculture, 2018, 145: 311- 318
doi: 10.1016/j.compag.2018.01.009
16 HASSAN S M, MAJI A K, JASINSKI M, et al Identification of plant-leaf diseases using CNN and transfer-learning approach[J]. Electronics, 2021, 10 (12): 1388
doi: 10.3390/electronics10121388
17 彭玉寒, 李书琴 基于重参数化MobileNetV2的农作物叶片病害识别模型[J]. 农业工程学报, 2023, 39 (17): 132- 140
PENG Yuhan, LI Shuqin Identification model of crop leaf diseases based on re-parameterized MobileNetV2[J]. Journal of Agricultural Engineering, 2023, 39 (17): 132- 140
18 李大华, 仲婷, 王笋, 等 基于改进ShuffleNet v2的轻量化番茄叶片病害识别[J]. 江苏农业科学, 2024, 52 (3): 220- 228
LI Dahua, ZHONG Ting, WANG Sun, et al Identification of light tomato leaf diseases based on improved ShuffleNet v2[J]. Jiangsu Agricultural Sciences, 2024, 52 (3): 220- 228
19 CHEN J, ZHANG D, ZEB A, et al Identification of rice plant diseases using lightweight attention networks[J]. Expert Systems with Applications, 2021, 169: 114514
doi: 10.1016/j.eswa.2020.114514
20 HE K M, ZHANG X Y, REN S Q, et al. Deep residual learning for image recognition [C]// IEEE Conference on Computer Vision and Pattern Recognition . Columbus: IEEE, 2016: 770−778.
21 SZEGEDY C, VANHOUCKE V, IOFFE S, et al. Rethinking the inception architecture for computer vision [C ]// IEEE Conference on Computer Vision and Pattern recognition . Columbus: IEEE, 2016: 2818−2826.
22 HUANG G, LIU Z, VAN DER MAATEN L, et al. Densely connected convolutional networks [C]// IEEE Conference on Computer Vision and Pattern Recognition . Honolulu: IEEE, 2017: 4700−4708.
23 HOWARD A, SANDLER M, CHU G, et al. Searching for mobilenetv3 [C]// IEEE/CVF International Conference on Computer Vision . Seoul: IEEE, 2019: 1314−1324.
24 TAN M, LE Q. Efficientnet: rethinking model scaling for convolutional neural networks [C]// International Conference on Machine Learning . Long Beach: IEEE, 2019: 6105−6114.
[1] 李灿林,王新玥,马利庄,邵志文,张文娇. 融合注意力机制和结构线提取的图像卡通化[J]. 浙江大学学报(工学版), 2024, 58(8): 1728-1737.
[2] 李忠良,陈麒,石琳,杨朝,邹先明. 时间感知组合的动态知识图谱补全[J]. 浙江大学学报(工学版), 2024, 58(8): 1738-1747.
[3] 吴书晗,王丹,陈远方,贾子钰,张越棋,许萌. 融合注意力的滤波器组双视图图卷积运动想象脑电分类[J]. 浙江大学学报(工学版), 2024, 58(7): 1326-1335.
[4] 马现伟,范朝辉,聂为之,李东,朱逸群. 对失效传感器具备鲁棒性的故障诊断方法[J]. 浙江大学学报(工学版), 2024, 58(7): 1488-1497.
[5] 杨军,张琛. 基于边界点估计与稀疏卷积神经网络的三维点云语义分割[J]. 浙江大学学报(工学版), 2024, 58(6): 1121-1132.
[6] 李运堂,李恒杰,张坤,王斌锐,关山越,陈源. 基于新型编码解码网络的复杂输电线识别[J]. 浙江大学学报(工学版), 2024, 58(6): 1133-1141.
[7] 舒晴,刘喜平,谭钊,李希,万常选,刘德喜,廖国琼. 基于依存关系图注意力网络的SQL生成方法[J]. 浙江大学学报(工学版), 2024, 58(5): 908-917.
[8] 邢志伟,朱书杰,李彪. 基于改进图卷积神经网络的航空行李特征感知[J]. 浙江大学学报(工学版), 2024, 58(5): 941-950.
[9] 刘毅,陈一丹,高琳,洪姣. 基于多尺度特征融合的轻量化道路提取模型[J]. 浙江大学学报(工学版), 2024, 58(5): 951-959.
[10] 魏翠婷,赵唯坚,孙博超,刘芸怡. 基于改进Mask R-CNN与双目视觉的智能配筋检测[J]. 浙江大学学报(工学版), 2024, 58(5): 1009-1019.
[11] 曹寅,秦俊平,高彤,马千里,任家琪. 基于生成对抗网络的文本两阶段生成高质量图像方法[J]. 浙江大学学报(工学版), 2024, 58(4): 674-683.
[12] 宦海,盛宇,顾晨曦. 基于遥感图像道路提取的全局指导多特征融合网络[J]. 浙江大学学报(工学版), 2024, 58(4): 696-707.
[13] 唐善成,逯建辉,张莹,金子成,赵安新. 修复缺陷嫌疑区域的无监督磁瓦表面缺陷检测[J]. 浙江大学学报(工学版), 2024, 58(4): 718-728.
[14] 宋明俊,严文,邓益昭,张俊然,涂海燕. 轻量化机器人抓取位姿实时检测算法[J]. 浙江大学学报(工学版), 2024, 58(3): 599-610.
[15] 姚鑫骅,于涛,封森文,马梓健,栾丛丛,沈洪垚. 基于图神经网络的零件机加工特征识别方法[J]. 浙江大学学报(工学版), 2024, 58(2): 349-359.