Please wait a minute...
浙江大学学报(工学版)
Journal of ZheJiang University (Engineering Science)  2024, Vol. 58 Issue (9): 1757-1767    DOI: 10.3785/j.issn.1008-973X.2024.09.001
    
Identification of apple leaf diseases based on MA-ConvNext network and stepwise relational knowledge distillation
Huan LIU1(),Yunhong LI1,*(),Leitao ZHANG1,Yue GUO2,Xueping SU1,Yaolin ZHU1,Lele HOU1
1. School of Electronics and Information, Xi’an Polytechnic University, Xi’an 710048, China
2. School of Life Science, Shanxi University, Taiyuan 030031, China
Download: HTML     PDF(5637KB) HTML
Export: BibTeX | EndNote (RIS)      

Abstract  

The backgrounds are cluttered, the spot sizes of apple leaf disease are varying in complex environments, and the existing models have the problems of multiple parameters and a large amount of calculation. Thus, an apple leaf disease recognition network, ConvNext network based on attention and multiscale feature fusion (MA-ConvNext), was proposed. A multiscale spatial reconstruction and channel reconstruction block (MSCB) and a feature extraction block with triplet attention fusion (TAFB) were utilized to effectively extract the features at different scales and enhance the focus on leaf disease spots. Additionally, a stepwise relational knowledge distillation method was employed to fuse the "teacher" network (MA-ConvNext) with an "intermediate" network (DenseNet121) to guide the training of the "student" network (EfficientNet-B0) and achieve the model lightweighting. Experimental results showed that MA-ConvNext achieved a recognition accuracy of 99.38%, improving by 3.98 percentage points, 7.55 percentage points and 4.27 percentage points compared to ResNet50, MobileNet-V3, and EfficientNet-V2 networks, respectively. After the stepwise relational knowledge distillation, the recognition accuracy further improved by 1.76 percentage points, with a smaller network size and parameters of 1.56×107 and 5.29×106. respectively. The proposed method offers new insights and technical support for the precise detection of pests and diseases in agriculture.

Key words： apple leaf disease identification      attention      multiscale feature fusion      stepwise relationship      knowledge distillation     
Received: 10 May 2024      Published: 30 August 2024
CLC:  TP 393  
Fund:  国家自然科学基金资助项目（62203344）；陕西省自然科学基础研究重点资助项目（2022JZ-35）；陕西高校青年创新团队资助项目.
Corresponding Authors: Yunhong LI     E-mail: huanlabc@163.com;hitliyunhong@163.com
Service
E-mail this article
Add to my bookshelf
Add to citation manager
E-mail Alert
RSS
Articles by authors
Huan LIU
Yunhong LI
Leitao ZHANG
Yue GUO
Xueping SU
Yaolin ZHU
Lele HOU
Cite this article:

Huan LIU,Yunhong LI,Leitao ZHANG,Yue GUO,Xueping SU,Yaolin ZHU,Lele HOU. Identification of apple leaf diseases based on MA-ConvNext network and stepwise relational knowledge distillation. Journal of ZheJiang University (Engineering Science), 2024, 58(9): 1757-1767.

URL:

https://www.zjujournals.com/eng/10.3785/j.issn.1008-973X.2024.09.001     OR     https://www.zjujournals.com/eng/Y2024/V58/I9/1757


基于MA-ConvNext网络和分步关系知识蒸馏的苹果叶片病害识别

针对复杂环境下苹果叶片病害图像背景杂乱、病斑大小不一，以及现有模型参数多、计算量大的问题，提出基于注意力和多尺度特征融合的苹果叶片病害识别网络(MA-ConvNext). 通过引入多尺度空间通道重组块(MSCB)和融合三分支注意力机制的特征提取模块(TAFB)，有效提取苹果叶片病害图像不同尺度的特征，增强模型对叶片病斑的关注. 采用分步关系知识蒸馏方法，将“教师”网络(MA-ConvNext)和“中间”网络(DenseNet121)融合，指导“学生”网络(EfficientNet-B0)训练，实现模型轻量化. 实验结果表明，MA-ConvNext网络识别准确率为99.38%，较ResNet50、MobileNet-V3和EfficientNet-V2网络分别提高了3.98个百分点、7.55个百分点和4.27个百分点. 经过分步关系知识蒸馏后，识别准确率较蒸馏前提高了1.76个百分点，并且具有更小的网络规模和参数量，分别为1.56×107、5.29×106. 所提方法能为后续精准农业的病虫害检测提供新思路和技术支持.


关键词： 苹果叶片病害识别,  注意力,  多尺度特征融合,  分步关系,  知识蒸馏 
Fig.1 Images of apple leaf disease
病害类型N
AppleApple-IA
黑腐病8887284
锈病5474276
疮痂病9446384
健康叶片12856168
总数366424112
Tab.1 Paramaters of apple leaf disease dataset
Fig.2 MA-ConvNext network structure diagram
Fig.3 ScConv network structure diagram
Fig.4 TAFB module structure diagram
Fig.5 Structure of stepwise relational knowledge distillation network
网络层SISCSSO
Convolution＆
pooling		224×2247×72112×112
112×1123×3 max pool256×56
Dense
Block 1		56×56$ \left[\begin{array}{l}1 \times 1 {\mathrm{c o n v}} \\3 \times 3 {\mathrm{c o n v}}\end{array}\right] \times 6 $156×56
Transition
Layer 1		56×561×1 conv156×56
56×562×2 average pool228×28
Dense
Block 2		28×28$ \left[\begin{array}{l}1 \times 1 {\mathrm{c o n v}} \\3 \times 3 {\mathrm{c o n v}}\end{array}\right] \times 12 $128×28
Transition
Layer 2		28×281×1 conv128×28
28×282×2 average pool214×14
Dense
Block 3		14×14$ \left[\begin{array}{l}1 \times 1 {\mathrm{c o n v}} \\3 \times 3 {\mathrm{c o n v}}\end{array}\right] \times 24 $114×14
Transition
Layer 3		14×141×1 conv114×14
14×142×2 average pool27×7
Dense
Block 4		7×7$ \left[\begin{array}{l}1 \times 1 {\mathrm{c o n v}} \\3 \times 3 {\mathrm{c o n v}}\end{array}\right] \times 16 $17×7
Classification Layer7×7Global average pool—1×1
1×11000
Fully-connected		—1000
Tab.2 DenseNet121 network structure
Fig.6 Composite scaling method of EfficientNet-B0 model
网络模型FPS /(帧·s?1)W/106改进算法ACC/%F1/%R/%
MSCBTAFB
ConvNext73.510.5——98.7798.0198.20
A-ConvNext71.510.5—√99.1898.8398.83
M-ConvNext74.711.2√—99.2198.7698.93
MA-ConvNext75.212.7√√99.3899.0999.21
Tab.3 Results of ablation experiment
Fig.7 Thermogram of identifying black rot apple leaves by each module
Fig.8 Thermogram of identifying rusty apple leaves by each module
Fig.9 Thermogram of identifying scab apple leaves by each module
网络模型ACC/%
原始数据集扩充后数据集
ConvNext74.4598.77
MA-ConvNext78.3499.38
Tab.4 Comparison of accuracy before and after data expasion
网络模型S/106F/109P/106ACC/%F1/%R/%
ResNet50[20]90.004.1325.5695.4093.1793.70
Inception-V3[21]83.402.8627.1697.0995.4995.81
DenseNet121[22]27.602.907.9895.9494.0494.43
MobileNet-V3[23]2.280.235.4891.8388.4189.33
EfficientNet-V2[24]77.802.8921.4695.1192.7893.18
ConvNext[25]106.004.4928.5998.7798.0198.20
MA-ConvNext106.004.6027.8699.3899.0999.21
Tab.5 Comparison of different models on test set
Fig.10 Comparison of confusion matrix of network before and after improvement
网络模型ACC/%LossS/106F/109P/106
MA-ConvNext99.380.037106.04.6027.86
DenseNet12195.940.01927.62.907.98
EfficientNet-B093.720.06815.60.415.29
Tab.6 Performance of each network in stepwise relational knowledge distillation
网络模型ACC/%S/106F/106P/106
EfficientNet-B093.7215.6410.25.29
关系知识蒸馏
EfficientNet-B0		94.7515.615.65.29
分步关系知识蒸馏
EfficientNet-B0		95.4815.615.65.29
Tab.7 Comparison of different distillation methods in "student" network
网络模型S/106F/109P/106ACC/%
MA-ConvNext106.04.627.8699.38
DenseNet12127.62.97.9895.94
关系知识蒸馏
DenseNet121		27.62.97.9894.57
Tab.8 Distillation results of "intermediate" network relational knowledge
Fig.11 Accuracy curve during "intermediate" network training
网络模型ACC/%LossS/106F/109
MA-ConvNext99.380.037106.04.60
关系知识蒸馏
DenseNet121		94.571.41227.62.90
分步关系知识蒸馏
EfficientNet-B0		95.480.82315.60.41
Tab.9 Distillation results of stepwise relational knowledge
[1]   翟肇裕, 曹益飞, 徐焕良, 等 农作物病虫害识别关键技术研究综述[J]. 农业机械学报, 2021, 52 (7): 1- 18
ZHAI Zhaoyu, CAO Yifei, XU Huanliang, et al Review of key techniques for crop disease and pest detection[J]. Transactions of the Chinese Society for Agricultural Machinery, 2021, 52 (7): 1- 18
[2]   孟子恒, 朱海燕, 刘学忠 农业产业集聚对农业经济增长的影响研究: 基于苹果产业的实证分析[J]. 中国农业资源与区划, 2022, 43 (2): 231- 239
MENG Ziheng, ZHU Haiyan, LIU Xuezhong Research on the impact of agricultural industry agglomeration on agricultural economic growth: an empirical analysis based on apple industry[J]. China Agricultural Resources and Zoning, 2022, 43 (2): 231- 239
[3]   岳喜申 深度学习在农作物病害识别中的研究进展[J]. 安徽农学通报, 2024, 30 (6): 100- 103
YUE Xishen Research progress of deep learning in crop disease identification[J]. Anhui Agricultural Bulletin, 2024, 30 (6): 100- 103
[4]   盛帅, 段先华, 胡维康, 等. Dynamic-YOLOX: 复杂背景下的苹果叶片病害检测模型[J]. 计算机科学与探索, 2024, 18(8): 2118−2129.
SHENG Shuai, DUAN Xianhua, HU Weikang, et al. Dynamic-YOLOX: apple leaf disease detection model under complex background [J]. Computer Science and Exploration , 2024, 18(8): 2118−2129.
[5]   李佼隆. 基于LeafGAN的数据增强及木薯叶片病害分类识别系统研究[D]. 雅安: 四川农业大学, 2023.
LI Jiaolong. Research on data enhancement and cassava leaf disease classification and identification system based on LeafGAN [D]. Yaan: Sichuan Agricultural University, 2023.
[6]   李云红, 张蕾涛, 李丽敏, 等 基于CycleGAN-IA方法和M-ConvNext网络的苹果叶片病害图像识别[J]. 农业机械学报, 2024, 55 (4): 204- 212
LI Yunhong, ZHANG Leitao, LI Limin, et al Apple leaf disease image recognition based on CycleGAN-IA method and M-ConvNet network[J]. Journal of Agricultural Machinery, 2024, 55 (4): 204- 212
[7]   ZHANG Y, HUANG S, ZHOU G, et al Identification of tomato leaf diseases based on multi-channel automatic orientation recurrent attention network[J]. Computers and Electronics in Agriculture, 2023, 205: 107605
doi: 10.1016/j.compag.2022.107605
[8]   HAQUE M A, MARWAHA S, ARORA A, et al A lightweight convolutional neural network for recognition of severity stages of maydis leaf blight disease of maize[J]. Frontiers in Plant Science, 2022, 13: 1077568
doi: 10.3389/fpls.2022.1077568
[9]   MA Z, WANG Y, ZHANG T, et al Maize leaf disease identification using deep transfer convolutional neural networks[J]. International Journal of Agricultural and Biological Engineering, 2022, 15 (5): 187- 195
doi: 10.25165/j.ijabe.20221505.6658
[10]   YANG L, YU X, ZHANG S, et al GoogLeNet based on residual network and attention mechanism identification of rice leaf diseases[J]. Computers and Electronics in Agriculture, 2023, 204: 107543
doi: 10.1016/j.compag.2022.107543
[11]   ELFATIMI E, ERVIGIT R, ELFATIMI L Beans leaf diseases classification using mobilenet models[J]. IEEE Access, 2022, 10: 9471- 9482
doi: 10.1109/ACCESS.2022.3142817
[12]   LEE S H, CHAN C S, WILKIN P, et al. Deep-plant: plant identification with convolutional neural networks [C]// 2015 IEEE International Conference on Image Processing . Quebec City: IEEE, 2015: 452−456.
[13]   SLADOJEVIC S, ARSENOVIC M, ANDERLA A, et al Deep neural networks based recognition of plant diseases by leaf image classification[J]. Computational Intelligence and Neuroscience, 2016, (1): 1- 11
[14]   BRAHIMI M, BOUKHALFA K, MOUSSAOUI A Deep learning for tomato diseases: classification and symptoms visualization[J]. Applied Artificial Intelligence, 2017, 31 (4): 299- 315
doi: 10.1080/08839514.2017.1315516
[15]   FERENTINOS K P Deep learning models for plant disease detection and diagnosis[J]. Computers and Electronics in Agriculture, 2018, 145: 311- 318
doi: 10.1016/j.compag.2018.01.009
[16]   HASSAN S M, MAJI A K, JASINSKI M, et al Identification of plant-leaf diseases using CNN and transfer-learning approach[J]. Electronics, 2021, 10 (12): 1388
doi: 10.3390/electronics10121388
[17]   彭玉寒, 李书琴 基于重参数化MobileNetV2的农作物叶片病害识别模型[J]. 农业工程学报, 2023, 39 (17): 132- 140
PENG Yuhan, LI Shuqin Identification model of crop leaf diseases based on re-parameterized MobileNetV2[J]. Journal of Agricultural Engineering, 2023, 39 (17): 132- 140
[18]   李大华, 仲婷, 王笋, 等 基于改进ShuffleNet v2的轻量化番茄叶片病害识别[J]. 江苏农业科学, 2024, 52 (3): 220- 228
LI Dahua, ZHONG Ting, WANG Sun, et al Identification of light tomato leaf diseases based on improved ShuffleNet v2[J]. Jiangsu Agricultural Sciences, 2024, 52 (3): 220- 228
[19]   CHEN J, ZHANG D, ZEB A, et al Identification of rice plant diseases using lightweight attention networks[J]. Expert Systems with Applications, 2021, 169: 114514
doi: 10.1016/j.eswa.2020.114514
[20]   HE K M, ZHANG X Y, REN S Q, et al. Deep residual learning for image recognition [C]// IEEE Conference on Computer Vision and Pattern Recognition . Columbus: IEEE, 2016: 770−778.
[21]   SZEGEDY C, VANHOUCKE V, IOFFE S, et al. Rethinking the inception architecture for computer vision [C ]// IEEE Conference on Computer Vision and Pattern recognition . Columbus: IEEE, 2016: 2818−2826.
[22]   HUANG G, LIU Z, VAN DER MAATEN L, et al. Densely connected convolutional networks [C]// IEEE Conference on Computer Vision and Pattern Recognition . Honolulu: IEEE, 2017: 4700−4708.
[23]   HOWARD A, SANDLER M, CHU G, et al. Searching for mobilenetv3 [C]// IEEE/CVF International Conference on Computer Vision . Seoul: IEEE, 2019: 1314−1324.
[24]   TAN M, LE Q. Efficientnet: rethinking model scaling for convolutional neural networks [C]// International Conference on Machine Learning . Long Beach: IEEE, 2019: 6105−6114.
[1] Canlin LI,Xinyue WANG,Lizhuang MA,Zhiwen SHAO,Wenjiao ZHANG. Image cartoonization incorporating attention mechanism and structural line extraction[J]. Journal of ZheJiang University (Engineering Science), 2024, 58(8): 1728-1737.
[2] Zhongliang LI,Qi CHEN,Lin SHI,Chao YANG,Xianming ZOU. Dynamic knowledge graph completion of temporal aware combination[J]. Journal of ZheJiang University (Engineering Science), 2024, 58(8): 1738-1747.
[3] Shuhan WU,Dan WANG,Yuanfang CHEN,Ziyu JIA,Yueqi ZHANG,Meng XU. Attention-fused filter bank dual-view graph convolution motor imagery EEG classification[J]. Journal of ZheJiang University (Engineering Science), 2024, 58(7): 1326-1335.
[4] Xianwei MA,Chaohui FAN,Weizhi NIE,Dong LI,Yiqun ZHU. Robust fault diagnosis method for failure sensors[J]. Journal of ZheJiang University (Engineering Science), 2024, 58(7): 1488-1497.
[5] Jun YANG,Chen ZHANG. Semantic segmentation of 3D point cloud based on boundary point estimation and sparse convolution neural network[J]. Journal of ZheJiang University (Engineering Science), 2024, 58(6): 1121-1132.
[6] Yuntang LI,Hengjie LI,Kun ZHANG,Binrui WANG,Shanyue GUAN,Yuan CHEN. Recognition of complex power lines based on novel encoder-decoder network[J]. Journal of ZheJiang University (Engineering Science), 2024, 58(6): 1133-1141.
[7] Qing SHU,Xiping LIU,Zhao TAN,Xi LI,Changxuan WAN,Dexi LIU,Guoqiong LIAO. SQL generation method based on dependency relational graphattention network[J]. Journal of ZheJiang University (Engineering Science), 2024, 58(5): 908-917.
[8] Zhiwei XING,Shujie ZHU,Biao LI. Airline baggage feature perception based on improved graph convolutional neural network[J]. Journal of ZheJiang University (Engineering Science), 2024, 58(5): 941-950.
[9] Yi LIU,Yidan CHEN,Lin GAO,Jiao HONG. Lightweight road extraction model based on multi-scale feature fusion[J]. Journal of ZheJiang University (Engineering Science), 2024, 58(5): 951-959.
[10] Cuiting WEI,Weijian ZHAO,Bochao SUN,Yunyi LIU. Intelligent rebar inspection based on improved Mask R-CNN and stereo vision[J]. Journal of ZheJiang University (Engineering Science), 2024, 58(5): 1009-1019.
[11] Hai HUAN,Yu SHENG,Chenxi GU. Global guidance multi-feature fusion network based on remote sensing image road extraction[J]. Journal of ZheJiang University (Engineering Science), 2024, 58(4): 696-707.
[12] Shancheng TANG,Jianhui LU,Ying ZHANG,Zicheng JIN,Anxin ZHAO. Unsupervised surface defect detection of magnetic tile for repair of suspected area defects[J]. Journal of ZheJiang University (Engineering Science), 2024, 58(4): 718-728.
[13] Mingjun SONG,Wen YAN,Yizhao DENG,Junran ZHANG,Haiyan TU. Light-weight algorithm for real-time robotic grasp detection[J]. Journal of ZheJiang University (Engineering Science), 2024, 58(3): 599-610.
[14] Xinhua YAO,Tao YU,Senwen FENG,Zijian MA,Congcong LUAN,Hongyao SHEN. Recognition method of parts machining features based on graph neural network[J]. Journal of ZheJiang University (Engineering Science), 2024, 58(2): 349-359.
[15] Siyi QIN,Shaoyan GAI,Feipeng DA. Video object detection algorithm based on multi-level feature aggregation under mixed sampler[J]. Journal of ZheJiang University (Engineering Science), 2024, 58(1): 10-19.