RGB imaging-based detection of rice leaf blast spot and resistance evaluation at the canopy scale

doi:10.3785/j.issn.1008-9209.2021.05.131

Journal of Zhejiang University (Agriculture and Life Sciences)

2021, Vol. 47

Issue (4): 415-428 DOI: 10.3785/j.issn.1008-9209.2021.05.131

Special Topic: Crop Phenotyping Technologies and Applications

RGB imaging-based detection of rice leaf blast spot and resistance evaluation at the canopy scale

Pengyao XIE^1,²(

),Haowei FU³,Zheng TANG^1,²,Zhihong MA^1,²,Haiyan CEN^1,²(

)

^1.College of Biosystems Engineering and Food Science, Zhejiang University, Hangzhou 310058, China
^2.Key Laboratory of Spectroscopy Sensing, Ministry of Agriculture and Rural Affairs, Hangzhou 310058, China
^3.Jiaxing Academy of Agricultural Sciences, Jiaxing 314016, Zhejiang, China

Download:

HTML

HTML ( )

PDF(14414KB)
Export: BibTeX | EndNote (RIS)

Abstract

Visual inspection of rice leaf blast resistance is time-consuming and labor-intensive with low accuracy. Therefore, this study aims to identify and detect rice leaf blast spots based on RGB imaging of rice canopy combined with mask regions with convolutional neural network (Mask-RCNN), and develop multiple classification models to quantify the number of disease spots and evaluate the association between the number of disease spots and the resistance level by analyzing the quantitative information of different categories of disease spots in RGB images of rice. First, we collected RGB images from different rice breeding lines at the seedling stage, including japonica lines, early indica lines and indica recovery lines. Preprocessing and labeling of the input images were then performed. A Mask-RCNN model for the recognition of rice leaf blast spots was developed to perform the rectangular frame detection, mask segmentation and classification. The classification result of rice leaf blast spots with the mean intersection over union (mIoU) of 0.603 was achieved. The mean average precision (mAP) of the test dataset was 0.716, when the intersection over union (IoU) threshold of 0.5 was used. Among all the classification models, Gaussian process support vector machine obtained the highest prediction accuracy of 94.30% (proportion of disease spots in each category corresponding to different resistances) on the test dataset. The above results demonstrate that RGB images of rice canopy combined with Mask-RCNN have the great potential for the accurate identification of rice leaf blast spots, and the number of detected disease spots is highly correlated with the rice leaf blast resistance level. The proposed method is promising for efficient selection of disease-resistant rice varieties in breeding.

Key words： canopy scale rice disease leaf blast detection deep learning resistance evaluation

Received: 13 May 2021 Published: 02 September 2021

CLC:

TP 751

Corresponding Authors: Haiyan CEN E-mail: pyxie@zju.edu.cn;hycen@zju.edu.cn

	Service
	E-mail this article
	Add to my bookshelf
	Add to citation manager
	E-mail Alert
	RSS
	Articles by authors
	Pengyao XIE
	Haowei FU
	Zheng TANG
	Zhihong MA
	Haiyan CEN

Cite this article:

Pengyao XIE,Haowei FU,Zheng TANG,Zhihong MA,Haiyan CEN. RGB imaging-based detection of rice leaf blast spot and resistance evaluation at the canopy scale. Journal of Zhejiang University (Agriculture and Life Sciences), 2021, 47(4): 415-428.

URL:

http://www.zjujournals.com/agr/10.3785/j.issn.1008-9209.2021.05.131 OR http://www.zjujournals.com/agr/Y2021/V47/I4/415

基于RGB图像的冠层尺度水稻叶瘟病斑检测与抗性评估

针对依赖人工主观判断水稻叶瘟抗性费时费力且准确率低的问题，本文提出了一种基于水稻冠层尺度RGB图像和掩膜区域卷积神经网络（mask regions with convolutional neural network, Mask-RCNN）深度学习框架的水稻叶瘟病斑识别检测方法，通过分析水稻RGB图像中不同类别病斑的数量信息，构建多种分类模型来评估病斑数量和抗性水平之间的关联性。首先采集包括粳稻品系、早籼品系和籼型恢复系等不同品系的水稻育种材料的苗期RGB图像，然后通过对输入图像进行预处理和标记，最终建立了用于识别水稻叶瘟病斑的Mask-RCNN模型，实现了叶瘟病斑的矩形框检测、掩膜分割和分类，其平均交并比（mean intersection over union, mIoU）为0.603。当采用0.5的交并比（intersection over union, IoU）阈值时，测试数据集的病斑检测平均准确率均值（mean average precision, mAP）为0.716。在基于病斑数量的抗性评估模型中，高斯过程支持向量机在测试数据集上取得了94.30%的最高抗性评估准确率。研究结果表明，基于水稻冠层RGB图像和Mask-RCNN模型可实现水稻叶瘟病的准确识别，检测的病斑数量特征和叶瘟抗性水平高度相关。本研究为水稻抗病性品种的高效选育提供了技术支撑。

关键词： 冠层尺度, 水稻病害, 叶瘟检测, 深度学习, 抗性评估

Fig. 1 Examples of original images and segmented imagesRed solid line boxes indicate 1 500×1 500 cropping area; blue dashed line boxed images indicate that the camera field of view covers the block of planted single line; the other unboxed images indicate that the camera focuses on the local disease spots.

Fig. 2 Types of rice leaf blast spotsA. Chronic disease spots; B. Acute disease spots; C. Brown disease spots; D. White disease spots.

Fig. 3 Data processing flowchart

Fig. 4 Mask-RCNN model architectureFPN: Feature pyramid network; RPN: Region proposal network; RoI: Region of interest.

Fig. 5 Mask-RCNN loss function change curve

Fig. 6 Mask-RCNN prediction result examplesThe rectangular dashed boxes and masks are model-generated results with disease category prediction results and confidence coefficients.

Fig. 8 Partial visualization of Mask-RCNN model output

Fig. 9 Data distribution in classification datasetA. Distribution of rice resistance in each line; B. Proportion of disease spots in each category corresponding to different resistances. R: Resistant type; M: Medium susceptible type; S: Susceptible type; S＋: Highly susceptible type.

Table 1 Classification results of different classification models

Fig. 10 Confusion matrix of Gaussian process support vector machine classification results

Fig. 11 Chronic disease spot features

Fig. 12 Comparative analysis of spot masks generated by manual annotation and prediction resultsA. Spot annotation and training results; B. Percentage of manual annotation mask and prediction result mask.

Fig. 13 Challenges in recognition and segmentation of rice disease spots at the canopy scale1-2: Two cases of disease spots being obscured; 3: Defocus leads to lower clarity of disease spots; 4: Water surface highlighting interferes with disease spot detection.


[1]	OERKE E C. Remote sensing of diseases. Annual Review of Phytopathology, 2020,58(1):225-252. DOI:10.1146/annurev-phyto-010820-012832 doi: 10.1146/annurev-phyto-010820-012832

[2]	RATH A K, MEHER J K. Disease detection in infected plant leaf by computational method. Archives of Phytopathology and Plant Protection, 2019,52(19/20):1348-1358. DOI:10.1080/03235408.2019.1708546 doi: 10.1080/03235408.2019.1708546

[3]	贺闽,尹俊杰,冯志明,等.水稻稻瘟病和纹枯病抗性鉴定方法.植物学报,2020,55(5):577-587. DOI:10.11983/CBB20100 HE M, YIN J J, FENG Z M, et al. Identification method of rice blast and sheath blight resistance. Chinese Bulletin of Botany, 2020,55(5):577-587. (in Chinese with English abstract) doi: 10.11983/CBB20100

[4]	SENAN N, AAMIR M, IBRAHIM R, et al. An efficient convolutional neural network for paddy leaf disease and pest classification. International Journal of Advanced Computer Science and Applications, 2020,11(7):116-122. DOI:10.14569/IJACSA.2020.0110716 doi: 10.14569/IJACSA.2020.0110716

[5]	张凝,杨贵军,赵春江,等.作物病虫害高光谱遥感进展与展望.遥感学报,2021,25(1):403-422. DOI:10.118340rs.20210196 ZHANG N, YANG G J, ZHAO C J, et al. Progress and prospect of hyperspectral remote sensing of crop pests and diseases. National Remote Sensing Bulletin, 2021,25(1):403-422. (in Chinese with English abstract) doi: 10.118340rs.20210196

[6]	GUCHAIT N, BHAKTA I, PHADIKAR S, et al. Visual computing for blast and brown spot disease detection in rice leaves//Proceedings of the 2nd International Conference on Communication, Devices and Computing. Singapore: Springer, 2020:595-606. DOI:10.1007/978-981-15-0829-5_56 doi: 10.1007/978-981-15-0829-5_56

[7]	BAKAR M N ABU, ABDULLAH A H, ABDUL RAHIM N, et al. Rice leaf blast disease detection using multi-level colour image thresholding. Journal of Telecommunication, Electronic and Computer Engineering, 2018,10:1-6.

[8]	PEYMAN S H, ZIARATGAHI A B, JAFARI A, et al. Exploring the possibility of using digital image processing technique to detect diseases of rice leaf. Journal of Agricultural Machinery, 2016,6(1):69-79. DOI:10.22067/jam.v6i1.32031 doi: 10.22067/jam.v6i1.32031

[9]	GAYATHRI DEVI T, NEELAMEGAM P. Image processing based rice plant leaves diseases in Thanjavur, Tamilnadu. Cluster Computing, 2019,22:S13415-S13428. DOI:10.1007/s10586-018-1949-x doi: 10.1007/s10586-018-1949-x

[10]	LARIJANI M R, ASLIARDEH E A, KOZEGAR E, et al. Evaluation of image processing technique in identifying rice blast disease in field conditions based on KNN algorithm improvement by K-means. Food Science & Nutrition, 2019,7(12):3922-3930. DOI:10.1002/fsn3.1251 doi: 10.1002/fsn3.1251

[11]	KANG J J, MA Y, XIAO M H, et al. Rice blast recognition based on image processing and BP neural network. International Agricultural Engineering Journal, 2018,27(1):250-256.

[12]	RAMESH S, VYDEKI D. Rice blast disease detection and classification using machine learning algorithm//2018 2nd International Conference on Micro-Electronics and Telecom-munication Engineering (ICMETE). New York, U.S.: IEEE, 2018:255-259. DOI:10.1109/ICMETE.2018.00063 doi: 10.1109/ICMETE.2018.00063

[13]	LIANG W J, ZHANG H, ZHANG G F, et al. Rice blast disease recognition using a deep convolutional neural network. Scientific Reports, 2019,9(1):2869. DOI:10.1038/s41598-019-38966-0 doi: 10.1038/s41598-019-38966-0

[14]	SHRIVASTAVA V K, PRADHAN M K. Rice plant disease classification using transfer learning of deep convolution neural network. International Archives of the Photogrammetry, Remote Sensing and Spatial Information Sciences, 2019,XLII-3/W6:631-635. DOI:10.5194/isprs-archives-XLII-3-W6-631-2019 doi: 10.5194/isprs-archives-XLII-3-W6-631-2019

[15]	CHEN J D, ZHANG D F, NANEHKARAN Y A, et al. Detection of rice plant diseases based on deep transfer learning. Journal of the Science of Food and Agriculture, 2020,100(7):3246-3256. DOI:10.1002/jsfa.10365 doi: 10.1002/jsfa.10365

[16]	DONG M P, MU S M, SHI A J, et al. Novel method for identifying wheat leaf disease images based on differential amplification convolutional neural network. International Journal of Agricultural and Biological Engineering, 2020,13(4):205-210. DOI:10.25165/j.ijabe.20201304.4826 doi: 10.25165/j.ijabe.20201304.4826

[17]	BURHAN S A, MINHAS D S, TARIQ D A, et al. Comparative study of deep learning algorithms for disease and pest detection in rice crops//2020 12th International Conference on Electronics, Computers and Artificial Intelligence (ECAI). New York, U.S.: IEEE, 2020:1-5. DOI:10.1109/ECAI50035.2020.9223239 doi: 10.1109/ECAI50035.2020.9223239

[18]	SHAHRIAR T M, AYRIN A, HASAN M, et al. An image processing framework to identify rice blast: 2020 International Congress on Human-Computer Interaction, Optimization and Robotic Applications (HORA). New York, U.S.: IEEE, 2020:1-5. DOI: 10.1109/HORA49412.2020.9152912 doi: 10.1109/HORA49412.2020.9152912

[19]	CHEN J S, GONG M L, WANG D, et al. Leaf blast spot detection method based on Linknet//Second Target Recognition and Artificial Intelligence Summit Forum Proceedings of SPIE. [S. l.: s. n.], 2020,1142742:1-6. DOI:10.1117/12.2553085 doi: 10.1117/12.2553085

[20]	ZHOU G X, ZHANG W Z, CHEN A B, et al. Rapid detection of rice disease based on FCM-KM and faster R-CNN fusion. IEEE Access, 2019,7:143190-143206. DOI:10.1109/ACCESS.2019.2943454 doi: 10

[21]	黄双萍,孙超,齐龙,等.基于深度卷积神经网络的水稻穗瘟病检测方法.农业工程学报,2017,33(20):169-176. DOI:10.11975/j.issn.1002-6819.2017.20.021 HUANG S P, SUN C, QI L, et al. Detection method of rice ear blast based on deep convolutional neural network. Transactions of the CSAE, 2017,33(20):169-176. (in Chinese with English abstract) doi: 10.11975/j.issn.1002-6819.2017.20.021

[22]	YANGSEON K, ROH J H, KIM H Y, et al. Early forecasting of rice blast disease using long short-term memory recurrent neural networks. Sustainability, 2017,10(1):34. DOI:10.3390/su10010034 doi: 10.3390/su10010034

[23]	HE K M, GKIOXARI G, DOLLáR P, et al. Mask-RCNN//IEEE International Conference on Computer Vision (ICCV). New York, U.S.: IEEE, 2017:2980-2988. DOI:10.1109/ICCV.2017.322 doi: 10.1109/ICCV.2017.322

[24]	TODA Y, OKURA F. How convolutional neural networks diagnose plant disease?Plant Phenomics, 2019,2019:9237136. DOI:10.34133/2019/9237136 doi: 10.34133/2019/9237136

[25]	ZEILER M D, FERGUS R. Visualizing and Understanding Convolutional Networks. Cham, Switzerland: Springer, 2014:818-833. DOI:10.1007/978-3-319-10590-1_53 doi: 10.1007/978-3-319-10590-1_53

[26]	SHRIVASTAVA V K, PRADHAN M K. Rice plant disease classification using color features: a machine learning paradigm. Journal of Plant Pathology, 2021,103(1):17-26. DOI:10.1007/s42161-020-00683-3 doi: 10.1007/s42161-020-00683-3

[27]	ZHANG J, YANG L F, HOU J Y, et al. Recognition of rice leaf diseases based on salient characteristics//2018 13th World Congress on Intelligent Control and Automation. [S. l.: s. n.], 2018:801-806. DOI:10.1109/WCICA.2018.8630414 doi: 10.1109/WCICA.2018.8630414

[28]	DUPUIS D J, VICTORIAFESER M P. Fast robust model selection in large datasets. Journal of the American Statistical Association, 2011,106(493):203-212. DOI:10.1198/jasa.2011.tm09650 doi: 10.1198/jasa.2011.tm09650

[29]	CHEN L, YUAN Y. Agricultural Disease Image Dataset for Disease Identification Based on Machine Learning. Cham, Switzerland: Springer, 2019:263-274. DOI:10.1007/978-3-030-28061-1_26 doi: 10.1007/978-3-030-28061-1_26

[30]	林相泽,朱赛华,张俊媛,等.基于迁移学习和Mask R-CNN的稻飞虱图像分类方法.农业机械学报,2019,50(7):201-207. DOI:10.6041/j.issn.1000-1298.2019.07.021 LIN X Z, ZHU S H, ZHANG J Y, et al. Rice planthopper image classification method based on transfer learning and Mask R-CNN. Transactions of the Chinese Society for Agricultural Machinery, 2019,50(7):201-207. (in Chinese with English abstract) doi: 10.6041/j.issn.1000-1298.2019.07.021

[31]	OLLINGER S V. Sources of variability in canopy reflectance and the convergent properties of plants. The New Phytologist, 2011,189(2):375-394. DOI:10.1111/j.1469-8137.2010.03536.x doi: 10.1111/j.1469-8137.2010.03536.x

[32]	ALENYA G, DELLEN B, FOIX S, et al. Robotized plant probing: leaf segmentation utilizing time-of-flight data. IEEE Robotics & Automation Magazine, 2013,20(3):50-59. DOI:10.1109/MRA.2012.2230118 doi: 10.1109/MRA.2012.2230118

[33]	SEPULVEDA D, FERNANDEZ R, NAVAS E, et al. Robotic aubergine harvesting using dual-arm manipulation. IEEE Access, 2020,8:121889-121904. DOI:10.1109/ACCESS.2020.3006919 doi: 10.1109/ACCESS.2020.3006919

[34]	FENG L, WU B H, ZHU S S, et al. Investigation on data fusion of multisource spectral data for rice leaf diseases identification using machine learning methods. Frontiers in Plant Science, 2020,11:577063. DOI:10.3389/fpls.2020.577063 doi: 10.3389/fpls.2020.577063

[35]	冯雷,柴荣耀,孙光明,等.基于多光谱成像技术的水稻叶瘟检测分级方法研究.光谱学与光谱分析,2009,29(10):2730-2733. DOI:10.3964/j.issn.1000-0593(2009)10-2730-04 FENG L, CHAI R Y, SUN G M, et al. Research on detection and classification method of rice leaf blast based on multispectral imaging technology. Spectroscopy and Spectral Analysis, 2009,29(10):2730-2733. (in Chinese with English abstract) doi: 10.3964/j.issn.1000-0593(2009)10-2730-04

[1]	Xiaohui WANG,Kunpeng ZHOU. Research on recognition methods for red tomato image in the natural environment[J]. Journal of Zhejiang University (Agriculture and Life Sciences), 2021, 47(3): 395-403.

Viewed

Full text

Abstract

Cited

Shared

Discussed