基于深度学习方法和RGB影像的玉米雄穗分割

doi:10.3785/j.issn.1008-9209.2021.03.121

浙江大学学报（农业与生命科学版）

2021, Vol. 47

Issue (4): 451-463 DOI: 10.3785/j.issn.1008-9209.2021.03.121

作物表型分析技术及应用专题

基于深度学习方法和RGB影像的玉米雄穗分割

余汛^1,²(

),王哲¹,景海涛¹(

),金秀良²(

),聂臣巍²,白怡²,王铮³

^1.河南理工大学测绘与国土信息工程学院，河南焦作 454000
^2.中国农业科学院作物科学研究所，北京 100081
^3.成都理工大学能源学院，成都 610059

Maize tassel segmentation based on deep learning method and RGB image

Xun YU^1,²(

),Zhe WANG¹,Haitao JING¹(

),Xiuliang JIN²(

),Chenwei NIE²,Yi BAI²,Zheng WANG³

^1.School of Surveying and Land Information Engineering, Henan Polytechnic University, Jiaozuo 454000, Henan, China
^2.Institute of Crop Sciences, Chinese Academy of Agricultural Sciences, Beijing 100081, China
^3.College of Energy, Chengdu University of Technology, Chengdu 610059, China

全文: PDF(8614 KB) HTML

摘要：

为检验深度学习方法对不同品种玉米雄穗在不同生育时期的分割精度和稳定性，利用2019年7月—9月于河南省新乡市中国农业科学院试验基地内采集的RGB影像，通过构建以轻量级网络为特征提取层的PspNet、DeepLab V3＋、SegNet和U-Net 4种模型，比较不同模型对玉米雄穗分割精度的差异。结果显示：U-Net模型对不同生育时期玉米品种的雄穗分割精度最高（mIoU=0.780）。该模型在玉米雄穗不同生长阶段的分割精度总体上较好（mIoU=0.703～0.798），其中在完全抽雄期的分割精度最高（mIoU=0.798）；U-Net模型对不同玉米品种的雄穗分割精度差异明显，但对所有玉米品种雄穗的平均分割精度较高（mIoU=0.749），其中对郑单958（ZD958）的分割精度最高（mIoU=0.814）。表明U-Net模型对玉米雄穗分割具有较好的普适性与鲁棒性，为今后玉米表型试验中对雄穗的监测提供了一种有效的方法。

关键词： RGB影像; 深度学习; 特征提取层; 玉米雄穗; 分割

Abstract:

This study focuses on the accuracy and stability of deep learning method for maize tassel segmentation at different tasseling stages and varieties. The RGB images were collected in the experimental base of Chinese Academy of Agricultural Sciences in Xinxiang City of Henan Province from July to September in 2019, and four models, PspNet, DeepLab V3＋, SegNet and U-Net, based on the lightweight network as the feature extraction layer, were applied to compare the accuracy of different models for maize tassel segmentation. Then, the U-Net model with the best segmentation accuracy (mIoU=0.780) was selected to segment the maize tassel in different varieties at different tasseling stages. The results showed that the accuracy of U-Net model at different tasseling stages was generally better (mIoU=0.703 to 0.798), and the segmentation accuracy of maize tassel in fully emerged tassel stage was the highest (mIoU=0.798); the segmentation accuracy of maize tassel in different varieties was significantly different, but the average segmentation accuracy of maize tassel for all varieties was higher (mIoU=0.749), and the segmentation accuracy of Zhengdan 958 (ZD958) was the highest (mIoU=0.814). In summary, the U-Net model has good universality and robustness for maize tassel segmentation, which provides an effective method for tassel monitoring in maize phenotypic test in the future.

Key words: RGB image deep learning feature extraction layer maize tassel segmentation

收稿日期: 2021-03-12 出版日期: 2021-09-02

CLC:

TP 75

基金资助: 国家自然科学基金面上项目(42071426);中国农业科学院基本科研业务费专项院级统筹项目(Y2020YJ07);中国农业科学院科技创新工程和基本科研业务费专项(ICS2020YJ01BX)

通讯作者: 景海涛,金秀良 E-mail: 211804020012@home.hpu.edu.cn;jht@hpu.edu.cn;jinxiuliang@caas.cn

作者简介: 余汛（https://orcid.org/0000-0002-1780-7711），E-mail：211804020012@home.hpu.edu.cn

	服务
	把本文推荐给朋友
	加入引用管理器
	E-mail Alert
	RSS
	作者相关文章
	余汛
	王哲
	景海涛
	金秀良
	聂臣巍
	白怡
	王铮

引用本文:

余汛,王哲,景海涛,金秀良,聂臣巍,白怡,王铮. 基于深度学习方法和RGB影像的玉米雄穗分割[J]. 浙江大学学报（农业与生命科学版）, 2021, 47(4): 451-463.

Xun YU,Zhe WANG,Haitao JING,Xiuliang JIN,Chenwei NIE,Yi BAI,Zheng WANG. Maize tassel segmentation based on deep learning method and RGB image. Journal of Zhejiang University (Agriculture and Life Sciences), 2021, 47(4): 451-463.

链接本文:

http://www.zjujournals.com/agr/CN/10.3785/j.issn.1008-9209.2021.03.121 或 http://www.zjujournals.com/agr/CN/Y2021/V47/I4/451

图1 地面图像采集场景及设备A.野外拍摄场景；B.索尼DSC-RX0M2数码相机；C.吊杆。

表1 本研究中使用的几个数据集的特征

图2 不同生育时期的影像划分示例A.未完全抽雄期的玉米雄穗图像；B.完全抽雄期的玉米雄穗图像。

图3 使用Labelme软件对玉米雄穗的标注示例

图4 SegNet模型的网络框架

图5 PspNet模型的网络框架

图6 DeepLab V3＋模型的网络框架

图7 U-Net模型的网络框架

表2 MobileNet的网络结构

图8 残差网络结构

表3 MobileNet V2的网络结构

表4 混淆矩阵

图9 4种不同深度学习模型对全生育时期数据集中玉米雄穗的分割结果

表5 基于全生育时期数据集的各个模型精度评价

图10 以MobileNet为特征提取层的U-Net模型对不同生育时期数据集中玉米雄穗的分割精度A.全生育时期；B.完全抽雄期；C.未完全抽雄期。

表6 以MobileNet为特征提取层的U-Net模型对各生育时期数据集的分割精度评价

图11 以MobileNet为特征提取层的U-Net模型对不同品种数据集中玉米雄穗的分割精度

表7 以MobileNet为特征提取层的U-Net模型对各品种数据集的分割精度评价

1	JIN X L, ZARCO-TEJADA P, SCHMIDHALTER U, et al. High-throughput estimation of crop traits: a review of ground and aerial phenotyping platforms. IEEE Geoscience and Remote Sensing Magazine, 2021,9(1):200-231. DOI:10.1109/mgrs.2020.2998816 doi: 10
2	ARAUS J L, CAIRNS J E. Field high-throughput phenotyping: the new crop breeding frontier. Trends in Plant Science, 2014,19(1):52-61. DOI:10.1016/j.tplants.2013.09.008 doi: 10.1016/j.tplants.2013.09.008
3	DEERY D, JIMENEZ-BERNI J, JONES H, et al. Proximal remote sensing buggies and potential applications for field-based phenotyping. Agronomy, 2014,4(3):349-379. DOI:10.3390/agronomy4030349 doi: 10
4	LU H, CAO Z G, XIAO Y, et al. Region-based colour modelling for joint crop and maize tassel segmentation. Biosystems Engineering, 2016,147:139-150. DOI:10.1016/j.biosystemseng.2016.04.007 doi: 10.1016/j.biosystemseng.2016.04.007
5	HINTON G E, SALAKHUTDINOV R R. Reducing the dimensionality of data with neural networks. Science, 2006,313(5786):504-507. DOI:10.1126/science.1127647 doi: 10.1126/science.1127647
6	LECUN Y, BENGIO Y, HINTON G. Deep learning. Nature, 2015,521(7553):436-444. DOI:10.1038/nature14539 doi: 10.1038/nature14539
7	MESNIL G, DAUPHIN Y, GLOROT X, et al. Unsupervised and transfer learning challenge: a deep learning approach//JMLR Workshop and Conference Proceedings. [S. l.: s. n.], 2012:97-110.
8	DONAHUE J, JIA Y Q, VINYALS O, et al. DeCAF: a deep convolutional activation feature for generic visual recognition//Proceedings of the 31st International Conference on International Conference on Machine Learning. [S. l.: s. n.], 2014:647-655.
9	RAZAVIAN A S, AZIZPOUR H, SULLIVAN J, et al. CNN features off-the-shelf: an astounding baseline for recognition. New York, U. S.: IEEE, 2014. DOI:10.1109/CVPRW.2014.131 doi: 10.1109/CVPRW.2014.131
10	YOSINSKI J, CLUNE J, BENGIO Y, et al. How trans-ferable are features in deep neural networks?//Advances in Neural Information Processing Stystems. 2014:3320-3328[2021-03-10].
11	ZHANG M Y, JIANG Y, LI C Y, et al. Fully convolutional networks for blueberry bruising and calyx segmentation using hyperspectral transmittance imaging. Biosystems Engineering, 2020,192:159-175. DOI:10.1016/j.biosystemseng.2020.01.018 doi: 10.1016/j.biosystemseng.2020.01.018
12	NI X P, LI C Y, JIANG H Y, et al. Three-dimensional photogrammetry with deep learning instance segmentation to extract berry fruit harvestability traits. ISPRS Journal of Photogrammetry and Remote Sensing, 2021,171:297-309. DOI:10.1016/j.isprsjprs.2020.11.010 doi: 10.1016/j.isprsjprs.2020.11.010
13	GE Y Y, XIONG Y, TENORIO G L, et al. Fruit localization and environment perception for strawberry harvesting robots. IEEE Access, 2019(99):1-11. DOI:10.1109/ACCESS.2019.2946369 doi: 10.1109/ACCESS.2019.2946369
14	WU J T, YANG G J, YANG H, et al. Extracting apple tree crown information from remote imagery using deep learning. Computers and Electronics in Agriculture, 2020,174:105504. DOI:10.1016/j.compag.2020.105504 doi: 10.1016/j.compag.2020.105504
15	HUETHER P, SCHANDRY N, JANDRASITS K, et al. ARADEEPOPSIS, an automated workflow for top-view plant phenomics using semantic segmentation of leaf states. The Plant Cell, 2020,32(12):3674-3688. DOI:10.1105/tpc.20.00318 doi: 10.1105/tpc.20.00318
16	OH M H, OLSEN P, RAMAMURTHY K N. Counting and segmenting sorghum heads. [2021-03-10].
17	BADRINARAYANAN V, HANDA A, CIPOLLA R. SegNet: a deep convolutional encoder-decoder architecture for robust semantic pixel-wise labelling. IEEE Transactions on Pattern Analysis & Machine Intelligence, 2017,39(12):2481-2495. DOI:10.1109/TPAMI.2016.2644615 doi: 10.1109/TPAMI.2016.2644615
18	ZHAO H S, SHI J P, QI X J, et al. Pyramid scene parsing network//2017 IEEE Conference on Computer Vision and Pattern Recognition (CVPR). New York, U. S.: IEEE, 2017:6230-6239. DOI:10.1109/CVPR.2017.660 doi: 10.1109/CVPR.2017.660
19	CHEN L C, ZHU Y K, PAPANDREOU G, et al. Encoder-decoder with atrous separable convolution for semantic image segmentation//European Conference on Computer Vision (ECCV). Cham, Switzerland: Springer, 2018:833-851. DOI:10.1007/978-3-030-01234-2_49 doi: 10.1007/978-3-030-01234-2_49
20	RONNEBERGER O, FISCHER P, BROX T. U-Net: convolutional networks for biomedical image segmentation//International Conference on Medical Image Computing and Computer-Assisted Intervention-MICCAI. Cham, Switzerland: Springer, 2015:234-241. DOI:10.1007/978-3-319-24574-4_28 doi: 10.1007/978-3-319-24574-4_28
21	HOWARD A G, ZHU M L, CHEN B, et al. MobileNets: efficient convolutional neural networks for mobile vision applications. [2021-03-10].
22	CHOLLET F. Xception: deep learning with depthwise separable convolutions//2017 IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR). New York, U.S.: IEEE, 2017. DOI:10.1109/CVPR.2017.195 doi: 10.1109/CVPR.2017.195
23	LU H, CAO Z G, XIAO Y, et al. Towards fine-grained maize tassel flowering status recognition: dataset, theory and practice. Applied Soft Computing, 2017,56:34-45. DOI:10.1016/j.asoc.2017.02.026 doi: 10.1016/j.asoc.2017.02.026

[1]	谢鹏尧,富昊伟,唐政,麻志宏,岑海燕. 基于RGB图像的冠层尺度水稻叶瘟病斑检测与抗性评估[J]. 浙江大学学报（农业与生命科学版）, 2021, 47(4): 415-428.
[2]	束美艳,魏家玺,周也莹,董奇宙,陈浩翀,黄智刚,马韫韬. 基于朴素贝叶斯分类的柑橘叶片溃疡病诊断[J]. 浙江大学学报（农业与生命科学版）, 2021, 47(4): 429-438.
[3]	王晓慧,周昆鹏. 自然环境中的红色番茄图像识别方法研究[J]. 浙江大学学报（农业与生命科学版）, 2021, 47(3): 395-403.
[4]	张万红. 基于图像法的离体小麦叶片几何参数计算[J]. 浙江大学学报（农业与生命科学版）, 2018, 44(6): 748-754.
[5]	张万红，刘文兆. 覆膜玉米冠层图像分割方法[J]. 浙江大学学报(农业与生命科学版), 2017, 43(5): 649-656.
[6]	韩凝, 张秀英, 王小明, 王珂. 基于面向对象的IKONOS 影像香榧树分布信息提取研究[J]. 浙江大学学报(农业与生命科学版), 2009, 35(6): 670-676.
[7]	付峰　应义斌. 生物图像阈值分割方法的研究[J]. 浙江大学学报(农业与生命科学版), 2003, 29(1): 108-112.
[8]	应义斌. 水果图像的背景分割和边缘检测技术研究 [J]. 浙江大学学报(农业与生命科学版), 2000, 26(1): 35-38.

Viewed

Full text

Abstract

Cited

Shared

Discussed