基于机器视觉和机器学习技术的浙贝母外观品质等级区分

doi:10.3785/j.issn.1008-9209.2022.10.181

浙江大学学报（农业与生命科学版）

2023, Vol. 49

Issue (6): 881-892 DOI: 10.3785/j.issn.1008-9209.2022.10.181

农业工程

基于机器视觉和机器学习技术的浙贝母外观品质等级区分

董成烨(

),李东方,冯槐区,龙思放,奚特,周芩安,王俊(

)

浙江大学生物系统工程与食品科学学院，浙江杭州 310058

Classification of Fritillaria thunbergii appearance quality based on machine vision and machine learning technology

Chengye DONG(

),Dongfang LI,Huaiqu FENG,Sifang LONG,Te XI,Qin’an ZHOU,Jun WANG(

)

College of Biosystems Engineering and Food Science, Zhejiang University, Hangzhou 310058, Zhejiang, China

全文: PDF(8568 KB) HTML

摘要：

为区分浙贝母外观品质等级，本研究利用数字电子眼系统及图像标注工具构建浙贝母数据集，选择若干统计学习算法和目标检测算法在该数据集上进行训练与测试。结果表明：目标检测算法YOLO（you only look once）系列YOLO-X所得模型的效果最佳。为优化YOLO-X，根据浙贝母数据集的特点，针对性地向YOLO-X的主干特征提取网络末端嵌入空洞卷积结构，以加强模型对尺度特征的敏感度。改进后模型（空洞率为4）的平均精确率均值为99.01%，对于特级、一级、二级、虫蛀、霉变、破碎浙贝母的平均精确率分别为99.97%、98.33%、98.47%、98.71%、99.73%、98.85%，精确率和召回率的加权调和平均数（F₁）分别为0.99、0.92、0.94、0.97、0.99、0.97。本研究在不增加参数量、计算量或者对算法进行大规模改动的情况下，改善了模型的检测效果，为后续浙贝母检测平台的搭建提供了科学依据。

关键词： 浙贝母; 统计学习; 深度学习; 目标检测; 目标检测算法YOLO-X; 空洞卷积

Abstract:

In order to classify the appearance quality level of Fritillaria thunbergii, the F. thunbergii dataset was constructed with the DigiEye system followed by an image annotation tool. Several statistical learning and object detection algorithms were selected to train and test the F. thunbergii dataset. The results showed that the model trained by the YOLO-X of YOLO (you only look once) series had relatively better performance. In addition, to optimize YOLO-X, according to the unique features of F. thunbergii dataset, a dilated convolution structure was embedded into the end of the backbone feature extraction network of YOLO-X as it could improve the model sensitivity to the dimension feature. The mean average precision (mAP) of the improved model was raised to 99.01%; the average precision (AP) for superfine, level one, level two, moth-eaten, mildewed, and broken F. thunbergii were raised to 99.97%, 98.33%, 98.47%, 98.71%, 99.73%, and 98.85%, respectively; and the weighted harmonic mean of precision and recall (F₁) were raised to 0.99, 0.92, 0.94, 0.97, 0.99, and 0.97, respectively. The tune-up in this study enhanced the detection performance of the model without increasing the number of parameters, computational complexity, or major changes to the original model. This study provides a scientific basis for the subsequent construction of F. thunbergii detection platform.

Key words: Fritillaria thunbergii statistical learning deep learning object detection object detection algorithm YOLO-X dilated convolution

收稿日期: 2022-10-18 出版日期: 2023-12-25

CLC:

TP391.4

基金资助: 浙江省重点研发计划项目(2021C02016)

通讯作者: 王俊 E-mail: DongChengye@zju.edu.cn;jwang@zju.edu.cn

作者简介: 董成烨（https://orcid.org/0000-0003-2716-3339），E-mail：DongChengye@zju.edu.cn

	服务
	把本文推荐给朋友
	加入引用管理器
	E-mail Alert
	RSS
	作者相关文章
	董成烨
	李东方
	冯槐区
	龙思放
	奚特
	周芩安
	王俊

引用本文:

董成烨,李东方,冯槐区,龙思放,奚特,周芩安,王俊. 基于机器视觉和机器学习技术的浙贝母外观品质等级区分[J]. 浙江大学学报（农业与生命科学版）, 2023, 49(6): 881-892.

Chengye DONG,Dongfang LI,Huaiqu FENG,Sifang LONG,Te XI,Qin’an ZHOU,Jun WANG. Classification of Fritillaria thunbergii appearance quality based on machine vision and machine learning technology. Journal of Zhejiang University (Agriculture and Life Sciences), 2023, 49(6): 881-892.

链接本文:

https://www.zjujournals.com/agr/CN/10.3785/j.issn.1008-9209.2022.10.181 或 https://www.zjujournals.com/agr/CN/Y2023/V49/I6/881

图1 数据集部分图像（A1～A6 ）及图像采集设备（B）A1～A6.依次为特级、一级、二级、霉变、虫蛀、破碎浙贝母。

图2 研究方法示意图

表1 YOLO系列的主干特征提取网络特点

图3 YOLOX-DC结构（A）及其主干特征提取网络推断流程（B）示意图

表2 预选算法训练所得模型在浙贝母测试集上的测试结果

表 3 不同空洞率的YOLOX-DC训练所得模型在浙贝母测试集上的测试结果

图4 YOLO-X的预测效果A～F.单目标图像检测结果；G～J.多目标图像检测结果。图片下方红色叉代表模型检测时发生误检或漏检。标签中数字为模型检测浙贝母类别时的置信度，图5同。

图5 YOLOX-DC（空洞率为4）的预测效果A～F.单目标图像检测结果；G～J.多目标图像检测结果。

图6 YOLO-X和YOLOX-DC基于浙贝母单目标图像对应位置的中间激活

图7 YOLO-X和YOLOX-DC基于浙贝母多目标图像对应位置的中间激活

1	黄政晖,章勇杰,倪忠进,等.丘陵山区浙贝母机械化生产模式的效益分析[J].南方农机,2021,52(16):1-3. DOI:10.3969/j.issn.1672-3872.2021.16.001 HUANG Z H, ZHANG Y J, NI Z J, et al. Benefit analysis of mechanized production mode of Fritillaria thunbergii in hilly and mountainous areas[J]. China Southern Agricultural Machinery, 2021, 52(16): 1-3. (in Chinese with English abstract) doi: 10.3969/j.issn.1672-3872.2021.16.001
2	中华中医药学会. 中药材商品规格等级浙贝母:T/ [S].北京:中国标准出版社,2018. China Association of Chinese Medicine. Commercial Grades for Chinese Medicinal Materials—Fritillariae thunbergii Bulbus: T/CACM 1021.24—2018 [S]. Beijing: Standards Press of China, 2018. (in Chinese)
3	王金辉,田忠静,陈启永.贝母筛分机的设计与研究[J].安徽农学通报,2013,19(3):143-144. DOI:10.16377/j.cnki.issn1007-7731.2013.03.029 WANG J H, TIAN Z J, CHEN Q Y. Design of the Fritillaria sieving machine[J]. Anhui Agricultural Science Bulletin, 2013, 19(3): 143-144. (in Chinese with English abstract) doi: 10.16377/j.cnki.issn1007-7731.2013.03.029
4	宋江,邱胜蓝.基于UG的平贝母等级筛分机设计[J].机电工程技术,2013,42(8):31-33. DOI:10.3969/j.issn.1009-9492.2013.08.007 SONG J, QIU S L. Design of Fritillaria ussuriensis grade screen vessel based on UG[J]. Mechanical & Electrical Engineering Technology, 2013, 42(8): 31-33. (in Chinese with English abstract) doi: 10.3969/j.issn.1009-9492.2013.08.007
5	宋江,邱胜蓝.平贝母等级筛分机机架的有限元分析[J].机电工程技术,2014,43(8):68-69. DOI:10.3969/j.issn.1009-9492.2014.08.019 SONG J, QIU S L. The finite element analysis of Fritillaria ussuriensis grade screen vessel frame[J]. Mechanical & Electrical Engineering Technology, 2014, 43(8): 68-69. (in Chinese with English abstract) doi: 10.3969/j.issn.1009-9492.2014.08.019
6	宋江,王明,刘丽华,等.振动式平贝母等级筛分机设计与试验[J].黑龙江八一农垦大学学报,2013,25(1):28-31. DOI:10.3969/j.issn.1002-2090.2013.01.006 SONG J, WANG M, LIU L H, et al. Design and test of vibration type Fritillaria ussuriensis grade screen vessel[J]. Journal of Heilongjiang Bayi Agricultural University, 2013, 25(1): 28-31. (in Chinese with English abstract) doi: 10.3969/j.issn.1002-2090.2013.01.006
7	杨东,王纪华,陆安祥.肉品质量无损检测技术研究进展[J].食品安全质量检测学报,2015,6(10):4083-4090. DOI:10.19812/j.cnki.jfsq11-5956/ts.2015.10.054 YANG D, WANG J H, LU A X. Research development on non-destructive determination technology for meat product quality[J]. Journal of Food Safety & Quality, 2015, 6(10): 4083-4090. (in Chinese with English abstract) doi: 10.19812/j.cnki.jfsq11-5956/ts.2015.10.054
8	NTURAMBIRWE J F I, OPARA U L. Machine learning applications to non-destructive defect detection in horticultural products[J]. Biosystems Engineering, 2020, 189: 60-83. DOI: 10.1016/j.biosystemseng.2019.11.011 doi: 10.1016/j.biosystemseng.2019.11.011
9	KAMILARIS A, PRENAFETA-BOLDÚ F X. Deep learning in agriculture: a survey[J]. Computers and Electronics in Agriculture, 2018, 147: 70-90. DOI: 10.1016/j.compag.2018.02.016 doi: 10.1016/j.compag.2018.02.016
10	REDMON J, DIVVALA S, GIRSHICK R, et al. You only look once: unified, real-time object detection[C]//2016 IEEE Conference on Computer Vision and Pattern Recognition (CVPR). Las Vegas, USA: IEEE, 2016: 779-788. DOI: 10.1109/CVPR.2016.91 doi: 10.1109/CVPR.2016.91
11	REDMON J, FARHADI A. YOLO9000: better, faster, stronger[C]//2017 IEEE Conference on Computer Vision and Pattern Recognition (CVPR). Honolulu, USA: IEEE, 2017: 6517-6525. DOI: 10.1109/CVPR.2017.690 doi: 10.1109/CVPR.2017.690
12	REDMON J, FARHADI A. YOLOv3: an incremental improve-ment[CP/OL]. arXiv, 2018: abs/1804.02767.
13	BOCHKOVSKIY A, WANG C Y, LIAO H Y M. YOLOv4: optimal speed and accuracy of object detection[CP/OL]. arXiv, 2020: abs/2004.10934.
14	GE Z, LIU S T, WANG F, et al. YOLOX: exceeding YOLO series in 2021[CP/OL]. arXiv, 2021: abs/2107.08430.
15	GIRSHICK R, DONAHUE J, DARRELL T, et al. Rich feature hierarchies for accurate object detection and semantic segmentation[C]//2014 IEEE Conference on Computer Vision and Pattern Recognition. Columbus, USA: IEEE, 2014: 580-587. DOI: 10.1109/CVPR.2014.81 doi: 10.1109/CVPR.2014.81
16	GIRSHICK R. Fast R-CNN[C]//2015 IEEE International Conference on Computer Vision (ICCV). Santiago, Chile: IEEE, 2015: 1440-1448. DOI: 10.1109/ICCV.2015.169 doi: 10.1109/ICCV.2015.169
17	REN S Q, HE K M, GIRSHICK R, et al. Faster R-CNN: towards real-time object detection with region proposal networks[J]. IEEE Transactions on Pattern Analysis & Machine Intelligence, 2017, 39(6): 1137-1149. DOI: 10.1109/TPAMI.2016.2577031 doi: 10.1109/TPAMI.2016.2577031
18	TIAN Y N, YANG G D, WANG Z, et al. Apple detection during different growth stages in orchards using the improved YOLO-V3 model[J]. Computers and Electronics in Agriculture, 2019, 157: 417-426. DOI: 10.1016/j.compag.2019.01.012 doi: 10.1016/j.compag.2019.01.012
19	LÜ J D, XU H, HAN Y, et al. A visual identification method for the apple growth forms in the orchard[J]. Computers and Electronics in Agriculture, 2022, 197: 106954. DOI: 10.1016/j.compag.2022.106954 doi: 10.1016/j.compag.2022.106954
20	ZHANG Y C, ZHANG W B, YU J Y, et al. Complete and accurate holly fruits counting using YOLOX object detection[J]. Computers and Electronics in Agriculture, 2022, 198: 107062. DOI: 10.1016/j.compag.2022.107062 doi: 10.1016/j.compag.2022.107062
21	XU W K, ZHAO L G, LI J, et al. Detection and classifi-cation of tea buds based on deep learning[J]. Computers and Electronics in Agriculture, 2022, 192: 106547. DOI: 10.1016/j.compag.2021.106547 doi: 10.1016/j.compag.2021.106547
22	ZHENG T X, JIANG M Z, LI Y F, et al. Research on tomato detection in natural environment based on RC-YOLOv4[J]. Computers and Electronics in Agriculture, 2022, 198: 107029. DOI: 10.1016/j.compag.2022.107029 doi: 10.1016/j.compag.2022.107029
23	QI J T, LIU X N, LIU K, et al. An improved YOLOv5 model based on visual attention mechanism: application to recognition of tomato virus disease[J]. Computers and Electronics in Agriculture, 2022, 194: 106780. DOI: 10.1016/j.compag.2022.106780 doi: 10.1016/j.compag.2022.106780
24	ZHANG D Y, LUO H S, WANG D Y, et al. Assessment of the levels of damage caused by Fusarium head blight in wheat using an improved YoloV5 method[J]. Computers and Electronics in Agriculture, 2022, 198: 107086. DOI: 10.1016/j.compag.2022.107086 doi: 10.1016/j.compag.2022.107086
25	YAO Z S, LIU T, YANG T L, et al. Rapid detection of wheat ears in orthophotos from unmanned aerial vehicles in fields based on YOLOX[J]. Frontiers in Plant Science, 2022, 13: 851245. DOI: 10.3389/fpls.2022.851245 doi: 10.3389/fpls.2022.851245
26	LI X, PAN J D, XIE F P, et al. Fast and accurate green pepper detection in complex backgrounds via an improved Yolov4-tiny model[J]. Computers and Electronics in Agriculture, 2021, 191: 106503. DOI: 10.1016/j.compag.2021.106503 doi: 10.1016/j.compag.2021.106503
27	谢鹏尧,富昊伟,唐政,等.基于RGB图像的冠层尺度水稻叶瘟病斑检测与抗性评估[J].浙江大学学报(农业与生命科学版),2021,47(4):415-428. DOI:10.3785/j.issn.1008-9209.2021.05.131 XIE P Y, FU H W, TANG Z, et al. RGB imaging-based detection of rice leaf blast spot and resistance evaluation at the canopy scale[J]. Journal of Zhejiang University (Agriculture & Life Sciences), 2021, 47(4): 415-428. (in Chinese with English abstract) doi: 10.3785/j.issn.1008-9209.2021.05.131

[1]	谢鹏尧,富昊伟,唐政,麻志宏,岑海燕. 基于RGB图像的冠层尺度水稻叶瘟病斑检测与抗性评估[J]. 浙江大学学报（农业与生命科学版）, 2021, 47(4): 415-428.
[2]	余汛,王哲,景海涛,金秀良,聂臣巍,白怡,王铮. 基于深度学习方法和RGB影像的玉米雄穗分割[J]. 浙江大学学报（农业与生命科学版）, 2021, 47(4): 451-463.

Viewed

Full text

Abstract

Cited

Shared

Discussed