Please wait a minute...
浙江大学学报(工学版)  2024, Vol. 58 Issue (12): 2447-2458    DOI: 10.3785/j.issn.1008-973X.2024.12.004
计算机技术     
基于改进YOLOv7的复杂环境下苹果目标检测
莫恒辉(),魏霖静*()
甘肃农业大学 信息科学技术学院,甘肃 兰州 730070
Improved YOLOv7 based apple target detection in complex environment
Henghui MO(),linjing WEI*()
College of Information Science and Technology, Gansu Agricultural University, Lanzhou 730070, China
 全文: PDF(1927 KB)   HTML
摘要:

采摘机器人在不稳定光照、果实多样性和树叶遮挡等复杂自然环境下识别苹果时,检测模型难以捕获关键特征,导致采摘效率和准确性较低. 提出基于YOLOv7模型的针对复杂场景下苹果目标检测的改进算法. 通过限制对比度自适应直方图均衡化算法增强苹果图像对比度,以减少背景干扰,增强目标轮廓清晰度;提出多尺度混合自适应注意力机制,通过特征解构与重构,协同整合空间和通道维度的注意力导向,优化多层次特征的长短距离建模,增强模型对苹果特征的提取能力与抗背景干扰能力;引入全维度动态卷积,通过精细化的注意力机制优化特征选择过程;增加检测头个数,解决小目标检测问题;采用Meta-ACON激活函数,优化特征提取过程中的关注度分配. 结果表明,改进后的YOLOv7模型对苹果的平均检测准确率和召回率分别为85.7%、87.0%,相比于Faster R-CNN、SSD、YOLOv5、YOLOv7,平均检测精度分别提高了15.2、7.5、4.5、2.5个百分比,平均召回率分别提高了13.7、6.5、3.6、1.3个百分比. 模型效果表现优异,为苹果生长监测及机械摘果研究提供了坚实的技术支撑.

关键词: 苹果目标检测YOLOv7注意力机制小目标检测激活函数Grad-CAM    
Abstract:

Robotic harvesters face challenges in identifying apples under complex natural conditions such as unstable lighting, high fruit diversity, and severe leaf occlusion, which impedes the capture of key features, reducing harvesting efficiency and accuracy. An enhanced apple detection algorithm based on the YOLOv7 model for complex scenarios was proposed. A limited contrast adaptive histogram equalization technique was employed to enhance the contrast of apple images, reducing the background interference and clarifying the target contours. A multi-scale hybrid adaptive attention mechanism was introduced. The features were decomposed and reconstructed, and the spatial and channel attention directives were synergistically integrated to optimize multi-layer feature modeling over various distances, thereby boosting the model’s capability to extract apple features and resist background noise. Full-dimensional dynamic convolution was implemented to refine the feature selection process through a meticulous attention mechanism. The number of detection heads was increased to address the challenges of detecting small targets. The Meta-ACON activation function was used to optimize the attention allocation during feature extraction process. Experimental results demonstrated that the improved YOLOv7 model, achieved average accuracy and recall rates of 85.7% and 87.0%, respectively. Compared to Faster R-CNN, SSD, YOLOv5, and the original YOLOv7, the average detection precision was improved by 15.2, 7.5, 4.5, and 2.5 percentage points, and the average recall was improved by 13.7, 6.5, 3.6, and 1.3 percentage points, respectively. The model exhibits exceptional performance, providing robust technical support for apple growth monitoring and mechanical harvesting research.

Key words: apple target detection    YOLOv7    attention mechanism    small target detection    activation function    Grad-CAM
收稿日期: 2023-11-09 出版日期: 2024-11-25
CLC:  TP 391.4  
基金资助: 科技部国家外专资助项目(G2022042005L);甘肃省高等学校产业支撑资助项目(2023CYZC-54);甘肃省重点研发计划资助项目(23YFWA0013);兰州市人才创新创业资助项目(2021-RC-47);2020年甘肃农业大学研究生教育研究资助项目(2020-19);2021年甘肃农业大学校级“三全育人”试点推广教学研究资助项目(2022-9);2022年甘肃农业大学校级专业综合改革项目(2021-4).
通讯作者: 魏霖静     E-mail: 973733507@qq.com;wlj@gsau.edu.cn
作者简介: 莫恒辉(2000—),男,硕士生,从事农业工程与信息技术研究. orcid.org/0009-0000-6815-2738. E-mail:973733507@qq.com
服务  
把本文推荐给朋友
加入引用管理器
E-mail Alert
作者相关文章  
莫恒辉
魏霖静

引用本文:

莫恒辉,魏霖静. 基于改进YOLOv7的复杂环境下苹果目标检测[J]. 浙江大学学报(工学版), 2024, 58(12): 2447-2458.

Henghui MO,linjing WEI. Improved YOLOv7 based apple target detection in complex environment. Journal of ZheJiang University (Engineering Science), 2024, 58(12): 2447-2458.

链接本文:

https://www.zjujournals.com/eng/CN/10.3785/j.issn.1008-973X.2024.12.004        https://www.zjujournals.com/eng/CN/Y2024/V58/I12/2447

图 1  改进YOLOv7结构
图 2  多尺度混合自适应注意力模块结构
图 3  ODConv模块结构
图 4  改进CBS模块结构
图 5  图像增强前后对比
图 6  检测系统界面
影响因素图像类别Ns
幼果顺光317
逆光233
成熟期顺光675
逆光775
树枝遮挡顺光472
逆光491
远距离872
光照不均517
光照低372
表 1  数据集拍摄信息
图 7  不同时期光照与遮挡情况
图 8  图像增强示例
图 9  平均准确度对比图
图 10  平均召回率对比图
模型P/%R/%WS/MBt/s
Faster R-CNN
(voc_resnet.pth)
70.273.3108.00.154
SSD(voc_vgg.pth)78.180.590.60.079
YOLOv5(yolov5l.pt)81.283.489.30.045
YOLOv7(yolov7.pt)83.285.773.80.040
改进YOLOv785.787.080.60.047
表 2  不同模型的指标对比
CLAHEMHAAMODConv四检测头激活函数P/%R/%t/s
83.285.70.040
84.085.90.041
84.686.20.044
84.486.10.042
84.386.00.042
84.185.80.040
85.787.00.047
表 3  改进模型消融实验结果
图 11  不同模型检测结果对比图
模型P/%R/%WS/MBt/s
YOLOv783.285.773.80.040
YOLOv7+SE83.385.775.10.043
YOLOv7+CA83.185.674.90.042
YOLOv7+CBAM83.585.875.20.043
YOLOv7+MHAAM183.986.075.00.042
YOLOv7+MHAAM283.685.874.70.041
YOLOv7+MHAAM84.686.276.30.044
表 4  注意力机制在YOLOv7中的应用消融实验结果
模型P/%R/%WS/MBt/s
YOLOv783.285.773.80.040
ODConv_neck83.685.974.70.042
ODConv_backbone183.485.974.40.041
ODConv_backbone283.385.874.20.041
ODConv_backbone383.385.774.10.040
ODConv_backbone483.285.774.10.040
ODConv_backbone84.486.174.90.042
ODConv_all84.386.275.60.047
表 5  ODConv在YOLOv7中的应用消融实验结果
图 12  不同模型检测热力图分析
1 霍学喜, 刘天军, 刘军弟, 等 2020年度中国苹果产业发展报告: 精简版[J]. 中国果菜, 2022, 42 (2): 1- 6
HUO Xuexi, LIU Tianjun, LIU Jundi, et al 2020 China apple industry development report: simplified version[J]. Chinese Fruits and Vegetables, 2022, 42 (2): 1- 6
2 《中国果业信息》编辑部 2021年全国各省(区, 市)主要水果产量分布及变化[J]. 中国果业信息, 2023, 38 (11): 39
Editorial Board of China Fruit Industry Information Distribution and change of major fruit production in provinces (autonomous regions and municipalities) of China in 2021[J]. China Fruit Industry Information, 2023, 38 (11): 39
3 门小鹏, 许雪峰, 王玉斌, 等 我国苹果生产的现状, 问题与发展对策[J]. 农村实用技术, 2022, (1): 25- 27
MEN Xiaopeng, XU Xuefeng, WANG Yubin, et al Current status, problems and development strategies of apple production in my country[J]. Rural Practical Technology, 2022, (1): 25- 27
4 赵映, 肖宏儒, 梅松, 等 我国果园机械化生产现状与发展策略[J]. 中国农业大学学报, 2017, 22 (6): 116- 127
ZHAO Ying, XIAO Hongru, MEI Song, et al Current situation and development strategy of mechanized production of orchards in China[J]. Journal of China Agricultural University, 2017, 22 (6): 116- 127
5 卢蒙蒙, 江珊, 张国浩, 等 苹果化学疏花疏果技术研究进展[J]. 中国果树, 2021, 210 (4): 4- 7
LU Mengmeng, JIANG Shan, ZHANG Guohao, et al Research progress on chemical flower and fruit thinning technology of apples[J]. Chinese Fruit Tree, 2021, 210 (4): 4- 7
6 李成鹏, 尚书旗, 王东伟, 等 振动式高酸苹果采摘机的设计与试验[J]. 农机化研究, 2024, 46 (4): 106- 113
LI Chengpeng, SHANG Shuqi, WANG Dongwei, et al Design and test of vibrating high-acid apple picking machine[J]. Agricultural Mechanization Research, 2024, 46 (4): 106- 113
7 BU L, HU G, CHEN C, et al Experimental and simulation analysis of optimum picking patterns for robotic apple harvesting[J]. Scientia Horticulturae, 2020, 261: 108937
doi: 10.1016/j.scienta.2019.108937
8 HE K, ZHANG X, REN S, et al. Deep residual learning for image recognition [C]// IEEE Conference on Computer Vision and Pattern Recognition . Las Vegas: IEEE, 2016: 770−778.
9 LI J, LIANG X, SHEN S M, et al Scale-aware fast R-CNN for pedestrian detection[J]. IEEE Transactions on Multimedia, 2017, 20 (4): 985- 996
10 REN S, HE K, GIRSHICK R, et al Faster R-CNN: towards real-time object detection with region proposal networks[J]. IEEE Transactions on Pattern Analysis and Machine Intelligence, 2016, 39 (6): 1137- 1149
11 LIU W, ANGUELOV D, ERHAN D, et al. Ssd: single shot multibox detector [C]/ / Computer Vision-ECCV . Amsterdam: Springer, 2016: 21−37.
12 REDMON J, DIVVALA S, GIRSHICK R, et al. You only look once: unified, real-time object detection [C]// IEEE Conference on Computer Vision and Pattern Recognition . Las Vegas: IEEE, 2016: 779−788.
13 GAO F, FU L, ZHANG X, et al Multi-class fruit-on-plant detection for apple in SNAP system using Faster R-CNN[J]. Computers and Electronics in Agriculture, 2020, 176: 105634
doi: 10.1016/j.compag.2020.105634
14 FU L, FENG Y, MAJEED Y, et al Kiwifruit detection in field images using Faster R-CNN with ZFNet[J]. IFAC-PapersOnLine, 2018, 51 (17): 45- 50
doi: 10.1016/j.ifacol.2018.08.059
15 张恩宇, 成云玲, 胡广锐, 等 基于SSD算法的自然条件下青苹果识别[J]. 中国科技论文, 2020, 15 (3): 274- 281
ZHANG Enyu, CHENG Yunling, HU Guangrui, et al Green apple recognition under natural conditions based on SSD algorithm[J]. Chinese Science and Technology Papers, 2020, 15 (3): 274- 281
doi: 10.3969/j.issn.2095-2783.2020.03.004
16 赵辉, 乔艳军, 王红君, 等 基于改进YOLOv3的果园复杂环境下苹果果实识别[J]. 农业工程学报, 2021, 37 (16): 127- 135
ZHAO Hui, QIAO Yanjun, WANG Hongjun, et al Apple fruit recognition in complex environment of orchard based on improved YOLOv3[J]. Transactions of the Chinese Society of Agricultural Engineering, 2021, 37 (16): 127- 135
17 LYU S, LI R, ZHAO Y, et al Green citrus detection and counting in orchards based on YOLOv5-CS and AI edge system[J]. Sensors, 2022, 22 (2): 576
doi: 10.3390/s22020576
18 WANG C Y, BOCHKOVSKIY A, LIAO H Y M. YOLOv7: trainable bag-of-freebies sets new state-of-the-art for real-time object detectors [C]// IEEE/CVF Conference on Computer Vision and Pattern Recognition . Phuket: IEEE, 2023: 7464−7475.
19 ZHU X, CHENG D, ZHANG Z, et al. An empirical study of spatial attention mechanisms in deep networks [C]// IEEE/CVF International Conference on Computer Vision . Seoul: IEEE, 2019: 6688−6697.
20 HU J, SHEN L, SUN G. Squeeze-and-Excitation Networks [C]// IEEE/CVF Conference on Computer Vision and Pattern Recognition . Salt Lake City: IEEE, 2018: 1−10.
21 CHENG S, ZHU Y, WU S Deep learning based efficient ship detection from drone-captured images for maritime surveillance[J]. Ocean Engineering, 2023, 285: 115440
doi: 10.1016/j.oceaneng.2023.115440
22 高敏, 李玲, 张辉, 等 复杂环境下煤矿井下胶带运输异物在线检测算法优化与分析[J]. 煤炭工程, 2024, 56 (6): 174- 180
GAO Min, LI Ling, ZHANG Hui, et al Optimization and analysis of online detection algorithm for foreign objects in underground coal mine tape transportation under complex environment[J]. Coal Engineering, 2024, 56 (6): 174- 180
23 MA N, ZHANG X, LIU M, et al. Activate or not: learning customized activation [C]// IEEE/CVF Conference on Computer Vision and Pattern Recognition . Nashville: IEEE, 2021: 8032−8042.
24 PISANO E, ZONG S, HEMMINGER B, et al Contrast limited adaptive histogram equalization image processing to improve the detection of simulated spiculations in dense mammograms[J]. Journal of Digital Imaging, 1998, 11 (4): 1- 9
25 KINGA D, ADAM J B. A method for stochastic optimization [C]// International Conference on Learning Representations . San Diego: ICLR, 2015, 5: 6.
[1] 李灿林,王新玥,马利庄,邵志文,张文娇. 融合注意力机制和结构线提取的图像卡通化[J]. 浙江大学学报(工学版), 2024, 58(8): 1728-1737.
[2] 李忠良,陈麒,石琳,杨朝,邹先明. 时间感知组合的动态知识图谱补全[J]. 浙江大学学报(工学版), 2024, 58(8): 1738-1747.
[3] 吴书晗,王丹,陈远方,贾子钰,张越棋,许萌. 融合注意力的滤波器组双视图图卷积运动想象脑电分类[J]. 浙江大学学报(工学版), 2024, 58(7): 1326-1335.
[4] 马现伟,范朝辉,聂为之,李东,朱逸群. 对失效传感器具备鲁棒性的故障诊断方法[J]. 浙江大学学报(工学版), 2024, 58(7): 1488-1497.
[5] 杨军,张琛. 基于边界点估计与稀疏卷积神经网络的三维点云语义分割[J]. 浙江大学学报(工学版), 2024, 58(6): 1121-1132.
[6] 李运堂,李恒杰,张坤,王斌锐,关山越,陈源. 基于新型编码解码网络的复杂输电线识别[J]. 浙江大学学报(工学版), 2024, 58(6): 1133-1141.
[7] 邢志伟,朱书杰,李彪. 基于改进图卷积神经网络的航空行李特征感知[J]. 浙江大学学报(工学版), 2024, 58(5): 941-950.
[8] 刘毅,陈一丹,高琳,洪姣. 基于多尺度特征融合的轻量化道路提取模型[J]. 浙江大学学报(工学版), 2024, 58(5): 951-959.
[9] 魏翠婷,赵唯坚,孙博超,刘芸怡. 基于改进Mask R-CNN与双目视觉的智能配筋检测[J]. 浙江大学学报(工学版), 2024, 58(5): 1009-1019.
[10] 宦海,盛宇,顾晨曦. 基于遥感图像道路提取的全局指导多特征融合网络[J]. 浙江大学学报(工学版), 2024, 58(4): 696-707.
[11] 张会娟,李坤鹏,姬淼鑫,刘振江,刘建娟,张弛. 基于空间相关性增强的无人机检测算法[J]. 浙江大学学报(工学版), 2024, 58(3): 468-479.
[12] 宋明俊,严文,邓益昭,张俊然,涂海燕. 轻量化机器人抓取位姿实时检测算法[J]. 浙江大学学报(工学版), 2024, 58(3): 599-610.
[13] 姚鑫骅,于涛,封森文,马梓健,栾丛丛,沈洪垚. 基于图神经网络的零件机加工特征识别方法[J]. 浙江大学学报(工学版), 2024, 58(2): 349-359.
[14] 宋耀莲,王粲,李大焱,刘欣怡. 基于改进YOLOv5s的无人机小目标检测算法[J]. 浙江大学学报(工学版), 2024, 58(12): 2417-2426.
[15] 熊昌镇,郭传玺,王聪. 基于动态位置编码和注意力增强的目标跟踪算法[J]. 浙江大学学报(工学版), 2024, 58(12): 2427-2437.