Please wait a minute...
浙江大学学报(工学版)
浙江大学学报(工学版)  2022, Vol. 56 Issue (8): 1640-1647    DOI: 10.3785/j.issn.1008-973X.2022.08.018
计算机与控制工程     
光伏航拍红外图像的热斑自动检测方法
夏杰锋1(),唐武勤1,杨强1,2,*()
1. 浙江大学 电气工程学院，浙江 杭州 310027
2. 之江实验室，浙江 杭州 310000
Automatic hot spot detection method for photovoltaic aerial infrared image
Jie-feng XIA1(),Wu-qin TANG1,Qiang YANG1,2,*()
1. College of Electrical Engineering, Zhejiang University, Hangzhou 310027, China
2. Zhejiang Lab, Hangzhou 310000, China
 全文: PDF(2340 KB)   HTML
摘要：

针对光伏电站传统巡检技术的高成本、低效率以及准确率不高等问题，提出二阶段式的航拍红外图像热斑检测方法，实现对红外图像中热斑缺陷的组件级定位及精细化分类诊断. 该方法将传统图像处理技术与深度学习方法融合，进一步提升缺陷诊断的准确率与效率. 基于航拍红外图像前、后景灰度值的差异，提出基于边缘检测的组件分割方法来提取光伏组件轮廓以实现组件级定位，该方法以相对较小的硬件需求实现光伏组件有效检出率可达99.3%. 考虑到热斑成因、危害及对应处理方式的差异性，提出基于EfficientNet的红外缺陷分类模型对热斑进行精细的四分类，为电站运维人员提供更为精准的决策支撑，该模型在空间占用20.17 MB的情况下获得97.0%的热斑分类准确率. 经过实验对比分析，论证了本研究所提出的方法在缺陷诊断的效率以及准确率上都较高.
关键词： 航拍图像;  EfficientNet;  深度学习;  热斑检测;  光伏;  边缘检测    
Abstract:

A two-stage hot spot detection method of aerial infrared image was proposed to realize component level positioning and fine classification diagnosis of hot spot defects in infrared image, aiming at the problems of high cost, low efficiency and low accuracy of traditional inspection technology of photovoltaic power station. This method combined the traditional image processing technology with the deep learning method to further improve the accuracy and efficiency of defect diagnosis. Specifically, firstly, based on the difference between the gray values of the front and back scenes of aerial infrared images, a component segmentation method based on edge detection was proposed to extract the contour of photovoltaic components to achieve component level positioning. This method achieved the effective detection rate of photovoltaic components up to 99.3% with relatively small hardware requirements. Secondly, considering the differences in the causes, hazards and corresponding treatment methods of hot spots, an infrared defect classification model based on EfficientNet was proposed to finely classify the hot spots, so as to provide more accurate decision support for the operation and maintenance personnel of the power station. The model obtained hot spot classification accuracy of 97.0% when it occupied 20.17 MB. Through experimental comparison and analysis, it is demonstrated that the proposed method has greatly improved the efficiency and accuracy of defect diagnosis.
Key words: aerial image    EfficientNet    deep-learning    hotspot detection    photovoltaic module    edge detection
收稿日期: 2021-08-12 出版日期: 2022-08-30
CLC:  TP 29  
基金资助: 国家自然科学基金资助项目（51777183）
通讯作者: 杨强     E-mail: jfxia@zju.edu.cn;qyang@zju.edu.cn
作者简介: 夏杰锋（1997—），男，硕士生，从事机器视觉研究. orcid.org/0000-0002-5273-5995. E-mail: jfxia@zju.edu.cn
服务　　
把本文推荐给朋友
加入引用管理器
E-mail Alert
作者相关文章　　
夏杰锋
唐武勤
杨强

引用本文:

夏杰锋,唐武勤,杨强. 光伏航拍红外图像的热斑自动检测方法[J]. 浙江大学学报(工学版), 2022, 56(8): 1640-1647.

Jie-feng XIA,Wu-qin TANG,Qiang YANG. Automatic hot spot detection method for photovoltaic aerial infrared image. Journal of ZheJiang University (Engineering Science), 2022, 56(8): 1640-1647.

链接本文:

https://www.zjujournals.com/eng/CN/10.3785/j.issn.1008-973X.2022.08.018    或    https://www.zjujournals.com/eng/CN/Y2022/V56/I8/1640

图 1  检测算法整体流程
图 2  开运算原理示意图
图 3  Canny算法梯度方向与边缘方向关系
图 4  组件黏连解决方法
图 5  组件再分割原理
图 6  组件热斑类别
网络层 S/像素 C L
Conv3×3 224×224 32 1
MBConv1,k3×3 112×112 16 1
MBConv6,k3×3 112×112 24 2
MBConv6,k3×3 56×56 40 2
MBConv6,k3×3
MBConv6,k3×3
MBConv6,k3×3
MBConv6,k3×3
Conv1*1&Pooling&FC 		28×28
14×14
14×14
7×7
7×7 		80
112
192
320
1280 		3
3
4
1
1
表 1  EfficientNet-B0的网络结构
图 7  全图像集灰度分布
图 8  光伏组件检测过程
图 9  组件分类结果混淆矩阵
图 10  光伏组件提取方法效果对比
图 11  全数据集方法检出率对比
算法 Precision Recall Accuracy P/MB Me/MB
VGG16 0.9948 0.9847 0.9898 448.18 232.28
ResNet50 0.9949 0.9949 0.9949 97.49 309.46
DenseNet121 0.9913 0.9951 0.9930 30.44 299.11
SqueezeNet 0.9847 0.9753 0.9804 30.16 290.67
MobileNetV2 0.9608 0.9755 0.9657 12.62 408.23
ShuffleNetV2 0.9755 0.9659 0.9706 8.54 813.55
EfficientNet 0.9949 1.0000 0.9974 20.17 231.66
表 2  分类模型性能对比
图 12  算法运算速度与准确性能比较
1 国家能源局. 国家能源局网上新闻发布会文字实录(2012-2019)[EB/OL]. [2021-08-01]. http://www.nea.gov.cn/.
2 楼卓. 光伏电站自主巡检中的无人机视觉定位算法研究[D]. 杭州: 浙江大学, 2019.
LOU Zhuo. Research on UAV visual positioning algorithm in autonomous inspection of photovoltaic power station [D]. Hangzhou: Zhejiang University, 2019.
3 NIE J, LUO T, LI H Automatic hotspots detection based on UAV infrared images for large-scale PV plant[J]. Electronics Letters, 2020, 56 (19): 993- 995
doi: 10.1049/el.2020.1542
4 SHERMAN M, GAMMILL M, RAISSI A, et al. Solar UAV for the inspection and monitoring of photovoltaic (PV) systems in solar power plants[C]// AIAA Scitech 2021 Forum. 2021: 1683.
5 MUOZ J, LORENZO E, MARTÍNEZ-MORENO F, et al An investigation into hot-spots in two large grid-connected PV plants[J]. The Prostate, 2008, 16 (8): 693- 701
6 蒋琳, 苏建徽, 施永, 等 基于红外热图像处理的光伏阵列热斑检测方法[J]. 太阳能学报, 2020, 41 (8): 180- 184
JIANG Lin, SU Jian-hui, SHI Yong, et al Hot spot detection method of photovoltaic array based on infrared thermal image processing[J]. Journal of Solar Energy, 2020, 41 (8): 180- 184
7 SU Y, TAO F, JIN J, et al Automated overheated region object detection of photovoltaic module with thermography image[J]. IEEE Journal of Photovoltaics, 2021, 11 (2): 535- 544
doi: 10.1109/JPHOTOV.2020.3045680
8 位硕权. 基于红外图像的光伏组件热斑智能检测[D]. 杭州: 浙江大学, 2020.
WEI Shuo-quan. Intelligent detection of photovoltaic module hot spot based on infrared image [D]. Hangzhou: Zhejiang University, 2020.
9 TSANAKAS J A, CHRYSOSTOMOU D, BOTSARIS P N Fault diagnosis of photovoltaic modules through image processing and canny edge detection on field thermographic measurements[J]. International Journal of Sustainable Energy, 2015, 34 (6): 351- 372
10 NGO G C, MACABEBE E Q B. Image segmentation using K-means color quantization and density-based spatial clustering of applications with noise (DBSCAN) for hotspot detection in photovoltaic modules [C]// 2016 IEEE Region 10 Conference (TENCON). Singapore: IEEE, 2016: 1614-1618.
11 CHEN J, LI Y, LING Q. Hot-spot detection for thermographic images of solar panels[C]// 2020 Chinese Control And Decision Conference (CCDC). Hefei : IEEE, 2020: 4651-4655.
12 REN S, HE K, GIRSHICK R, et al Faster R-CNN: towards real-time object detection with region proposal networks[J]. Advances in Neural Information Processing Systems, 2015, 28: 91- 99
13 REDMON J, DIVVALA S, GIRSHICK R, et al. You only look once: unified, real-time object detection [C]// Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition. Las Vegas: IEEE, 2016: 779-788.
14 LIU W, ANGUELOV D, ERHAN D, et al. SSD: single shot multibox detector [C]// European Conference on Computer Vision. Amsterdam: Springer, 2016: 21-37.
15 吴涛, 赖菲 基于 LeNet-5 模型的太阳能电池板缺陷识别分类[J]. 热力发电, 2019, 48 (3): 120- 125
WU Tao, LAI Fei Defect identification and classification of solar panels based on lenet-5 model[J]. Thermal Power Generation, 2019, 48 (3): 120- 125
16 郭梦浩, 徐红伟 基于Faster RCNN的红外热图像热斑缺陷检测研究[J]. 计算机系统应用, 2019, 28 (11): 6
GUO Meng-hao, XU Hong-wei Research on infrared thermal image hot spot defect detection based on fast RCNN[J]. Computer System Application, 2019, 28 (11): 6
17 GRECO A, PIRONTI C, SAGGESE A, et al. A deep learning based approach for detecting panels in photovoltaic plants [C]// Proceedings of the 3rd International Conference on Applications of Intelligent Systems. Las Palmas de Gran Canaria: Acm Digital Library, 2020: 1-7.
18 KRIZHEVSKY A, SUTSKEVER I, HINTON G. ImageNet classification with deep convolutional neural networks [C]// International Conference on Neural Information Processing Systems. Lake Tahoe: Curran Associates Inc, 2012: 1097-1105.
19 CANNY J A computational approach to edge detection[J]. IEEE Transactions on Pattern Analysis and Machine Intelligence, 1986, PAMI-8 (6): 679- 698
doi: 10.1109/TPAMI.1986.4767851
20 SUZUKI S, BE K Topological structural analysis of digitized binary images by border following[J]. Computer Vision Graphics and Image Processing, 1985, 30 (1): 32- 46
doi: 10.1016/0734-189X(85)90016-7
21 ROTHER C, KOLMOGOROV V, BLAKE A GrabCut: interactive foreground extraction using iterated graph cuts[J]. ACM Transactions on Graphics, 2004, 23 (3): 309- 314
doi: 10.1145/1015706.1015720
22 IEC Central Office. Photovoltaic (PV) systems: requirements for testing, documentation and maintenance: IEC TS 62446-3[S]. [s. l. ]: IEC, 2017.
23 TAN M, LE Q. Efficientnet: rethinking model scaling for convolutional neural networks[C]// International Conference on Machine Learning. Long Beach: PMLR, 2019: 6105-6114.
24 Von GIOI R G, JAKUBOWICZ J, MOREL J M, et al LSD: a line segment detector[J]. Image Processing on Line, 2012, 2 (4): 35- 55
25 SIMONYAN K, ZISSERMAN A. Very deep convolutional networks for large-scale image recognition[EB/OL]. [2021-08-01]. https://doi.org/10.48550/arXiv.1409.1556.
26 HE K, ZHANG X, REN S, et al. Deep residual learning for image recognition[C]// Proceedings of the IEEE conference on computer vision and pattern recognition. Las Vegas: IEEE, 2016: 770-778.
27 HUANG G, LIU Z, VAN DER MAATEN L, et al. Densely connected convolutional networks [C]// Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition. Honolulu: IEEE, 2017: 4700-4708.
28 IANDOLA F N, HAN S, MOSKEWICZ M W, et al. SqueezeNet: AlexNet-level accuracy with 50x fewer parameters and< 0.5 MB model size[EB/OL]. [2021-08-01]. https://doi.org/10.48550/arXiv.1602.07360.
29 HOWARD A G, ZHU M, CHEN B, et al. Mobilenets: efficient convolutional neural networks for mobile vision applications [EB/OL]. [2021-08-01]. https://doi.org/10.48550/arXiv.1704.04861.
30 ZHANG X, ZHOU X, LIN M, et al. Shufflenet: an extremely efficient convolutional neural network for mobile devices[C]// Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition. Salt Lake City: IEEE, 2018: 6848-6856.
[1] 郝琨,王阔,王贝贝. 基于改进Mobilenet-YOLOv3的轻量级水下生物检测算法[J]. 浙江大学学报(工学版), 2022, 56(8): 1622-1632.
[2] 赵永胜,李瑞祥,牛娜娜,赵志勇. 数字孪生驱动的机身形状控制方法[J]. 浙江大学学报(工学版), 2022, 56(7): 1457-1463.
[3] 何立,庞善民. 结合年龄监督和人脸先验的语音-人脸图像重建[J]. 浙江大学学报(工学版), 2022, 56(5): 1006-1016.
[4] 王中豪,徐正国,章筠. 基于改进型相似性建模的光伏积灰监测方法[J]. 浙江大学学报(工学版), 2022, 56(4): 718-726.
[5] 张雪芹,李天任. 基于Cycle-GAN和改进DPN网络的乳腺癌病理图像分类[J]. 浙江大学学报(工学版), 2022, 56(4): 727-735.
[6] 褚晶辉,史李栋,井佩光,吕卫. 适用于目标检测的上下文感知知识蒸馏网络[J]. 浙江大学学报(工学版), 2022, 56(3): 503-509.
[7] 程若然,赵晓丽,周浩军,叶翰辰. 基于深度学习的中文字体风格转换研究综述[J]. 浙江大学学报(工学版), 2022, 56(3): 510-519, 530.
[8] 陈彤,郭剑锋,韩心中,谢学立,席建祥. 基于生成对抗模型的可见光-红外图像匹配方法[J]. 浙江大学学报(工学版), 2022, 56(1): 63-74.
[9] 任松,朱倩雯,涂歆玥,邓超,王小书. 基于深度学习的公路隧道衬砌病害识别方法[J]. 浙江大学学报(工学版), 2022, 56(1): 92-99.
[10] 刘兴,余建波. 注意力卷积GRU自编码器及其在工业过程监控的应用[J]. 浙江大学学报(工学版), 2021, 55(9): 1643-1651.
[11] 陈雪云,黄小巧,谢丽. 基于多尺度条件生成对抗网络血细胞图像分类检测方法[J]. 浙江大学学报(工学版), 2021, 55(9): 1772-1781.
[12] 刘嘉诚,冀俊忠. 基于宽度学习系统的fMRI数据分类方法[J]. 浙江大学学报(工学版), 2021, 55(7): 1270-1278.
[13] 金立生,华强,郭柏苍,谢宪毅,闫福刚,武波涛. 基于优化DeepSort的前方车辆多目标跟踪[J]. 浙江大学学报(工学版), 2021, 55(6): 1056-1064.
[14] 许佳辉,王敬昌,陈岭,吴勇. 基于图神经网络的地表水水质预测模型[J]. 浙江大学学报(工学版), 2021, 55(4): 601-607.
[15] 王虹力,郭斌,刘思聪,刘佳琪,仵允港,於志文. 边端融合的终端情境自适应深度感知模型[J]. 浙江大学学报(工学版), 2021, 55(4): 626-638.