Please wait a minute...
浙江大学学报(工学版)
Journal of ZheJiang University (Engineering Science)  2022, Vol. 56 Issue (8): 1640-1647    DOI: 10.3785/j.issn.1008-973X.2022.08.018
    
Automatic hot spot detection method for photovoltaic aerial infrared image
Jie-feng XIA1(),Wu-qin TANG1,Qiang YANG1,2,*()
1. College of Electrical Engineering, Zhejiang University, Hangzhou 310027, China
2. Zhejiang Lab, Hangzhou 310000, China
Download: HTML     PDF(2340KB) HTML
Export: BibTeX | EndNote (RIS)      

Abstract  

A two-stage hot spot detection method of aerial infrared image was proposed to realize component level positioning and fine classification diagnosis of hot spot defects in infrared image, aiming at the problems of high cost, low efficiency and low accuracy of traditional inspection technology of photovoltaic power station. This method combined the traditional image processing technology with the deep learning method to further improve the accuracy and efficiency of defect diagnosis. Specifically, firstly, based on the difference between the gray values of the front and back scenes of aerial infrared images, a component segmentation method based on edge detection was proposed to extract the contour of photovoltaic components to achieve component level positioning. This method achieved the effective detection rate of photovoltaic components up to 99.3% with relatively small hardware requirements. Secondly, considering the differences in the causes, hazards and corresponding treatment methods of hot spots, an infrared defect classification model based on EfficientNet was proposed to finely classify the hot spots, so as to provide more accurate decision support for the operation and maintenance personnel of the power station. The model obtained hot spot classification accuracy of 97.0% when it occupied 20.17 MB. Through experimental comparison and analysis, it is demonstrated that the proposed method has greatly improved the efficiency and accuracy of defect diagnosis.

Key words： aerial image      EfficientNet      deep-learning      hotspot detection      photovoltaic module      edge detection     
Received: 12 August 2021      Published: 30 August 2022
CLC:  TP 29  
Fund:  国家自然科学基金资助项目（51777183）
Corresponding Authors: Qiang YANG     E-mail: jfxia@zju.edu.cn;qyang@zju.edu.cn
Service
E-mail this article
Add to my bookshelf
Add to citation manager
E-mail Alert
RSS
Articles by authors
Jie-feng XIA
Wu-qin TANG
Qiang YANG
Cite this article:

Jie-feng XIA,Wu-qin TANG,Qiang YANG. Automatic hot spot detection method for photovoltaic aerial infrared image. Journal of ZheJiang University (Engineering Science), 2022, 56(8): 1640-1647.

URL:

https://www.zjujournals.com/eng/10.3785/j.issn.1008-973X.2022.08.018     OR     https://www.zjujournals.com/eng/Y2022/V56/I8/1640


光伏航拍红外图像的热斑自动检测方法

针对光伏电站传统巡检技术的高成本、低效率以及准确率不高等问题，提出二阶段式的航拍红外图像热斑检测方法，实现对红外图像中热斑缺陷的组件级定位及精细化分类诊断. 该方法将传统图像处理技术与深度学习方法融合，进一步提升缺陷诊断的准确率与效率. 基于航拍红外图像前、后景灰度值的差异，提出基于边缘检测的组件分割方法来提取光伏组件轮廓以实现组件级定位，该方法以相对较小的硬件需求实现光伏组件有效检出率可达99.3%. 考虑到热斑成因、危害及对应处理方式的差异性，提出基于EfficientNet的红外缺陷分类模型对热斑进行精细的四分类，为电站运维人员提供更为精准的决策支撑，该模型在空间占用20.17 MB的情况下获得97.0%的热斑分类准确率. 经过实验对比分析，论证了本研究所提出的方法在缺陷诊断的效率以及准确率上都较高.


关键词： 航拍图像,  EfficientNet,  深度学习,  热斑检测,  光伏,  边缘检测 
Fig.1 Whole flow of detection algorithm
Fig.2 Schematic diagram of open operation principle
Fig.3 Relationship between gradient direction and edge direction of Canny algorithm
Fig.4 Solution to component adhesion
Fig.5 Principle of component resegmentation
Fig.6 Component hot spot category
网络层 S/像素 C L
Conv3×3 224×224 32 1
MBConv1,k3×3 112×112 16 1
MBConv6,k3×3 112×112 24 2
MBConv6,k3×3 56×56 40 2
MBConv6,k3×3
MBConv6,k3×3
MBConv6,k3×3
MBConv6,k3×3
Conv1*1&Pooling&FC 		28×28
14×14
14×14
7×7
7×7 		80
112
192
320
1280 		3
3
4
1
1
Tab.1 Network structure of EfficientNet-B0
Fig.7 Gray distribution of full image set
Fig.8 PV module detection process
Fig.9 Confusion matrix of component classification result
Fig.10 Comparison of photovoltaic module extraction methods
Fig.11 Comparison of detection rate of full dataset method
算法 Precision Recall Accuracy P/MB Me/MB
VGG16 0.9948 0.9847 0.9898 448.18 232.28
ResNet50 0.9949 0.9949 0.9949 97.49 309.46
DenseNet121 0.9913 0.9951 0.9930 30.44 299.11
SqueezeNet 0.9847 0.9753 0.9804 30.16 290.67
MobileNetV2 0.9608 0.9755 0.9657 12.62 408.23
ShuffleNetV2 0.9755 0.9659 0.9706 8.54 813.55
EfficientNet 0.9949 1.0000 0.9974 20.17 231.66
Tab.2 Comparison of classification model performance
Fig.12 Comparison of operation speed and accuracy of algorithm
[1]   国家能源局. 国家能源局网上新闻发布会文字实录(2012-2019)[EB/OL]. [2021-08-01]. http://www.nea.gov.cn/.
[2]   楼卓. 光伏电站自主巡检中的无人机视觉定位算法研究[D]. 杭州: 浙江大学, 2019.
LOU Zhuo. Research on UAV visual positioning algorithm in autonomous inspection of photovoltaic power station [D]. Hangzhou: Zhejiang University, 2019.
[3]   NIE J, LUO T, LI H Automatic hotspots detection based on UAV infrared images for large-scale PV plant[J]. Electronics Letters, 2020, 56 (19): 993- 995
doi: 10.1049/el.2020.1542
[4]   SHERMAN M, GAMMILL M, RAISSI A, et al. Solar UAV for the inspection and monitoring of photovoltaic (PV) systems in solar power plants[C]// AIAA Scitech 2021 Forum. 2021: 1683.
[5]   MUOZ J, LORENZO E, MARTÍNEZ-MORENO F, et al An investigation into hot-spots in two large grid-connected PV plants[J]. The Prostate, 2008, 16 (8): 693- 701
[6]   蒋琳, 苏建徽, 施永, 等 基于红外热图像处理的光伏阵列热斑检测方法[J]. 太阳能学报, 2020, 41 (8): 180- 184
JIANG Lin, SU Jian-hui, SHI Yong, et al Hot spot detection method of photovoltaic array based on infrared thermal image processing[J]. Journal of Solar Energy, 2020, 41 (8): 180- 184
[7]   SU Y, TAO F, JIN J, et al Automated overheated region object detection of photovoltaic module with thermography image[J]. IEEE Journal of Photovoltaics, 2021, 11 (2): 535- 544
doi: 10.1109/JPHOTOV.2020.3045680
[8]   位硕权. 基于红外图像的光伏组件热斑智能检测[D]. 杭州: 浙江大学, 2020.
WEI Shuo-quan. Intelligent detection of photovoltaic module hot spot based on infrared image [D]. Hangzhou: Zhejiang University, 2020.
[9]   TSANAKAS J A, CHRYSOSTOMOU D, BOTSARIS P N Fault diagnosis of photovoltaic modules through image processing and canny edge detection on field thermographic measurements[J]. International Journal of Sustainable Energy, 2015, 34 (6): 351- 372
[10]   NGO G C, MACABEBE E Q B. Image segmentation using K-means color quantization and density-based spatial clustering of applications with noise (DBSCAN) for hotspot detection in photovoltaic modules [C]// 2016 IEEE Region 10 Conference (TENCON). Singapore: IEEE, 2016: 1614-1618.
[11]   CHEN J, LI Y, LING Q. Hot-spot detection for thermographic images of solar panels[C]// 2020 Chinese Control And Decision Conference (CCDC). Hefei : IEEE, 2020: 4651-4655.
[12]   REN S, HE K, GIRSHICK R, et al Faster R-CNN: towards real-time object detection with region proposal networks[J]. Advances in Neural Information Processing Systems, 2015, 28: 91- 99
[13]   REDMON J, DIVVALA S, GIRSHICK R, et al. You only look once: unified, real-time object detection [C]// Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition. Las Vegas: IEEE, 2016: 779-788.
[14]   LIU W, ANGUELOV D, ERHAN D, et al. SSD: single shot multibox detector [C]// European Conference on Computer Vision. Amsterdam: Springer, 2016: 21-37.
[15]   吴涛, 赖菲 基于 LeNet-5 模型的太阳能电池板缺陷识别分类[J]. 热力发电, 2019, 48 (3): 120- 125
WU Tao, LAI Fei Defect identification and classification of solar panels based on lenet-5 model[J]. Thermal Power Generation, 2019, 48 (3): 120- 125
[16]   郭梦浩, 徐红伟 基于Faster RCNN的红外热图像热斑缺陷检测研究[J]. 计算机系统应用, 2019, 28 (11): 6
GUO Meng-hao, XU Hong-wei Research on infrared thermal image hot spot defect detection based on fast RCNN[J]. Computer System Application, 2019, 28 (11): 6
[17]   GRECO A, PIRONTI C, SAGGESE A, et al. A deep learning based approach for detecting panels in photovoltaic plants [C]// Proceedings of the 3rd International Conference on Applications of Intelligent Systems. Las Palmas de Gran Canaria: Acm Digital Library, 2020: 1-7.
[18]   KRIZHEVSKY A, SUTSKEVER I, HINTON G. ImageNet classification with deep convolutional neural networks [C]// International Conference on Neural Information Processing Systems. Lake Tahoe: Curran Associates Inc, 2012: 1097-1105.
[19]   CANNY J A computational approach to edge detection[J]. IEEE Transactions on Pattern Analysis and Machine Intelligence, 1986, PAMI-8 (6): 679- 698
doi: 10.1109/TPAMI.1986.4767851
[20]   SUZUKI S, BE K Topological structural analysis of digitized binary images by border following[J]. Computer Vision Graphics and Image Processing, 1985, 30 (1): 32- 46
doi: 10.1016/0734-189X(85)90016-7
[21]   ROTHER C, KOLMOGOROV V, BLAKE A GrabCut: interactive foreground extraction using iterated graph cuts[J]. ACM Transactions on Graphics, 2004, 23 (3): 309- 314
doi: 10.1145/1015706.1015720
[22]   IEC Central Office. Photovoltaic (PV) systems: requirements for testing, documentation and maintenance: IEC TS 62446-3[S]. [s. l. ]: IEC, 2017.
[23]   TAN M, LE Q. Efficientnet: rethinking model scaling for convolutional neural networks[C]// International Conference on Machine Learning. Long Beach: PMLR, 2019: 6105-6114.
[24]   Von GIOI R G, JAKUBOWICZ J, MOREL J M, et al LSD: a line segment detector[J]. Image Processing on Line, 2012, 2 (4): 35- 55
[25]   SIMONYAN K, ZISSERMAN A. Very deep convolutional networks for large-scale image recognition[EB/OL]. [2021-08-01]. https://doi.org/10.48550/arXiv.1409.1556.
[26]   HE K, ZHANG X, REN S, et al. Deep residual learning for image recognition[C]// Proceedings of the IEEE conference on computer vision and pattern recognition. Las Vegas: IEEE, 2016: 770-778.
[27]   HUANG G, LIU Z, VAN DER MAATEN L, et al. Densely connected convolutional networks [C]// Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition. Honolulu: IEEE, 2017: 4700-4708.
[28]   IANDOLA F N, HAN S, MOSKEWICZ M W, et al. SqueezeNet: AlexNet-level accuracy with 50x fewer parameters and< 0.5 MB model size[EB/OL]. [2021-08-01]. https://doi.org/10.48550/arXiv.1602.07360.
[29]   HOWARD A G, ZHU M, CHEN B, et al. Mobilenets: efficient convolutional neural networks for mobile vision applications [EB/OL]. [2021-08-01]. https://doi.org/10.48550/arXiv.1704.04861.
[30]   ZHANG X, ZHOU X, LIN M, et al. Shufflenet: an extremely efficient convolutional neural network for mobile devices[C]// Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition. Salt Lake City: IEEE, 2018: 6848-6856.
[1] Tong CHEN,Jian-feng GUO,Xin-zhong HAN,Xue-li XIE,Jian-xiang XI. Visible and infrared image matching method based on generative adversarial model[J]. Journal of ZheJiang University (Engineering Science), 2022, 56(1): 63-74.
[2] Xun CHENG,Jian-bo YU. Monitoring method for machining tool wear based on machine vision[J]. Journal of ZheJiang University (Engineering Science), 2021, 55(5): 896-904.
[3] GONG Xiao-hong, ZHENG Yin-fei, QIN Jia-le, ZHOU Hao.
Multiplicative gradient based edge detection method for medical ultrasound image
[J]. Journal of ZheJiang University (Engineering Science), 2014, 48(10): 1871-1878.
[4] CHEN Xiao-hong,WANG Wei-dong. A HDTV video de-noising algorithm based on spatial-temporal filtering[J]. Journal of ZheJiang University (Engineering Science), 2013, 47(5): 853-859.