Solar cell defect segmentation model based on improved SegFormer
Wei LUO1,2(),Zuotao YAN1,Jiahao GUAN1,Jian HAN1,3
1. School of Physics and Electronic Engineering, Northeast Petroleum University, Daqing 163318, China 2. Heilongjiang Province University and Enterprise Joint Construction of Testing and Measurement Technology and Instrument Engineering R&D Center, Daqing 163318, China 3. Sanya Marine Oil and Gas Research Institute of Northeast Petroleum University, Sanya 572024, China
A multi-scale defect segmentation model, EL-SegFormer, was proposed based on an improved SegFormer architecture, aiming at the defects affecting the lifetime and efficiency in solar cell manufacturing. The model was specifically designed to segment defects in solar cells, providing a reliable detection tool for manufacturers. A lightweight modulation module was incorporated in the shallow layers of the network, and multi-head hybrid convolutions were used to capture defect features across various scales. Fixed-scale convolutions and receptive fields were employed to effectively capture early local information in the network. Diverse defects in solar cells can be accurately located by aggregating the extracted features. A hierarchical encoder structure was employed to integrate multi-scale contextual information from shallow to deep layers into the decoder. The decoder utilized a lightweight multi-layer perceptron to consolidate feature information from different levels and generate segmentation masks. The model was loaded and traversed to compute the mean intersection over union (MIoU) using the defect image segmentation masks and label masks. Experimental results indicated that EL-SegFormer, with only 68.2 M parameters, achieved the MIoU of 67.60% on the Buerhop2018 dataset, surpassing recent state-of-the-art models. This outstanding performance indicates the model’s strong potential for addressing complex solar cell defect segmentation tasks, opening up promising avenues for its application in the solar cell manufacturing industry.
Wei LUO,Zuotao YAN,Jiahao GUAN,Jian HAN. Solar cell defect segmentation model based on improved SegFormer. Journal of ZheJiang University (Engineering Science), 2024, 58(12): 2459-2468.
Fig.1Overall framework and intermediate layers of EL-SegFormer
Fig.2EL-SegFormer encoder
Fig.3MHMC module
Fig.4SAA module
Fig.5Feature maps before and after using SAA
Fig.6EL-SegFormer decoder
Fig.7Electroluminescence image defects of solar cells
n
MIoU/%
FLOPs/G
Params/M
T/(张·s?1)
1
66.50
102.3
68.2
621.0
2
67.00
102.5
68.3
547.4
4
67.60
102.60
68.2
519.2
8
67.20
102.6
69.1
413.5
Tab.1Effect of different number of heads in MHMC on network
编码方式
尺寸
FLOPs/G
Params/M
MIoU/%
Mix-FFN
300$ \times $300
36.0
67.9
67.90
512$ \times $512
102.6
67.9
67.60
PE
300$ \times $300
40.56
68.5
66.70
512$ \times $512
106.2
68.5
63.20
Tab.2Performance comparison between Mix-FFN and PE
添加策略
FLOPs/G
Params/M
MIoU/%
S
96.4
85.0
63.40
S+A
96.4
84.6
62.00
S+M
99.8
76.6
65.80
S+A+M
102.6
68.2
67.60
Tab.3Effect of MHMC and SAA on model performance
堆叠策略
FLOPs/G
Params/M
MIoU/%
T/(张·s?1)
MSM+MSM+ MSM+MSA
103.2
69.8
66.90
503.6
MSM+MSM+ MSA+MSA
102.0
66.5
67.00
525.0
MSM+MSA+ MSA+MSA
100.7
65.9
66.30
539.0
MSM+MIX+MSA+ MSA
102.6
68.1
66.80
518.9
MSM+MSM+ MIX+MSA
102.6
68.2
67.60
519.2
Tab.4Impact of stacking strategies on performance and latency
Fig.8Comparison of MIoU and performance of SOTA model in data set of defection in recent years
Fig.9Relationship between MIoU and Epoch for different models
模型
MIoU/%
Fragment
Crack
Corner
Finger
FCN[28]
53.10
51.30
49.50
12.30
U-Net[29]
60.20
55.30
33.06
46.10
DeepLabv3[30]
62.10
62.00
42.34
52.17
PSPNet[31]
61.40
49.13
62.00
39.00
Convnext
63.75
62.31
50.40
55.26
Mask2former
62.33
62.97
31.90
54.08
Swin
61.13
62.66
43.03
54.32
Twins
59.30
61.02
44.27
51.79
Segmenter
69.16
56.25
44.64
37.58
SegFormer
68.42
58.27
45.06
51.00
EL-SegFormer（本研究）
69.00
61.26
56.19
54.38
Tab.5Comparison of segmentation MIoU under different models and defects
大小类别
模型
尺寸
FLOPs/G
Params/M
MIoU/%
mPA/%
小
Segmenter-T
512×512
12.3
6.7
49.00
55.87
Swin-T
512×512
242.7
59.0
53.20
65.59
Convnext-T
512×512
204.6
59.3
54.30
69.28
SegFormer-b1
512×512
16.0
13.7
57.00
66.46
Twins-S
512×512
232.4
53.1
58.60
66.26
Mask2former-S
512×512
246.0
44.0
59.40
69.37
EL-SegFormer-S(本研究)
512×512
14.0
12.0
60.49
70.45
基本
Segmenter-S
512×512
38.5
26.0
56.00
68.63
Swin-B
512×512
305.0
122.9
60.40
75.16
SegFormer-b3
512×512
77.5
47.2
61.00
71.54
Twins-B
512×512
256.2
86.7
61.40
72.27
Mask2former-B
512×512
293.0
63.0
61.60
74.03
Convnext-S
512×512
262.1
80.9
63.70
74.35
EL-SegFormer-B(本研究)
512×512
75.6
39.2
64.30
75.36
大
Segmenter-B
512×512
129.0
104.4
61.00
68.68
Twins-L
512×512
303.0
134.0
63.00
75.91
SegFormer-b5
512×512
111.5
85.0
63.40
78.43
Swin-L
512×512
416.8
237.6
63.90
66.43
Mask2former-L
512×512
542.0
153.0
64.00
74.29
Convnext-B
512×512
299.0
123.9
66.00
76.38
EL-SegFormer-L(本研究)
512×512
102.6
68.2
67.60
79.85
Tab.6Comparison of indexes of each model
Fig.10Comparison of segmentation results of each model
[1]
吕玉荣 太阳能电池的发展背景及应用[J]. 化工时刊, 2021, 35 (2): 26- 29 LV Yurong The development background and applications of solar cells[J]. Chemical Industry Times, 2021, 35 (2): 26- 29
[2]
BREITENSTEIN O, BAUER J, ALTERMATT P P, et al Influence of defects on solar cell characteristics[J]. Solid State Phenomena, 2010, 156: 1- 10
[3]
施光辉, 崔亚楠, 刘小娇, 等 电致发光 (EL) 在光伏电池组件缺陷检测中的应用[J]. 云南师范大学学报: 自然科学版, 2016, 36 (2): 17- 21 SHI Guanghui, CUI Yanan, LIU Xiaojiao, et al Electroluminescent application in defects detection of photovoltaic-module[J]. Journal of Yunnan Normal University: Natural Sciences Edition, 2016, 36 (2): 17- 21
[4]
MANSOURI A, ZETTL M, MAYER O, et al. Defect detection in photovoltaic modules using electroluminescence imaging [C]// 27th European Photovoltaic Solar Energy Conference and Exhibition . Frankfurt: PVTECH, 2012, 64617926: 3374-3378.
[5]
KANAI A, SUGIYAMA M Emission properties of intrinsic and extrinsic defects in Cu2SnS3 thin films and solar cells[J]. Japanese Journal of Applied Physics, 2020, 60 (1): 015504
[6]
徐辉, 祝玉华, 甄彤, 等 深度神经网络图像语义分割方法综述[J]. 计算机科学与探索, 2021, 15 (1): 47- 59 XU Hui, ZHU Yuhua, ZHEN Tong, et al Survey of image semantic segmentation methods based on deep neural network[J]. Journal of Frontiers of Computer Science and Technology, 2021, 15 (1): 47- 59
[7]
陈海永, 刘新如 交叉门控融合的改进语义分割网络及应用[J]. 重庆理工大学学报 : 自然科学, 2023, 37 (6): 187- 195 CHEN Haiyong, LIU Xinru An improved semantic segmentation network and its application by using cross-gated fusion[J]. Journal of Chongqing University of Technology: Natural Science, 2023, 37 (6): 187- 195
[8]
RAHMAN M R U, CHEN H, XI W. U-Net based defects inspection in photovoltaic electroluminecscence images [C]// 2019 IEEE International Conference on Big Knowledge . Changsha: IEEE, 2019: 215–220.
[9]
王盛, 吴浩, 彭宁, 等 改进U2-Net的太阳能电池片缺陷分割方法[J]. 国外电子测量技术, 2023, 42 (2): 177- 184 WANG Sheng, WU Hao, PENG Ning, et al Improved U2-Net defect segmentation method for solar cells[J]. Foreign Electronic Measurement Technology, 2023, 42 (2): 177- 184
[10]
BALZATEGUI J, ECIOLAZA L, ARANA-AREXOLALEIBA N. Defect detection on polycrystalline solar cells using electroluminescence and fully convolutional neural networks [C]// IEEE/SICE International Symposium on System Integration . Kunming: IEEE, 2020: 949–953.
[11]
张海波, 蔡磊, 任俊平, 等 基于Transformer的高效自适应语义分割网络[J]. 浙江大学学报: 工学版, 2023, 57 (6): 1205- 1214 ZHANG Haibo, CAI Lei, REN Junping, et al Efficient and adaptive semantic segmentation network based on Transformer[J]. Journal of Zhejiang University: Engineering Science, 2023, 57 (6): 1205- 1214
[12]
DOSOVITSKIY A, BEYER L, KOLESNIKOV A, et al. An image is worth 16x16 words: Transformers for image recognition at scale [EB/OL]. (2020-10-22)[2023-11-07]. https://arxiv.org/search/?query=An+image+is+worth+16x16+words%3A+Transformers+for+image+recognition+at+scale&searchtype=all&source=header.
[13]
YAMADA M, D'AMARIO V, TAKEMOTO K, et al. Transformer module networks for systematic generalization in visual question answering [EB/OL]. (2022-01-27)[2023-11-07]. https://arxiv.org/abs/2201.11316.
[14]
ZHOU B, ZHAO H, PUIG X, et al. Scene parsing through ade20k dataset [C]// Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition . Hawaii: IEEE, 2017: 633–641.
[15]
DENG J, DONG W, SOCHER R, et al. Imagenet: a large-scale hierarchical image database [C]// 2009 IEEE Conference on Computer Vision and Pattern Recognition . Tokyo: IEEE, 2009: 248–255.
[16]
LIU Z, LIN Y, CAO Y, et al. Swin transformer: hierarchical vision transformer using shifted windows [C]// Proceedings of the IEEE/CVF International Conference on Computer Vision . Sanya: IEEE, 2021: 10012–10022.
[17]
LIU J, WANG C, ZHA L A middle-level learning feature interaction method with deep learning for multi-feature music genre classification[J]. Electronics, 2021, 10 (18): 2206
doi: 10.3390/electronics10182206
[18]
XIE E, WANG W, YU Z, et al SegFormer: simple and efficient design for semantic segmentation with transformers[J]. Advances in Neural Information Processing Systems, 2021, 34: 12077- 12090
[19]
LIU X, YU H F, DHILLON I, et al. Learning to encode position for transformer with continuous dynamical model [C]// International Conference on Machine Learning . PMLR: [s.n.], 2020: 6327–6335.
[20]
BRAUWERS G, FRASINCAR F A general survey on attention mechanisms in deep learning[J]. IEEE Transactions on Knowledge and Data Engineering, 2021, 35 (4): 3279- 3298
[21]
LIN W, WU Z, CHEN J, et al. Scale-aware modulation meet transformer [C]// Proceedings of the IEEE/CVF International Conference on Computer Vision . Vancouver: IEEE, 2023: 6015–6026.
[22]
STRUDEL R, GARCIA R, LAPTEV I, et al. Segmenter: Transformer for semantic segmentation [C]// Proceedings of the IEEE/CVF International Conference on Computer Vision . Kuala Lumpur: IEEE, 2021: 7262–7272.
[23]
DEITSCH S, CHRISLTEIN V, BERGER S, at el Automatic classification of defective photovoltaic module cells in electroluminescence images[J]. Solar Energy, 2019, 185: 455- 468
doi: 10.1016/j.solener.2019.02.067
[24]
CHU X, TIAN Z, WANG Y, et al Twins: revisiting the design of spatial attention in vision transformers[J]. Advances in Neural Information Processing Systems, 2021, 34: 9355- 9366
[25]
STRUDEL R, GARCIA R, LAPTEV I, et al. Segmenter: Transformer for semantic segmentation [C]// Proceedings of the IEEE/CVF International Conference on Computer Vision . Venice: IEEE, 2021: 7262–7272.
[26]
LIU Z, MAO H, WU C Y, et al. A convnet for the 2020s [C]// Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition . Paris: IEEE, 2022: 11976–11986.
[27]
CHENG B, MISRA I, SCHWING A G, et al. Masked-attention mask transformer for universal image segmentation [C]// Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition . Paris: IEEE, 2022: 1290–1299.
[28]
LONG J, SHELHAMER E, DARRELL T. Fully convolutional networks for semantic segmentation [C]// Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition . Boston: IEEE, 2015: 3431–3440.
[29]
RONNEBERGER O, FISCHER P, BROX T. U-net: convolutional networks for biomedical image segmentation [C]// Medical Image Computing and Computer-Assisted Intervention–MICCAI 2015: 18th International Conference . Munich: Springer International Publishing, 2015: 234–241.
[30]
CHEN L C, PAPANDREOU G, SCHROFF F, et al. Rethinking atrous convolution for semantic image segmentation [EB/OL]. (2017-06-17)[2023-11-07]. https://arxiv.org/abs/1706.05587.
