In view of the poor generalization performance of the deep learning model at surface defect identification of cross scene strip, an end-to-end multi-level aligned domain adaptation neural network (MADA) was proposed, which could achieve pixel-level illumination distribution alignment and feature-level texture distribution alignment, respectively. The source and target domain data were projected into the illumination subspace by MADA to achieve the pixel-level illumination distribution alignment, through the non-reference pixel-level illumination distribution alignment module and the illumination loss function. The adversarial learning of texture feature extractor and feature-level domain discriminator were used to achieve the texture distribution alignment of the source and target domain. The experiment achieved an F1 measure of 98% in Handan strip surface defect dataset and 86.6% in Severstal strip surface defect dataset. Experimental results showed that the proposed method has better generalization performance than other domain adaptation methods.
Kun LIU,Xiao-song YANG. Surface defect identification of cross scene strip based on unsupervised domain adaptation. Journal of ZheJiang University (Engineering Science), 2023, 57(3): 477-485.
Fig.1Structure of multi-level alignment domain adaptation neural network
Fig.2Strip surface defect datasets with different lighting scenarios
方法
HSDD_N1?HSDD_N2
HSDD_N1?HSDD_N3
HSDD_N2?HSDD_N1
HSDD_N2?HSDD_N3
HSDD_N3?HSDD_N1
HSDD_N3?HSDD_N2
$ \bar F $
P
R
F
P
R
F
P
R
F
P
R
F
P
R
F
P
R
F
ResNet50
53
43
47
31
36
35
76
50
49
62
60
68
39
39
39
82
72
69
51
DAN
89
86
86
62
45
40
87
84
85
91
88
88
79
66
68
96
96
96
77
JAN
85
85
85
62
61
61
90
90
90
88
87
87
74
74
74
90
90
90
81
DANN
94
94
94
91
91
91
92
92
92
94
94
94
92
92
92
92
92
92
92
ASAN
92
92
92
94
94
94
91
91
91
94
94
95
90
90
90
92
93
93
92
GVB
95
95
95
96
96
96
95
95
95
96
96
97
91
91
90
93
93
93
94
MADA
98
99
99
99
99
99
98
98
98
98
98
98
97
97
97
98
98
98
98
Tab.1Comparison of evaluation indicators with different methods on HSDD %
方法
SSDD_1?SSDD_2
SSDD_1?SSDD_3
SSDD_2?SSDD_1
SSDD_2?SSDD_3
SSDD_3?SSDD_1
SSDD_3?SSDD_2
$ \bar F $
P
R
F
P
R
F
P
R
F
P
R
F
P
R
F
P
R
F
ResNet50
78
78
78
60
61
59
80
72
73
79
79
79
67
67
66
82
77
77
72
DAN
84
84
84
67
66
66
78
78
78
85
85
85
69
69
69
83
84
84
77
JAN
84
83
83
65
65
65
84
84
84
84
84
84
70
70
70
84
84
84
78
DANN
82
82
82
72
73
72
88
86
86
88
88
88
74
72
73
85
85
85
81
ASAN
88
87
88
75
73
74
89
89
86
91
91
91
75
75
75
85
85
85
83
GVB
88
88
87
76
75
76
90
90
90
91
91
91
79
77
78
86
86
86
84
MADA
90
92
90
80
80
79
91
91
91
92
92
92
82
82
82
85
85
85
86
Tab.2Comparison of evaluation indicators with different methods on SSDD %
Fig.3Loss and accuracy of target domain for different methods
Fig.4Feature visualization results extracted by different methods for target domain
Fig.5Heat map of features extracted by different methods for target domain
${L_{{\rm{exp}}} }$
${L_{{\rm{spa}}} }$
${L_{ {\rm{tuA} } } }$
P/%
R/%
F/%
—
—
—
90.04
86.07
85.11
√
—
—
94.30
92.93
92.35
—
√
—
90.83
88.76
88.08
—
—
√
91.29
88.27
86.75
√
√
—
97.36
97.03
97.20
√
—
√
96.27
95.50
95.56
—
√
√
90.92
88.10
86.99
√
√
√
98.36
98.03
98.20
Tab.3Ablation experiment of light loss function
[1]
HU L, DUAN F, DING K, et al Research on surface detects on line detection system for steel plate using computer vision[J]. Iron and Steel, 2005, 40 (2): 59- 61
[2]
WANG J, LI Q, GAN J, et al Surface defect detection via entity sparsity pursuit with intrinsic priors[J]. IEEE Transactions on Industrial Informatics, 2020, 16 (1): 141- 150
doi: 10.1109/TII.2019.2917522
[3]
SAMSUDIAN S S, AROF H, HARUN S W, et al Steel surface defect classification using multi-resolution empirical mode decomposition and LBP[J]. Measurement Science and Technology, 2020, 32 (1): 015601
[4]
MENTOURI Z, MOUSSAOUI A, BOUDJEHEM D, et al Steel strip surface defect identification using multiresolution binarized image features[J]. Journal of Failure Analysis and Prevention, 2020, 20: 1917- 1927
doi: 10.1007/s11668-020-01012-7
[5]
ZAGHDOUDI R, SERIDI H, BOUDIAF A, et al. Binary Gabor pattern (BGP) descriptor and principal component analysis (PCA) for steel surface defects classification [C]// 2020 International Conference on Advanced Aspects of Software Engineering.Constantine: IEEE, 2020: 1–7.
[6]
KNITTER-PIĄTKOWSKA A, DOBRZYCKI A Application of wavelet transform to damage identification in the steel structure elements[J]. Applied Sciences, 2020, 10 (22): 8198
[7]
MENTOURI Z, DOGHMANE H, GHERFI K, et al. Tool combination for the description of steel surface image and defect classification [C]// The 2nd International Conference on Embedded Systems and Artificial Intelligence. Fez:[s.n.], 2021: 1-13.
[8]
SARDA K, ACERNESE A, NOLÈ V, et al A multi-step anomaly detection strategy based on robust distances for the steel industry[J]. IEEE Access, 2021, 9: 53827
doi: 10.1109/ACCESS.2021.3070659
[9]
TSAI D M, CHEN M C, LI W C, et al A fast regularity measure for surface defect detection[J]. Machine Vision and Applications, 2012, 23: 869- 886
doi: 10.1007/s00138-011-0403-3
[10]
BOUDANI F Z, NACEREDDINE N, LAICHE N. Content-based image retrieval for surface defects of hot rolled steel strip using wavelet-based LBP [C]// International Workshop on Artificial Intelligence and Pattern Recognition. [S.l.]: Springer, 2021: 404–413.
[11]
徐科, 宋敏, 杨朝霖, 等 隐马尔可夫树模型在带钢表面缺陷在线检测中的应用[J]. 机械工程学报, 2013, 49 (22): 34- 40 XU Ke, SONG Min, YANG Chao-lin, et al Application of hidden Markov tree model to on-line detection of surface defects for steel strips[J]. Journal of Mechanical Engineering, 2013, 49 (22): 34- 40
doi: 10.3901/JME.2013.22.034
[12]
TAO X, ZHANG D, MA W, et al Automatic metallic surface defect detection and recognition with convolutional neural networks[J]. Applied Sciences, 2018, 8 (9): 1575
[13]
HE Y, SONG K, MENG Q, et al An end-to-end steel surface defect detection approach via fusing multiple hierarchical features[J]. IEEE Transactions on Instrumentation and Measurement, 2019, 69 (4): 1493- 1504
[14]
DONG H, SONG K, HE Y, et al PGA-net: pyramid feature fusion and global context attention network for automated surface defect detection[J]. IEEE Transactions on Industrial Informatics, 2019, 16 (12): 7448- 7458
[15]
CHEN F C, JAHANSHAHI M R NB-CNN: deep learning-based crack detection using convolutional neural network and naïve bayes data fusion[J]. IEEE Transactions on Industrial Electronics, 2018, 65 (5): 4392- 4400
doi: 10.1109/TIE.2017.2764844
[16]
彭大芹, 刘恒, 许国良, 等 基于双向特征融合卷积神经网络的液晶面板缺陷检测算法[J]. 广东通信技术, 2019, 39 (4): 66- 73 PENG Da-qin, LIU Heng, XU Guo-liang, et al Defect detection algorithm of liquid crystal panel based on bidirectional feature fusion convolution neural network[J]. Guangdong Communication Technology, 2019, 39 (4): 66- 73
[17]
ZHANG Y, QIU Z, YAO T, et al. Fully convolutional adaptation networks for semantic segmentation [C]// IEEE/CVF Conference on Computer Vision and Pattern Recognition. Salt Lake City: IEEE, 2018: 6810–6818.
[18]
LIU D, ZHANG D, SONG Y, et al. Unsupervised instance segmentation in microscopy images via panoptic domain adaptation and task re-weighting [C]// IEEE/CVF Conference on Computer Vision and Pattern Recognition. Seattle: IEEE, 2020: 4242–4251.
[19]
ZHENG Y, HUANG D, LIU S, et al. Cross-domain object detection through coarse-to-fine feature adaptation [C]// IEEE/CVF Conference on Computer Vision and Pattern Recognition. Seattle: IEEE, 2020: 13763–13772.
[20]
SINDAGI V A, SRIVASTAVA S Domain adaptation for automatic OLED panel defect detection using adaptive support vector data description[J]. International Journal of Computer Vision, 2017, 122: 193- 211
doi: 10.1007/s11263-016-0953-y
[21]
GOETZ A, DURMAZ A R, MÜLLER M, et al Addressing materials’ microstructure diversity using transfer learning[J]. NPJ Computational Materials, 2022, 8: 27
doi: 10.1038/s41524-021-00695-2
[22]
FAN R, WANG H, BOCUS M J, et al. We learn better road pothole detection: from attention aggregation to adversarial domain adaptation [C]// European Conference on Computer Vision. [S.l.]: Springer, 2020: 285–300.
[23]
GUO C, LI C, GUO J, et al. Zero-reference deep curve estimation for low-light image enhancement [C]// IEEE/CVF Conference on Computer Vision and Pattern Recognition. Seattle: IEEE, 2020: 1777–1786.
[24]
HE K, ZHANG X, REN S, et al. Deep residual learning for image recognition [C]// IEEE Conference on Computer Vision and Pattern Recognition. Las Vegas: IEEE, 2016: 770–778.
[25]
LONG M, CAO Y, WANG J, et al. Learning transferable features with deep adaptation networks[C]// Proceedings of the 32th International Conference on Machine Learning. Lille:[s.n.], 2015: 97-105.
[26]
LONG M, ZHU H, WANG J, et al. Deep transfer learning with joint adaptation networks [C]// Proceedings of the 34thInternational Conference on Machine Learning. Sydney:[s.n.], 2017: 2208–2217.
[27]
GANIN Y, LEMPITSKY V. Unsupervised domain adaptation by backpropagation [C]// Proceedings of the 32nd International Conference on Machine Learning. Lille:[s.n.], 2015: 1180–1189.
[28]
RAAB C, VATH P, MEIER P, et al. Bridging adversarial and statistical domain transfer via spectral adaptation networks [C]// Proceedings of the Asian Conference on Computer Vision.[S.l.]: Spring, 2020: 457–473.
[29]
CUI S, WANG S, ZHUO J, et al. Gradually vanishing bridge for adversarial domain adaptation [C]// IEEE/CVF Conference on Computer Vision and Pattern Recognition. Seattle: IEEE, 2020: 12452–12461.
[30]
VAN DER MAATEN L, HINTON G Visualizing data using t-SNE[J]. Journal of Machine Learning Research, 2008, 9 (11): 2579- 2605
