Please wait a minute...
浙江大学学报(工学版)
浙江大学学报(工学版)  2023, Vol. 57 Issue (3): 477-485    DOI: 10.3785/j.issn.1008-973X.2023.03.005
计算机与控制工程     
基于无监督域适应的跨场景带钢表面缺陷识别
刘坤(),杨晓松
河北工业大学 人工智能与数据科学学院，天津 300131
Surface defect identification of cross scene strip based on unsupervised domain adaptation
Kun LIU(),Xiao-song YANG
College of Artificial Intelligence, Hebei University of Technology, Tianjin 300131, China
 全文: PDF(1277 KB)   HTML
摘要：

深度学习模型面对跨场景的带钢表面缺陷识别时存在泛化性能差的问题，为此提出端到端的多级对齐域适应神经网络模型(MADA)，实现源域与目标域数据的像素级光照分布对齐与特征级纹理分布对齐. MADA通过无参考像素级光照分布对齐模块和光照校正损失函数，将源域与目标域数据投影到光照子空间，实现源域与目标域的像素级光照分布对齐. 利用纹理特征提取器和特征级域鉴别器的对抗学习，实现源域和目标域数据的纹理分布对齐. 实验在邯郸钢铁集团带钢表面缺陷数据集的F1指数达到98%，在谢维尔钢铁集团带钢表面缺陷数据集上的F1指数达到86.6%. 实验结果表明，与其他域适应方法相比，所提方法具有更好的泛化性能.
关键词： 带钢表面缺陷识别;  域适应;  跨场景;  泛化;  光照;  纹理    
Abstract:

In view of the poor generalization performance of the deep learning model at surface defect identification of cross scene strip, an end-to-end multi-level aligned domain adaptation neural network (MADA) was proposed, which could achieve pixel-level illumination distribution alignment and feature-level texture distribution alignment, respectively. The source and target domain data were projected into the illumination subspace by MADA to achieve the pixel-level illumination distribution alignment, through the non-reference pixel-level illumination distribution alignment module and the illumination loss function. The adversarial learning of texture feature extractor and feature-level domain discriminator were used to achieve the texture distribution alignment of the source and target domain. The experiment achieved an F1 measure of 98% in Handan strip surface defect dataset and 86.6% in Severstal strip surface defect dataset. Experimental results showed that the proposed method has better generalization performance than other domain adaptation methods.
Key words: strip surface defect identification    domain adaptation    cross scene    generalization    illumination    texture
收稿日期: 2022-03-10 出版日期: 2023-03-31
CLC:  TP 391  
基金资助: 国家自然科学基金资助项目（62173124）；河北省自然科学基金资助项目（F2019202305）
作者简介: 刘坤（1980—），女，副教授，从事图像处理、机器视觉研究. orcid.org/0000-0002-5034-9249. E-mail： liukun@hebut.edu.cn
服务　　
把本文推荐给朋友
加入引用管理器
E-mail Alert
作者相关文章　　
刘坤
杨晓松

引用本文:

刘坤,杨晓松. 基于无监督域适应的跨场景带钢表面缺陷识别[J]. 浙江大学学报(工学版), 2023, 57(3): 477-485.

Kun LIU,Xiao-song YANG. Surface defect identification of cross scene strip based on unsupervised domain adaptation. Journal of ZheJiang University (Engineering Science), 2023, 57(3): 477-485.

链接本文:

https://www.zjujournals.com/eng/CN/10.3785/j.issn.1008-973X.2023.03.005    或    https://www.zjujournals.com/eng/CN/Y2023/V57/I3/477

图 1  多级对齐域适应神经网络模型结构
图 2  不同光照场景的带钢表面缺陷数据集
方法 HSDD_N1?HSDD_N2 HSDD_N1?HSDD_N3 HSDD_N2?HSDD_N1 HSDD_N2?HSDD_N3 HSDD_N3?HSDD_N1 HSDD_N3?HSDD_N2 $ \bar F $
P R F P R F P R F P R F P R F P R F
ResNet50 53 43 47 31 36 35 76 50 49 62 60 68 39 39 39 82 72 69 51
DAN 89 86 86 62 45 40 87 84 85 91 88 88 79 66 68 96 96 96 77
JAN 85 85 85 62 61 61 90 90 90 88 87 87 74 74 74 90 90 90 81
DANN 94 94 94 91 91 91 92 92 92 94 94 94 92 92 92 92 92 92 92
ASAN 92 92 92 94 94 94 91 91 91 94 94 95 90 90 90 92 93 93 92
GVB 95 95 95 96 96 96 95 95 95 96 96 97 91 91 90 93 93 93 94
MADA 98 99 99 99 99 99 98 98 98 98 98 98 97 97 97 98 98 98 98
表 1  不同方法在HSDD上的评价指标对比
方法 SSDD_1?SSDD_2 SSDD_1?SSDD_3 SSDD_2?SSDD_1 SSDD_2?SSDD_3 SSDD_3?SSDD_1 SSDD_3?SSDD_2 $ \bar F $
P R F P R F P R F P R F P R F P R F
ResNet50 78 78 78 60 61 59 80 72 73 79 79 79 67 67 66 82 77 77 72
DAN 84 84 84 67 66 66 78 78 78 85 85 85 69 69 69 83 84 84 77
JAN 84 83 83 65 65 65 84 84 84 84 84 84 70 70 70 84 84 84 78
DANN 82 82 82 72 73 72 88 86 86 88 88 88 74 72 73 85 85 85 81
ASAN 88 87 88 75 73 74 89 89 86 91 91 91 75 75 75 85 85 85 83
GVB 88 88 87 76 75 76 90 90 90 91 91 91 79 77 78 86 86 86 84
MADA 90 92 90 80 80 79 91 91 91 92 92 92 82 82 82 85 85 85 86
表 2  不同方法在SSDD上的评价指标对比
图 3  不同方法目标域的损失曲线和准确率曲线
图 4  不同方法针对目标域提取的特征可视化结果
图 5  不同方法针对目标域的提取特征热力图
${L_{{\rm{exp}}} }$ ${L_{{\rm{spa}}} }$ ${L_{ {\rm{tuA} } } }$ P/% R/% F/%
— — — 90.04 86.07 85.11
√ — — 94.30 92.93 92.35
— √ — 90.83 88.76 88.08
— — √ 91.29 88.27 86.75
√ √ — 97.36 97.03 97.20
√ — √ 96.27 95.50 95.56
— √ √ 90.92 88.10 86.99
√ √ √ 98.36 98.03 98.20
表 3  光照损失函数的消融实验
1 HU L, DUAN F, DING K, et al Research on surface detects on line detection system for steel plate using computer vision[J]. Iron and Steel, 2005, 40 (2): 59- 61
2 WANG J, LI Q, GAN J, et al Surface defect detection via entity sparsity pursuit with intrinsic priors[J]. IEEE Transactions on Industrial Informatics, 2020, 16 (1): 141- 150
doi: 10.1109/TII.2019.2917522
3 SAMSUDIAN S S, AROF H, HARUN S W, et al Steel surface defect classification using multi-resolution empirical mode decomposition and LBP[J]. Measurement Science and Technology, 2020, 32 (1): 015601
4 MENTOURI Z, MOUSSAOUI A, BOUDJEHEM D, et al Steel strip surface defect identification using multiresolution binarized image features[J]. Journal of Failure Analysis and Prevention, 2020, 20: 1917- 1927
doi: 10.1007/s11668-020-01012-7
5 ZAGHDOUDI R, SERIDI H, BOUDIAF A, et al. Binary Gabor pattern (BGP) descriptor and principal component analysis (PCA) for steel surface defects classification [C]// 2020 International Conference on Advanced Aspects of Software Engineering.Constantine: IEEE, 2020: 1–7.
6 KNITTER-PIĄTKOWSKA A, DOBRZYCKI A Application of wavelet transform to damage identification in the steel structure elements[J]. Applied Sciences, 2020, 10 (22): 8198
7 MENTOURI Z, DOGHMANE H, GHERFI K, et al. Tool combination for the description of steel surface image and defect classification [C]// The 2nd International Conference on Embedded Systems and Artificial Intelligence. Fez:[s.n.], 2021: 1-13.
8 SARDA K, ACERNESE A, NOLÈ V, et al A multi-step anomaly detection strategy based on robust distances for the steel industry[J]. IEEE Access, 2021, 9: 53827
doi: 10.1109/ACCESS.2021.3070659
9 TSAI D M, CHEN M C, LI W C, et al A fast regularity measure for surface defect detection[J]. Machine Vision and Applications, 2012, 23: 869- 886
doi: 10.1007/s00138-011-0403-3
10 BOUDANI F Z, NACEREDDINE N, LAICHE N. Content-based image retrieval for surface defects of hot rolled steel strip using wavelet-based LBP [C]// International Workshop on Artificial Intelligence and Pattern Recognition. [S.l.]: Springer, 2021: 404–413.
11 徐科, 宋敏, 杨朝霖, 等 隐马尔可夫树模型在带钢表面缺陷在线检测中的应用[J]. 机械工程学报, 2013, 49 (22): 34- 40
XU Ke, SONG Min, YANG Chao-lin, et al Application of hidden Markov tree model to on-line detection of surface defects for steel strips[J]. Journal of Mechanical Engineering, 2013, 49 (22): 34- 40
doi: 10.3901/JME.2013.22.034
12 TAO X, ZHANG D, MA W, et al Automatic metallic surface defect detection and recognition with convolutional neural networks[J]. Applied Sciences, 2018, 8 (9): 1575
13 HE Y, SONG K, MENG Q, et al An end-to-end steel surface defect detection approach via fusing multiple hierarchical features[J]. IEEE Transactions on Instrumentation and Measurement, 2019, 69 (4): 1493- 1504
14 DONG H, SONG K, HE Y, et al PGA-net: pyramid feature fusion and global context attention network for automated surface defect detection[J]. IEEE Transactions on Industrial Informatics, 2019, 16 (12): 7448- 7458
15 CHEN F C, JAHANSHAHI M R NB-CNN: deep learning-based crack detection using convolutional neural network and naïve bayes data fusion[J]. IEEE Transactions on Industrial Electronics, 2018, 65 (5): 4392- 4400
doi: 10.1109/TIE.2017.2764844
16 彭大芹, 刘恒, 许国良, 等 基于双向特征融合卷积神经网络的液晶面板缺陷检测算法[J]. 广东通信技术, 2019, 39 (4): 66- 73
PENG Da-qin, LIU Heng, XU Guo-liang, et al Defect detection algorithm of liquid crystal panel based on bidirectional feature fusion convolution neural network[J]. Guangdong Communication Technology, 2019, 39 (4): 66- 73
17 ZHANG Y, QIU Z, YAO T, et al. Fully convolutional adaptation networks for semantic segmentation [C]// IEEE/CVF Conference on Computer Vision and Pattern Recognition. Salt Lake City: IEEE, 2018: 6810–6818.
18 LIU D, ZHANG D, SONG Y, et al. Unsupervised instance segmentation in microscopy images via panoptic domain adaptation and task re-weighting [C]// IEEE/CVF Conference on Computer Vision and Pattern Recognition. Seattle: IEEE, 2020: 4242–4251.
19 ZHENG Y, HUANG D, LIU S, et al. Cross-domain object detection through coarse-to-fine feature adaptation [C]// IEEE/CVF Conference on Computer Vision and Pattern Recognition. Seattle: IEEE, 2020: 13763–13772.
20 SINDAGI V A, SRIVASTAVA S Domain adaptation for automatic OLED panel defect detection using adaptive support vector data description[J]. International Journal of Computer Vision, 2017, 122: 193- 211
doi: 10.1007/s11263-016-0953-y
21 GOETZ A, DURMAZ A R, MÜLLER M, et al Addressing materials’ microstructure diversity using transfer learning[J]. NPJ Computational Materials, 2022, 8: 27
doi: 10.1038/s41524-021-00695-2
22 FAN R, WANG H, BOCUS M J, et al. We learn better road pothole detection: from attention aggregation to adversarial domain adaptation [C]// European Conference on Computer Vision. [S.l.]: Springer, 2020: 285–300.
23 GUO C, LI C, GUO J, et al. Zero-reference deep curve estimation for low-light image enhancement [C]// IEEE/CVF Conference on Computer Vision and Pattern Recognition. Seattle: IEEE, 2020: 1777–1786.
24 HE K, ZHANG X, REN S, et al. Deep residual learning for image recognition [C]// IEEE Conference on Computer Vision and Pattern Recognition. Las Vegas: IEEE, 2016: 770–778.
25 LONG M, CAO Y, WANG J, et al. Learning transferable features with deep adaptation networks[C]// Proceedings of the 32th International Conference on Machine Learning. Lille:[s.n.], 2015: 97-105.
26 LONG M, ZHU H, WANG J, et al. Deep transfer learning with joint adaptation networks [C]// Proceedings of the 34th International Conference on Machine Learning. Sydney:[s.n.], 2017: 2208–2217.
27 GANIN Y, LEMPITSKY V. Unsupervised domain adaptation by backpropagation [C]// Proceedings of the 32nd International Conference on Machine Learning. Lille:[s.n.], 2015: 1180–1189.
28 RAAB C, VATH P, MEIER P, et al. Bridging adversarial and statistical domain transfer via spectral adaptation networks [C]// Proceedings of the Asian Conference on Computer Vision.[S.l.]: Spring, 2020: 457–473.
29 CUI S, WANG S, ZHUO J, et al. Gradually vanishing bridge for adversarial domain adaptation [C]// IEEE/CVF Conference on Computer Vision and Pattern Recognition. Seattle: IEEE, 2020: 12452–12461.
30 VAN DER MAATEN L, HINTON G Visualizing data using t-SNE[J]. Journal of Machine Learning Research, 2008, 9 (11): 2579- 2605
[1] 陈雪云,黄金汉,胡子灿,岑升才. 基于线性逻辑矢量模式的遥感图像目标检测[J]. 浙江大学学报(工学版), 2022, 56(1): 47-55.
[2] 宋炯辉,李奇,王静,徐之海,冯华君,陈跃庭. 基于纹理修复的双焦相机连续数字变焦算法[J]. 浙江大学学报(工学版), 2021, 55(8): 1510-1517.
[3] 战友,李强,马啸天,王郴平,邱延峻. 基于宏微观纹理特征融合的路面摩擦性能预测[J]. 浙江大学学报(工学版), 2021, 55(4): 684-694.
[4] 张太恒,梅标,乔磊,杨浩杰,朱伟东. 纹理边界引导的复合材料圆孔检测方法[J]. 浙江大学学报(工学版), 2020, 54(12): 2294-2300.
[5] 梁栋,刘昕宇,潘家兴,孙涵,周文俊,金子俊一. 动态背景下基于自更新像素共现的前景分割[J]. 浙江大学学报(工学版), 2020, 54(12): 2405-2413.
[6] 隋昊,覃高峰,崔祥波,陆新江. 基于误差均值与方差最小化的鲁棒T-S模糊建模方法[J]. 浙江大学学报(工学版), 2019, 53(2): 382-387.
[7] 董立岩, 朱琪, 李永丽. 基于最大共识的模型组合算法[J]. 浙江大学学报(工学版), 2017, 51(2): 416-421.
[8] 张建廷,张立民. 新型自适应稳健双边滤波图像分割[J]. 浙江大学学报(工学版), 2016, 50(9): 1703-1710.
[9] 董巍, 沈会良. 双向纹理函数稀疏采集与重建[J]. 浙江大学学报(工学版), 2016, 50(12): 2364-2370.
[10] 冯志林,周佳男,陈伟杰,尹建伟. 喷墨印花运动纹理的混态MRF检测算法[J]. 浙江大学学报(工学版), 2015, 49(9): 1642-1650.
[11] 马利刚, 张乐平, 邓劲松, 汪雅婕, 王珂. 资源一号“02C”遥感影像土地利用分类[J]. J4, 2013, 47(8): 1508-1516.
[12] 马丽莎, 周文晖, 龚小谨, 刘济林. 基于运动约束的泛化Field D*路径规划[J]. J4, 2012, 46(8): 1546-1552.
[13] 李迪, 陈向坚, 续志军, 白越. 模糊神经网络在机载相机稳像中的应用[J]. J4, 2012, 46(8): 1540-1545.
[14] 戴渊明, 韦巍, 林亦宁. 基于颜色纹理特征的均值漂移目标跟踪算法[J]. J4, 2012, 46(2): 212-217.
[15] 肖冬峰,杨春节,宋执环. 基于改进BP网络的高炉煤气发生量预测模型[J]. J4, 2012, 46(11): 2103-2108.