Please wait a minute...
浙江大学学报(工学版)
Journal of ZheJiang University (Engineering Science)  2020, Vol. 54 Issue (6): 1228-1239    DOI: 10.3785/j.issn.1008-973X.2020.06.021
Mechanical Engineering     
Wafer map defect recognition based on transfer learning and deep forest
Zong-li SHEN(),Jian-bo YU*()
School of Mechanical Engineering, Tongji University, Shanghai 201804, China
Download: HTML     PDF(1461KB) HTML
Export: BibTeX | EndNote (RIS)      

Abstract  

A wafer map pattern recognition (WMPR) model was proposed based on transfer learning and deep forest, in order to identify the defect pattern of the wafer maps and to timely diagnose the source of the fault in the manufacturing process. Transfer learning was used to migrate the network weight parameters of the deep CNN DenseNet pre-trained on ImageNet to this model, and the classification layer of the model was redesigned, in order to solve the problems of difficulties of deep learning model training and imbalance in the number of defect types in wafer maps. Thus, the training time of the model was reduced and the feature extraction ability was improved. Deep forest model was introduced to identify the wafer defect pattern, based on the abstract features of the wafer maps extracted by DenseNet. The experimental results on an industrial case demonstrated that the average recognition rate was about 96.8%. This method can improve the recognition efficiency and its performance is better than those well-known CNNs and other typical classifiers.

Key words： semiconductor manufacturing      wafer defect      transfer learning      CNN      deep forest     
Received: 30 April 2019      Published: 06 July 2020
CLC:  TP 391.41  
Corresponding Authors: Jian-bo YU     E-mail: 979949752@qq.com;jbyu@tongji.edu.cn
Service
E-mail this article
Add to my bookshelf
Add to citation manager
E-mail Alert
RSS
Articles by authors
Zong-li SHEN
Jian-bo YU
Cite this article:

Zong-li SHEN,Jian-bo YU. Wafer map defect recognition based on transfer learning and deep forest. Journal of ZheJiang University (Engineering Science), 2020, 54(6): 1228-1239.

URL:

http://www.zjujournals.com/eng/10.3785/j.issn.1008-973X.2020.06.021     OR     http://www.zjujournals.com/eng/Y2020/V54/I6/1228


基于迁移学习与深度森林的晶圆图缺陷识别

为了有效识别晶圆图缺陷模式并及时诊断制造过程的故障源，提出基于迁移学习和深度森林集成的DenseNet-GCForest晶圆图缺陷模式识别模型. 为了解决深度学习模型训练困难和晶圆图缺陷类型数目不平衡的问题，利用迁移学习将深度卷积神经网络DenseNet在ImageNet上预训练的网络权重参数迁移至本模型并重新设计分类层，以减少深度网络模型的训练时间并提高模型的特征提取能力；基于DenseNet网络提取的高维抽象晶圆图特征，引入深度森林模型进行晶圆图特征缺陷模式识别. 工业案例的实验验证结果表明，该方法的识别准确率达到了96.8%，并提高了识别效率，其性能优于典型的卷积神经网络以及其他常用识别方法.


关键词： 半导体制造,  晶圆缺陷,  迁移学习,  卷积神经网络,  深度森林 
Fig.1 Dense block connection module framework
Fig.2 DenseNet network structure based on transfer learning
Fig.3 Class vector generation in GCForest
Fig.4 Deep forest network structure（taking 400-dimensional input as example）
Fig.5 Defect identification process of DenseNet-GCForest wafer map
Fig.6 Normal wafer maps and 8 defect modes
Fig.7 Number of different types of wafer maps
Fig.8 Training process of DenseNet169 based on transfer learning
Fig.9 Output features of wafer maps in multilevel networks
Fig.10 Visualization analysis of raw data and DenseNet features
%
预测真实 None Center Donut Edge-local Edge-ring Local Near-full Random Scratch
None 99.59 0.00 0 0.28 0 0 0 0 0.14
Center 0.83 97.52 0 0.83 0 0 0 0 0.83
Donut 0 0 94.74 0 0 5.26 0 0 0
Edge-local 0.86 0.29 0 96.83 0.29 1.44 0 0.29 0
Edge-ring 0 0 0 4.91 95.09 0 0 0 0
Local 0.75 0.38 0 1.51 0 96.23 0 0.75 0.38
Near-full 0 0 0 0 0 0 94.12 5.88 0
Random 0 2.56 0 0 2.56 5.13 2.56 87.18 0
Scratch 1.25 0 0 1.25 0 3.75 0 0 93.75
Tab.1 Confusion matrix of Densenet-GCForest wafer map defect recognition rate
Fig.11 Error identification of Random class
参数名称 参数大小 Racc/%
滑动窗口大小 400 97.31
700 96.7
1 000 96.2
1 300 96.4
1 664 96.2
决策树生成的
最小样本数 		0 96.2
0.1 96.7
0.2 96.2
0.3 95.8
0.4 95.4
决策树生成的
允许误差 		0 96.2
0.1 96.7
0.2 96.2
0.3 95.8
0.4 95.4
扫描层随机森林的
决策树数量 		200 96.9
400 95.8
600 97.0
800 96.7
1 000 96.0
Tab.2 Parameter sensitivity analysis of GCForest
%
实验方法 缺陷类别 Racc Rrec F
DenseNet-GCForest Center 97.5 97.5 97.5
Donut 100 94.7 97.3
Edge_local 95.5 96.8 96.1
Edge_ring 98.7 95.1 96.9
Local 95.9 96.2 96.0
Near_full 94.1 94.1 94.1
None 99.0 99.6 99.3
Random 89.5 87.2 88.3
Scratch 96.2 93.8 94.9
GoogleNet Center 72.2 34.5 46.7
Donut 59.1 43.3 50.0
Edge_local 63.9 79.8 71.0
Edge_ring 93.4 87.6 90.4
Local 57.3 48.8 52.7
Near_full 93.3 93.3 93.3
None 90.6 92.9 91.7
Random 85.4 92.1 88.6
Scratch 81.7 81.7 81.7
ResNet Center 85.0 77.1 80.9
Donut 47.2 81.0 59.6
Edge_local 87.6 81.4 84.4
Edge_ring 94.3 86.6 90.3
Local 81.7 68.6 74.6
Near_full 75.0 75.0 75.0
None 89.9 99.3 94.4
Random 88.2 73.2 80.0
Scratch 59.5 71.0 64.4
DenseNet Center 76.4 85.5 80.7
Donut 95.2 64.5 76.9
Edge_local 79.7 91.2 85.0
Edge_ring 96.1 81.5 88.2
Local 82.1 66.8 73.6
Near_full 91.7 100 95.7
None 95.2 99.3 97.2
Random 86.4 65.5 74.5
Scratch 81.5 72.6 76.8
Decision TreeEnsemble Center 95.6 93.75 94.7
Donut 92.6 92.3 92.4
Edge_local 83.5 87.3 85.4
Edge_ring 86.8 91.1 88.9
Local 83.5 82.3 82.9
Near_full 89.4 91.7 90.5
None 100 99.5 99.7
Random 91.7 87.3 88.4
Scratch 86.0 88.5 87.2
SDAE Center 98.5 87.3 92.6
Donut 87.6 89.1 88.4
Edge_local 87.5 85.4 86.4
Edge_ring 98.8 94.7 96.7
Local 83.5 55.8 66.9
Near_full 85.3 96.2 90.4
None 99.0 98.7 98.9
Random 93.3 89.3 91.2
Scratch 81.7 39.6 53.3
Tab.3 Performance comparison of DenseNet-GCForest and other algorithms
分类器 Racc/% 分类器 Racc/%
DenseNet-GCForest 96.8 GCForest 73.7
C4.5 Ensemble 90.8 RF 68.9
SDAE 89.4 SVML 72.5
DenseNet 88.6 SVMG 40.2
GoogleNet 74.3 KNN 30.1
ResNet 86.5 C4.5 62.4
Tab.4 Comparison of five-fold cross validation of various algorithms
分类器 Racc/% 分类器 Racc/%
GCForest 96.8 SVMG 95.5
BPN 85.1 KNN 92.6
RF 95.1 C4.5 87.2
SVML 95.6 ? ?
Tab.5 Comparison of five-fold cross recognition rate of various recognizers based on DenseNet features
[1]   TAO Y, WAY K, SUK J B Detection of spatial defect patterns generated in semiconductor fabrication processes[J]. IEEE Transactions on Semiconductor Manufacturing, 2011, 24 (3): 392- 403
doi: 10.1109/TSM.2011.2154870
[2]   KIM B, JEONG Y S, TONG S H, et al Step-down spatial randomness test for detecting abnormalities in DRAM wafers with multiple spatial maps[J]. IEEE Transactions on Semiconductor Manufacturing, 2016, 29 (1): 57- 65
doi: 10.1109/TSM.2015.2486383
[3]   YU J, LU X Wafer map defect detection and recognition using joint local and nonlocal linear discriminant analysis[J]. IEEE Transactions on Semiconductor Manufacturing, 2016, 29 (1): 33- 42
doi: 10.1109/TSM.2015.2497264
[4]   SHANKAR N G, ZHONG Z W Defect detection on semiconductor wafer surfaces[J]. Microelectronic Engineering, 2005, 77 (3): 337- 346
[5]   LEE S H, KOO H I, CHO N I New automatic defect classification algorithm based on a classification-after-segmentation framework[J]. Journal of Electronic Imaging, 2010, 19 (2): 334- 343
[6]   HWANG J Y, KUO W Model-based clustering for integrated circuit yield enhancement[J]. European Journal of Operational Research, 2007, 178 (1): 143- 153
doi: 10.1016/j.ejor.2005.11.032
[7]   BALY R, HAJJ H Wafer classification using support vector machines[J]. IEEE Transactions on Semiconductor Manufacturing, 2012, 25 (3): 373- 383
doi: 10.1109/TSM.2012.2196058
[8]   XIE L, HUANG R, GU N, et al A novel defect detection and identification method in optical inspection[J]. Neural Computing and Applications, 2014, 24 (7/8): 1953- 1962
doi: 10.1007/s00521-013-1442-7
[9]   CHAO L C, TONG L I Wafer defect pattern recognition by multi-class support vector machines by using a novel defect cluster index[J]. Expert Systems with Applications, 2009, 36 (6): 10158- 10167
doi: 10.1016/j.eswa.2009.01.003
[10]   ADLY F, ALUSSEIN O, YOO P, et al Simplified subspaced regression network for identification of defect patterns in semiconductor wafer maps[J]. IEEE Transactions on Semiconductor Manufacturing, 2015, 11 (6): 1267- 1276
[11]   KIM B, JEONG Y S, TONG S H, et al A regularized singular value decomposition-based approach for failure pattern classification on fail bit map in a DRAM wafer[J]. IEEE Transactions on Semiconductor Manufacturing, 2015, 28 (1): 41- 49
doi: 10.1109/TSM.2014.2388192
[12]   OOI M P L, SOK H K, KUANG Y C, et al Defect cluster recognition system for fabricated semiconductor wafers[J]. Engineering Applications of Artificial Intelligence, 2013, 26 (3): 1029- 1043
doi: 10.1016/j.engappai.2012.03.016
[13]   余建波, 卢笑蕾, 宗卫周 基于局部与非局部线性判别分析和高斯混合模型动态集成的晶圆表面缺陷探测与识别[J]. 自动化学报, 2016, 42 (1): 47- 59
YU Jian-bo, LU Xiao-lei, ZONG Wei-zhou Wafer defect detection and recognition based on local and nonlocal linear discriminant analysis and dynamic ensemble of gaussian mixture models[J]. ACTA Automatica Science, 2016, 42 (1): 47- 59
[14]   PIAO M, JIN C H, LEE J Y, et al Decision tree ensemble-based wafer map failure pattern recognition based on radon transform-based features[J]. IEEE Transactions on Semiconductor Manufacturing, 2018, 31 (2): 250- 257
doi: 10.1109/TSM.2018.2806931
[15]   SAQLAIN M, JARGALSAIKHAN B, LEE J Y A voting ensemble classifier for wafer map defect patterns identification in semiconductor Manufacturing[J]. IEEE Transactions on Semiconductor Manufacturing, 2019, 32 (2): 171- 182
doi: 10.1109/TSM.2019.2904306
[16]   LECUN Y, BENGIO Y, HINTON G Deep learning[J]. Nature, 2015, 521: 436- 444
doi: 10.1038/nature14539
[17]   杨婧, 耿辰, 王海林, 纪建松, 等 基于DenseNet的低分辨CT影像肺腺癌组织学亚型分类[J]. 浙江大学学报: 工学版, 2019, 53 (6): 1164- 1170
YANG Jing, GENG Chen, WANG Hai-lin, et al Classification on histological subtypes of lung adenocarcinoma from low-resolution CT images based on DenseNet[J]. Journal of Zhejiang Unversity: Engineering Science, 2019, 53 (6): 1164- 1170
[18]   THIRUKOVALLURU R, DIXIT S, SEVAKULA RK et al. Generating feature sets for fault diagnosis using denoising stacked auto-encoder [C] // IEEE International Conference in Prognostics and Health Management (ICPHM). Ottawa: IEEE, 2016: 1-7.
[19]   JIANG G, HE H, XIE P, et al Stacked Multilevel-Denoising Autoencoders: a new representation learning approach for wind turbine gearbox fault diagnosis[J]. IEEE Transactions on Instrumentation and Measurement, 2017, 66: 2391- 2402
doi: 10.1109/TIM.2017.2698738
[20]   KRIZHEVSKY A, SUTSKEVER I, HINTON GE. Imagenet classification with deep convolutional neural networks [C] // Advances in Neural Information Processing Systems (NIPS). Lake Tahoe: [s. n.], 2012: 1097–1105.
[21]   袁公萍, 汤一平, 韩旺明, 等 基于深度卷积神经网络的车型识别方法[J]. 浙江大学学报: 工学版, 2018, 52 (4): 694- 702
YUAN Gong-ping, TANG Yi-ping, et al Vehicle category recognition based on deep convolutional neural network[J]. Journal of Zhejiang University: Engineering Science, 2018, 52 (4): 694- 702
[22]   SZEGEDY C, LIU W, JIA Y, et al. Going deeper with convolutions [C] // 2015 IEEE Conference on Computer Vision and Pattern Recognition (CVPR). Boston: IEEE, 2015: 1-9.
[23]   HE K M, ZHANG X Y, REN S Q, et al. Deep residual learning for image recognition [C] // 2016 IEEE Conference on Computer Vision and Pattern Recognition (CVPR). Las Vegas: IEEE, 2016: 770-778..
[24]   HUANG G, LIU Z, MAATEN L V D et al. Densely connected convolutional networks [C] // 2017 IEEE Conference on Computer Vision and Pattern Recognition (CVPR). Honolulu IEEE, 2017: 2261-2260.
[25]   DANIEL W, BERND S R, MOSHE S Design of deep convolutional neural network architectures for automated feature extraction in industrial inspection[J]. CIRP Annual Manufacturing Technology, 2016, 65 (1): 417- 420
doi: 10.1016/j.cirp.2016.04.072
[26]   NAKAZAWA T, KULKARNI D V Wafer map defect pattern classification and image retrieval using convolutional neural network[J]. IEEE Transactions on Semiconductor Manufacturing, 2018, 31 (2): 309- 314
doi: 10.1109/TSM.2018.2795466
[27]   KYEONG K, KIM H Classification of mixed-type defect patterns in wafer bin maps using convolutional neural networks[J]. IEEE Transactions on Semiconductor Manufacturing, 2018, 31 (3): 395- 402
doi: 10.1109/TSM.2018.2841416
[28]   LEE H, KIM Y, KIM C O A deep learning model for robust wafer fault monitoring with sensor measurement noise[J]. IEEE Transactions on Semiconductor Manufacturing, 2017, 30 (1): 23- 31
doi: 10.1109/TSM.2016.2628865
[29]   PAN S J, YANG Q A survey on transfer learning[J]. IEEE Transactions on Knowledge and Data Engineering, 2010, 22 (10): 1345- 1359
doi: 10.1109/TKDE.2009.191
[30]   张雪松, 庄严, 闫飞, 等 基于迁移学习的类别级物体识别与检测研究与进展[J]. 自动化学报, 2019, 45 (7): 1224- 1243
ZHANG Xue-song, ZHUANG Yan, YAN Fei, et al Status and development of transfer learning based category-Level object recognition and detection[J]. Acta Automatica Sinica, 2019, 45 (7): 1224- 1243
[31]   DONAHUE J, JIA Y, VINYALS O et al. DeCAF: a deep convolutional activation feature for generic visual recognition [C] // International Conference on Machine Learning (ICML). Bengjin: [s. n.], 2014: 647-655.
[32]   ZHOU Z H, FENG J. Deep forest: towards an alternative to deep neural networks [C] // International Joint Conference on Artificial Intelligence (IJCAI). Melbourne: [s. n.], 2017: 3553-3559.
[33]   UTKIN L V, RYABININ M A. A deep Forest for transductive transfer learning by using a consensus measure [C] // Conference on Artificial Intelligence and Natural Language (AINL). Petersburg: Springer, 2017: 194-208.
[34]   SRIVASTAVA R K, GREFF K, SCHMIDHUBER J, et al. Highway networks [EB/OL].（2015-11-03）[2019-04-30], https://arxiv.org/abs/1505.00387
[35]   LIU F T, TING K M, YU Y, et al Spectrum of variable-random trees[J]. Journal of Artificial Intelligence Research, 2008, 32 (1): 355- 384
[36]   GUO Y C. Knowledge-enabled short-term load forecasting based on pattern-base using classification & regression tree and support vector regression [C] // Fifth International Conference on Natural Computation. Tianjin: IEEE, 2009: 425-429.
[37]   WU M J, JANG J S R, CHEN J L Wafer map failure pattern recognition and similarity ranking for large-scale data sets[J]. IEEE Transactions on Semiconductor Manufacturing, 2015, 28 (1): 1- 12
doi: 10.1109/TSM.2014.2364237
[1] Yi-peng HUANG,Ji-su HU,Xu-sheng QIAN,Zhi-yong ZHOU,Wen-lu ZHAO,Qi MA,Jun-kang SHEN,Ya-kang DAI. SE-Mask-RCNN: segmentation method for prostate cancer on multi-parametric MRI[J]. Journal of ZheJiang University (Engineering Science), 2021, 55(1): 203-212.
[2] Qiao-hong CHEN,YI CHEN,Wen-shu Li,Yu-bo JIA. Clothing image classification based on multi-scale SE-Xception[J]. Journal of ZheJiang University (Engineering Science), 2020, 54(9): 1727-1735.
[3] Yan-nan ZHANG,Xiao-hong HUANG,Yan MA,Qun CONG. Method with recording text classification based on deep learning[J]. Journal of ZheJiang University (Engineering Science), 2020, 54(7): 1264-1271.
[4] Zhuang KANG,Jie YANG,Hao-qi GUO. Automatic garbage classification system based on machine vision[J]. Journal of ZheJiang University (Engineering Science), 2020, 54(7): 1272-1280.
[5] Hai-jin WANG,Zong-yu YIN,Zhen-zheng KE,Ying-jie GUO,Hui-yue DONG. Wear monitoring of helical milling tool based on one-dimensional convolutional neural network[J]. Journal of ZheJiang University (Engineering Science), 2020, 54(5): 931-939.
[6] Ping YANG,Dan WANG,Zi-jian KAGN,Tong LI,Li-hua FU,Yue-ren YU. Prediction model of paroxysmal atrial fibrillation based on pattern recognition and ensemble CNN-LSTM[J]. Journal of ZheJiang University (Engineering Science), 2020, 54(5): 1039-1048.
[7] Xiao-feng FU,Li NIU,Zhuo-qun HU,Jian-jun LI,Qing WU. Deep micro-expression spotting network training based on concept of transition frame[J]. Journal of ZheJiang University (Engineering Science), 2020, 54(11): 2128-2137.
[8] Zi-yu JIA,You-fang LIN,Hong-jun ZHANG,Jing WANG. Sleep stage classification model based ondeep convolutional neural network[J]. Journal of ZheJiang University (Engineering Science), 2020, 54(10): 1899-1905.
[9] Gang YE,Yi-bo LI,Zhu-xi MA,Jie CHENG. End-to-end aluminum strip surface defects detection and recognition method based on ViBe[J]. Journal of ZheJiang University (Engineering Science), 2020, 54(10): 1906-1914.
[10] Hong-guang LI,Ying GUO,Ping SUI,Zi-sen QI. Frequency hopping modulation recognition of convolutional neural network based on time-frequency characteristics[J]. Journal of ZheJiang University (Engineering Science), 2020, 54(10): 1945-1954.
[11] Yan LV,Meng ZHANG,Wu-hao JIANG,Yi-hua NI,Xiao-hong QIAN. Design of elderly fall detection system using CNN[J]. Journal of ZheJiang University (Engineering Science), 2019, 53(6): 1130-1138.
[12] Gui-ran HE,Qi LI,Hua-jun FENG,Zhi-hai XU,Yue-ting CHEN. Dual-focal camera continuous digital zoom based onCNN and feature extraction[J]. Journal of ZheJiang University (Engineering Science), 2019, 53(6): 1182-1189.
[13] Yue DONG,Hua-jun FENG,Zhi-hai XU,Yue-ting CHEN,Qi LI. Attention Res-Unet: an efficient shadow detection algorithm[J]. Journal of ZheJiang University (Engineering Science), 2019, 53(2): 373-381.
[14] Jing-chang WANG,Ling CHEN,Shan-shan YU,Chen-shu JIANG,Yong WU. Multi-factor perceived short-term tourist number prediction model based on gated recurrent unit[J]. Journal of ZheJiang University (Engineering Science), 2019, 53(12): 2357-2364.
[15] Hao LI,Wen-jie ZHAO,Bo HAN. Convolutional neural network acceleration algorithm based on filters pruning[J]. Journal of ZheJiang University (Engineering Science), 2019, 53(10): 1994-2002.