Please wait a minute...
浙江大学学报(工学版)
浙江大学学报(工学版)  2020, Vol. 54 Issue (6): 1228-1239    DOI: 10.3785/j.issn.1008-973X.2020.06.021
机械工程     
基于迁移学习与深度森林的晶圆图缺陷识别
沈宗礼(),余建波*()
同济大学 机械与能源工程学院，上海 201804
Wafer map defect recognition based on transfer learning and deep forest
Zong-li SHEN(),Jian-bo YU*()
School of Mechanical Engineering, Tongji University, Shanghai 201804, China
 全文: PDF(1461 KB)   HTML
摘要：

为了有效识别晶圆图缺陷模式并及时诊断制造过程的故障源，提出基于迁移学习和深度森林集成的DenseNet-GCForest晶圆图缺陷模式识别模型. 为了解决深度学习模型训练困难和晶圆图缺陷类型数目不平衡的问题，利用迁移学习将深度卷积神经网络DenseNet在ImageNet上预训练的网络权重参数迁移至本模型并重新设计分类层，以减少深度网络模型的训练时间并提高模型的特征提取能力；基于DenseNet网络提取的高维抽象晶圆图特征，引入深度森林模型进行晶圆图特征缺陷模式识别. 工业案例的实验验证结果表明，该方法的识别准确率达到了96.8%，并提高了识别效率，其性能优于典型的卷积神经网络以及其他常用识别方法.
关键词： 半导体制造;  晶圆缺陷;  迁移学习;  卷积神经网络;  深度森林    
Abstract:

A wafer map pattern recognition (WMPR) model was proposed based on transfer learning and deep forest, in order to identify the defect pattern of the wafer maps and to timely diagnose the source of the fault in the manufacturing process. Transfer learning was used to migrate the network weight parameters of the deep CNN DenseNet pre-trained on ImageNet to this model, and the classification layer of the model was redesigned, in order to solve the problems of difficulties of deep learning model training and imbalance in the number of defect types in wafer maps. Thus, the training time of the model was reduced and the feature extraction ability was improved. Deep forest model was introduced to identify the wafer defect pattern, based on the abstract features of the wafer maps extracted by DenseNet. The experimental results on an industrial case demonstrated that the average recognition rate was about 96.8%. This method can improve the recognition efficiency and its performance is better than those well-known CNNs and other typical classifiers.
Key words: semiconductor manufacturing    wafer defect    transfer learning    CNN    deep forest
收稿日期: 2019-04-30 出版日期: 2020-07-06
CLC:  TP 391.41  
基金资助: 国家自然科学基金资助项目 (71777173)；上海科委“科技创新行动计划”高新技术领域资助项目（19511106303）；中央高校基本科研业务费资助项目（22120180068，22120190196）
通讯作者: 余建波     E-mail: 979949752@qq.com;jbyu@tongji.edu.cn
作者简介: 沈宗礼(1996—)，男，硕士生，从事深度学习与故障诊断研究. orcid.org/0000-0002-0503-1709. E-mail： 979949752@qq.com
服务　　
把本文推荐给朋友
加入引用管理器
E-mail Alert
作者相关文章　　
沈宗礼
余建波

引用本文:

沈宗礼,余建波. 基于迁移学习与深度森林的晶圆图缺陷识别[J]. 浙江大学学报(工学版), 2020, 54(6): 1228-1239.

Zong-li SHEN,Jian-bo YU. Wafer map defect recognition based on transfer learning and deep forest. Journal of ZheJiang University (Engineering Science), 2020, 54(6): 1228-1239.

链接本文:

http://www.zjujournals.com/eng/CN/10.3785/j.issn.1008-973X.2020.06.021    或    http://www.zjujournals.com/eng/CN/Y2020/V54/I6/1228

图 1  Dense Block 密集连接模块框架
图 2  基于迁移学习的DenseNet网络结构
图 3  GCForest的类向量生成
图 4  深度森林网络结构（以400维输入为例）
图 5  DenseNet-GCForest晶圆图缺陷识别流程
图 6  正常晶圆图模式与8种缺陷模式
图 7  晶圆缺陷类型数量统计
图 8  基于迁移学习的DenseNet169的训练过程
图 9  晶圆图多级网络输出特征
图 10  原始数据和DenseNet特征的可视化分析
%
预测真实 None Center Donut Edge-local Edge-ring Local Near-full Random Scratch
None 99.59 0.00 0 0.28 0 0 0 0 0.14
Center 0.83 97.52 0 0.83 0 0 0 0 0.83
Donut 0 0 94.74 0 0 5.26 0 0 0
Edge-local 0.86 0.29 0 96.83 0.29 1.44 0 0.29 0
Edge-ring 0 0 0 4.91 95.09 0 0 0 0
Local 0.75 0.38 0 1.51 0 96.23 0 0.75 0.38
Near-full 0 0 0 0 0 0 94.12 5.88 0
Random 0 2.56 0 0 2.56 5.13 2.56 87.18 0
Scratch 1.25 0 0 1.25 0 3.75 0 0 93.75
表 1  Densenet-GCForest晶圆缺陷识别率混淆矩阵
图 11  Random类的误识别情况
参数名称 参数大小 Racc/%
滑动窗口大小 400 97.31
700 96.7
1 000 96.2
1 300 96.4
1 664 96.2
决策树生成的
最小样本数 		0 96.2
0.1 96.7
0.2 96.2
0.3 95.8
0.4 95.4
决策树生成的
允许误差 		0 96.2
0.1 96.7
0.2 96.2
0.3 95.8
0.4 95.4
扫描层随机森林的
决策树数量 		200 96.9
400 95.8
600 97.0
800 96.7
1 000 96.0
表 2  GCForest识别器参数灵敏度分析
%
实验方法 缺陷类别 Racc Rrec F
DenseNet-GCForest Center 97.5 97.5 97.5
Donut 100 94.7 97.3
Edge_local 95.5 96.8 96.1
Edge_ring 98.7 95.1 96.9
Local 95.9 96.2 96.0
Near_full 94.1 94.1 94.1
None 99.0 99.6 99.3
Random 89.5 87.2 88.3
Scratch 96.2 93.8 94.9
GoogleNet Center 72.2 34.5 46.7
Donut 59.1 43.3 50.0
Edge_local 63.9 79.8 71.0
Edge_ring 93.4 87.6 90.4
Local 57.3 48.8 52.7
Near_full 93.3 93.3 93.3
None 90.6 92.9 91.7
Random 85.4 92.1 88.6
Scratch 81.7 81.7 81.7
ResNet Center 85.0 77.1 80.9
Donut 47.2 81.0 59.6
Edge_local 87.6 81.4 84.4
Edge_ring 94.3 86.6 90.3
Local 81.7 68.6 74.6
Near_full 75.0 75.0 75.0
None 89.9 99.3 94.4
Random 88.2 73.2 80.0
Scratch 59.5 71.0 64.4
DenseNet Center 76.4 85.5 80.7
Donut 95.2 64.5 76.9
Edge_local 79.7 91.2 85.0
Edge_ring 96.1 81.5 88.2
Local 82.1 66.8 73.6
Near_full 91.7 100 95.7
None 95.2 99.3 97.2
Random 86.4 65.5 74.5
Scratch 81.5 72.6 76.8
Decision TreeEnsemble Center 95.6 93.75 94.7
Donut 92.6 92.3 92.4
Edge_local 83.5 87.3 85.4
Edge_ring 86.8 91.1 88.9
Local 83.5 82.3 82.9
Near_full 89.4 91.7 90.5
None 100 99.5 99.7
Random 91.7 87.3 88.4
Scratch 86.0 88.5 87.2
SDAE Center 98.5 87.3 92.6
Donut 87.6 89.1 88.4
Edge_local 87.5 85.4 86.4
Edge_ring 98.8 94.7 96.7
Local 83.5 55.8 66.9
Near_full 85.3 96.2 90.4
None 99.0 98.7 98.9
Random 93.3 89.3 91.2
Scratch 81.7 39.6 53.3
表 3  DenseNet-GCForest算法的对比实验结果
分类器 Racc/% 分类器 Racc/%
DenseNet-GCForest 96.8 GCForest 73.7
C4.5 Ensemble 90.8 RF 68.9
SDAE 89.4 SVML 72.5
DenseNet 88.6 SVMG 40.2
GoogleNet 74.3 KNN 30.1
ResNet 86.5 C4.5 62.4
表 4  五折交叉识别率对比
分类器 Racc/% 分类器 Racc/%
GCForest 96.8 SVMG 95.5
BPN 85.1 KNN 92.6
RF 95.1 C4.5 87.2
SVML 95.6 ? ?
表 5  基于DenseNet特征的多种识别器五折交叉识别率对比
1 TAO Y, WAY K, SUK J B Detection of spatial defect patterns generated in semiconductor fabrication processes[J]. IEEE Transactions on Semiconductor Manufacturing, 2011, 24 (3): 392- 403
doi: 10.1109/TSM.2011.2154870
2 KIM B, JEONG Y S, TONG S H, et al Step-down spatial randomness test for detecting abnormalities in DRAM wafers with multiple spatial maps[J]. IEEE Transactions on Semiconductor Manufacturing, 2016, 29 (1): 57- 65
doi: 10.1109/TSM.2015.2486383
3 YU J, LU X Wafer map defect detection and recognition using joint local and nonlocal linear discriminant analysis[J]. IEEE Transactions on Semiconductor Manufacturing, 2016, 29 (1): 33- 42
doi: 10.1109/TSM.2015.2497264
4 SHANKAR N G, ZHONG Z W Defect detection on semiconductor wafer surfaces[J]. Microelectronic Engineering, 2005, 77 (3): 337- 346
5 LEE S H, KOO H I, CHO N I New automatic defect classification algorithm based on a classification-after-segmentation framework[J]. Journal of Electronic Imaging, 2010, 19 (2): 334- 343
6 HWANG J Y, KUO W Model-based clustering for integrated circuit yield enhancement[J]. European Journal of Operational Research, 2007, 178 (1): 143- 153
doi: 10.1016/j.ejor.2005.11.032
7 BALY R, HAJJ H Wafer classification using support vector machines[J]. IEEE Transactions on Semiconductor Manufacturing, 2012, 25 (3): 373- 383
doi: 10.1109/TSM.2012.2196058
8 XIE L, HUANG R, GU N, et al A novel defect detection and identification method in optical inspection[J]. Neural Computing and Applications, 2014, 24 (7/8): 1953- 1962
doi: 10.1007/s00521-013-1442-7
9 CHAO L C, TONG L I Wafer defect pattern recognition by multi-class support vector machines by using a novel defect cluster index[J]. Expert Systems with Applications, 2009, 36 (6): 10158- 10167
doi: 10.1016/j.eswa.2009.01.003
10 ADLY F, ALUSSEIN O, YOO P, et al Simplified subspaced regression network for identification of defect patterns in semiconductor wafer maps[J]. IEEE Transactions on Semiconductor Manufacturing, 2015, 11 (6): 1267- 1276
11 KIM B, JEONG Y S, TONG S H, et al A regularized singular value decomposition-based approach for failure pattern classification on fail bit map in a DRAM wafer[J]. IEEE Transactions on Semiconductor Manufacturing, 2015, 28 (1): 41- 49
doi: 10.1109/TSM.2014.2388192
12 OOI M P L, SOK H K, KUANG Y C, et al Defect cluster recognition system for fabricated semiconductor wafers[J]. Engineering Applications of Artificial Intelligence, 2013, 26 (3): 1029- 1043
doi: 10.1016/j.engappai.2012.03.016
13 余建波, 卢笑蕾, 宗卫周 基于局部与非局部线性判别分析和高斯混合模型动态集成的晶圆表面缺陷探测与识别[J]. 自动化学报, 2016, 42 (1): 47- 59
YU Jian-bo, LU Xiao-lei, ZONG Wei-zhou Wafer defect detection and recognition based on local and nonlocal linear discriminant analysis and dynamic ensemble of gaussian mixture models[J]. ACTA Automatica Science, 2016, 42 (1): 47- 59
14 PIAO M, JIN C H, LEE J Y, et al Decision tree ensemble-based wafer map failure pattern recognition based on radon transform-based features[J]. IEEE Transactions on Semiconductor Manufacturing, 2018, 31 (2): 250- 257
doi: 10.1109/TSM.2018.2806931
15 SAQLAIN M, JARGALSAIKHAN B, LEE J Y A voting ensemble classifier for wafer map defect patterns identification in semiconductor Manufacturing[J]. IEEE Transactions on Semiconductor Manufacturing, 2019, 32 (2): 171- 182
doi: 10.1109/TSM.2019.2904306
16 LECUN Y, BENGIO Y, HINTON G Deep learning[J]. Nature, 2015, 521: 436- 444
doi: 10.1038/nature14539
17 杨婧, 耿辰, 王海林, 纪建松, 等 基于DenseNet的低分辨CT影像肺腺癌组织学亚型分类[J]. 浙江大学学报: 工学版, 2019, 53 (6): 1164- 1170
YANG Jing, GENG Chen, WANG Hai-lin, et al Classification on histological subtypes of lung adenocarcinoma from low-resolution CT images based on DenseNet[J]. Journal of Zhejiang Unversity: Engineering Science, 2019, 53 (6): 1164- 1170
18 THIRUKOVALLURU R, DIXIT S, SEVAKULA RK et al. Generating feature sets for fault diagnosis using denoising stacked auto-encoder [C] // IEEE International Conference in Prognostics and Health Management (ICPHM). Ottawa: IEEE, 2016: 1-7.
19 JIANG G, HE H, XIE P, et al Stacked Multilevel-Denoising Autoencoders: a new representation learning approach for wind turbine gearbox fault diagnosis[J]. IEEE Transactions on Instrumentation and Measurement, 2017, 66: 2391- 2402
doi: 10.1109/TIM.2017.2698738
20 KRIZHEVSKY A, SUTSKEVER I, HINTON GE. Imagenet classification with deep convolutional neural networks [C] // Advances in Neural Information Processing Systems (NIPS). Lake Tahoe: [s. n.], 2012: 1097–1105.
21 袁公萍, 汤一平, 韩旺明, 等 基于深度卷积神经网络的车型识别方法[J]. 浙江大学学报: 工学版, 2018, 52 (4): 694- 702
YUAN Gong-ping, TANG Yi-ping, et al Vehicle category recognition based on deep convolutional neural network[J]. Journal of Zhejiang University: Engineering Science, 2018, 52 (4): 694- 702
22 SZEGEDY C, LIU W, JIA Y, et al. Going deeper with convolutions [C] // 2015 IEEE Conference on Computer Vision and Pattern Recognition (CVPR). Boston: IEEE, 2015: 1-9.
23 HE K M, ZHANG X Y, REN S Q, et al. Deep residual learning for image recognition [C] // 2016 IEEE Conference on Computer Vision and Pattern Recognition (CVPR). Las Vegas: IEEE, 2016: 770-778..
24 HUANG G, LIU Z, MAATEN L V D et al. Densely connected convolutional networks [C] // 2017 IEEE Conference on Computer Vision and Pattern Recognition (CVPR). Honolulu IEEE, 2017: 2261-2260.
25 DANIEL W, BERND S R, MOSHE S Design of deep convolutional neural network architectures for automated feature extraction in industrial inspection[J]. CIRP Annual Manufacturing Technology, 2016, 65 (1): 417- 420
doi: 10.1016/j.cirp.2016.04.072
26 NAKAZAWA T, KULKARNI D V Wafer map defect pattern classification and image retrieval using convolutional neural network[J]. IEEE Transactions on Semiconductor Manufacturing, 2018, 31 (2): 309- 314
doi: 10.1109/TSM.2018.2795466
27 KYEONG K, KIM H Classification of mixed-type defect patterns in wafer bin maps using convolutional neural networks[J]. IEEE Transactions on Semiconductor Manufacturing, 2018, 31 (3): 395- 402
doi: 10.1109/TSM.2018.2841416
28 LEE H, KIM Y, KIM C O A deep learning model for robust wafer fault monitoring with sensor measurement noise[J]. IEEE Transactions on Semiconductor Manufacturing, 2017, 30 (1): 23- 31
doi: 10.1109/TSM.2016.2628865
29 PAN S J, YANG Q A survey on transfer learning[J]. IEEE Transactions on Knowledge and Data Engineering, 2010, 22 (10): 1345- 1359
doi: 10.1109/TKDE.2009.191
30 张雪松, 庄严, 闫飞, 等 基于迁移学习的类别级物体识别与检测研究与进展[J]. 自动化学报, 2019, 45 (7): 1224- 1243
ZHANG Xue-song, ZHUANG Yan, YAN Fei, et al Status and development of transfer learning based category-Level object recognition and detection[J]. Acta Automatica Sinica, 2019, 45 (7): 1224- 1243
31 DONAHUE J, JIA Y, VINYALS O et al. DeCAF: a deep convolutional activation feature for generic visual recognition [C] // International Conference on Machine Learning (ICML). Bengjin: [s. n.], 2014: 647-655.
32 ZHOU Z H, FENG J. Deep forest: towards an alternative to deep neural networks [C] // International Joint Conference on Artificial Intelligence (IJCAI). Melbourne: [s. n.], 2017: 3553-3559.
33 UTKIN L V, RYABININ M A. A deep Forest for transductive transfer learning by using a consensus measure [C] // Conference on Artificial Intelligence and Natural Language (AINL). Petersburg: Springer, 2017: 194-208.
34 SRIVASTAVA R K, GREFF K, SCHMIDHUBER J, et al. Highway networks [EB/OL].（2015-11-03）[2019-04-30], https://arxiv.org/abs/1505.00387
35 LIU F T, TING K M, YU Y, et al Spectrum of variable-random trees[J]. Journal of Artificial Intelligence Research, 2008, 32 (1): 355- 384
36 GUO Y C. Knowledge-enabled short-term load forecasting based on pattern-base using classification & regression tree and support vector regression [C] // Fifth International Conference on Natural Computation. Tianjin: IEEE, 2009: 425-429.
37 WU M J, JANG J S R, CHEN J L Wafer map failure pattern recognition and similarity ranking for large-scale data sets[J]. IEEE Transactions on Semiconductor Manufacturing, 2015, 28 (1): 1- 12
doi: 10.1109/TSM.2014.2364237
[1] 郑英杰,吴松荣,韦若禹,涂振威,廖进,刘东. 基于目标图像FCM算法的地铁定位点匹配及误报排除方法[J]. 浙江大学学报(工学版), 2021, 55(3): 586-593.
[2] 陈巧红,陈翊,李文书,贾宇波. 多尺度SE-Xception服装图像分类[J]. 浙江大学学报(工学版), 2020, 54(9): 1727-1735.
[3] 周登文,田金月,马路遥,孙秀秀. 基于多级特征并联的轻量级图像语义分割[J]. 浙江大学学报(工学版), 2020, 54(8): 1516-1524.
[4] 张彦楠,黄小红,马严,丛群. 基于深度学习的录音文本分类方法[J]. 浙江大学学报(工学版), 2020, 54(7): 1264-1271.
[5] 康庄,杨杰,郭濠奇. 基于机器视觉的垃圾自动分类系统设计[J]. 浙江大学学报(工学版), 2020, 54(7): 1272-1280.
[6] 闫旭,范晓亮,郑传潘,臧彧,王程,程明,陈龙彪. 基于图卷积神经网络的城市交通态势预测算法[J]. 浙江大学学报(工学版), 2020, 54(6): 1147-1155.
[7] 汪海晋,尹宗宇,柯臻铮,郭英杰,董辉跃. 基于一维卷积神经网络的螺旋铣刀具磨损监测[J]. 浙江大学学报(工学版), 2020, 54(5): 931-939.
[8] 杨萍,王丹,康子健,李童,付利华,余悦任. 基于模式识别和集成CNN-LSTM的阵发性房颤预测模型[J]. 浙江大学学报(工学版), 2020, 54(5): 1039-1048.
[9] 金列俊,詹建明,陈俊华,王涛. 基于一维卷积神经网络的钻杆故障诊断[J]. 浙江大学学报(工学版), 2020, 54(3): 467-474.
[10] 付晓峰,牛力,胡卓群,李建军,吴卿. 基于过渡帧概念训练的微表情检测深度网络[J]. 浙江大学学报(工学版), 2020, 54(11): 2128-2137.
[11] 李红光,郭英,眭萍,齐子森. 基于时频特征的卷积神经网络跳频调制识别[J]. 浙江大学学报(工学版), 2020, 54(10): 1945-1954.
[12] 叶刚,李毅波,马逐曦,成杰. 基于ViBe的端到端铝带表面缺陷检测识别方法[J]. 浙江大学学报(工学版), 2020, 54(10): 1906-1914.
[13] 贾子钰,林友芳,张宏钧,王晶. 基于深度卷积神经网络的睡眠分期模型[J]. 浙江大学学报(工学版), 2020, 54(10): 1899-1905.
[14] 王万良,杨小涵,赵燕伟,高楠,吕闯,张兆娟. 采用卷积自编码器网络的图像增强算法[J]. 浙江大学学报(工学版), 2019, 53(9): 1728-1740.
[15] 段然,周登文,赵丽娟,柴晓亮. 基于多尺度特征映射网络的图像超分辨率重建[J]. 浙江大学学报(工学版), 2019, 53(7): 1331-1339.