1. State Key Laboratory of Fluid Power and Mechatronic Systems, Zhejiang University, Hangzhou 310027, China 2. College of Mechanical Engineering, Zhejiang University of Technology, Hangzhou 310014, China
There are many problems in mechanical drawings, such as thousands of geometric features, confusing line styles and various manual drawing styles, which lead to low proofreading efficiency, false picking and missing inspection. Aiming at the above problem, an intelligent proofreading method of mechanical drawings based on transfer learning was presented. Firstly, the mechanical drawings were preprocessed, an improved threshold iterative algorithm was used to remove the background, noise and interference, and then the projections of mechanical drawings were obtained by segmentation operations. Secondly, by training the feature extractor of the source domain image, the network weight of the feature extractor was transferred to the mechanical drawing evaluation model and the knowledge transfer of similar domain was completed. Finally, the logistic regression classifier was trained and the intelligent evaluation model was established based on self-adjustment of neural network weight parameter. The standard elements of a mechanical drawing such as geometric feature, projection feature, line and section plane symbols were recognized. Experimental results show that the average proofreading time was 0.95 s, and the average judgement accuracy of engineering drawing was 98.82%. Compared with human proofreading of mechanical drawings, the proposed method has higher efficiency and accuracy.
Yi-cong GAO,Yan-kun WANG,Shao-mei FEI,Qiong LIN. Intelligent proofreading method of engineering drawing based on transfer learning. Journal of ZheJiang University (Engineering Science), 2022, 56(5): 856-863, 889.
Fig.1Original image and preprocessing of mechanical drawings
Fig.2Evaluation criteria of mechanical drawing
Fig.3Feature segmentation of mechanical drawing
Fig.4Example of data enhancement operation
Fig.5Structure of DenseNet121 network
Fig.6Flow chart of proposed intelligent proofreading method of engineering drawing based on transfer learning
Fig.7Comparison of loss value and accuracy of transfer learning and non-transfer learning
得分 点编号
ntra
Acc/%
DenseNet121 （无迁移 学习）
VGG16 （使用迁 移学习）
ResNet18 （使用迁 移学习）
DenseNet121 （使用迁 移学习）
1
80
81.11
95.56
94.44
94.44
150
88.10
95.56
97.78
97.78
2
80
77.78
90.85
89.63
91.64
150
81.25
97.48
93.71
98.11
3
80
75.00
82.89
76.32
84.21
150
75.68
83.67
86.49
89.19
4
80
94.32
95.45
93.18
97.73
150
95.83
95.83
97.22
98.61
5
80
88.89
95.15
96.12
95.15
150
93.07
97.03
97.03
97.03
6
80
91.21
96.04
94.06
98.90
150
97.80
100.00
100.00
100.00
7
80
82.91
91.45
90.60
94.02
150
84.42
98.29
97.44
97.44
Tab.1Comparison of recognition results between non-transfer learning and transfer learning in four models
Fig.8Thumbnails of review experiments
编号
实验内容
ns
ntra
ntes
1
线条补齐评阅
5
143
50
2
三视图绘制评阅
7
156
50
3
轴系判改评阅
10
152
50
Tab.2Design of contrast experiments of manual proof reading and proposed intelligent method
编号
t/s
Acc/%
人工
本研究方法
人工
本研究方法
1
224
35
100
98.40
2
298
50
100
98.86
3
472
58
100
99.20
平均值
331.3
47.7
100.0
98.82
Tab.3Results of manual proofreading and proposed intelligent method
[1]
谭建荣, 张树有, 陆国栋. 图学基础教程[M]. 北京: 高等教育出版社, 2006.
[2]
费少梅, 陆国栋, 顾大强 时空融合知行耦合的机械大类课程教学新范式探索实践[J]. 高等工程教育研究, 2017, (6): 76- 80 FEI Shao-mei, LU Guo-dong, GU Da-qiang Exploration and practice of a new teaching paradigm of mechanical courses based on the integration of time and space, knowledge and practice[J]. Research on Higher Engineering Education, 2017, (6): 76- 80
[3]
SANCHEZ J, MONZON N, AGUSTIN S An analysis and implementation of the Harris corner detector[J]. Image Processing On Line, 2018, 8: 305- 328
doi: 10.5201/ipol.2018.229
[4]
冯毅雄, 李康杰, 高一聪, 等 面向视觉伺服的工业机器人轮廓曲线角点识别[J]. 浙江大学学报:工学版, 2020, 54 (8): 6- 13 FENG Yi-xiong, LI Kang-jie, GAO Yi-cong, et al Corner recognition of industrial robot contour curve for visual servo[J]. Journal of Zhejiang University: Engineering Science, 2020, 54 (8): 6- 13
[5]
冯毅雄, 李康杰, 高一聪, 等 基于特征与形貌重构的轴件表面缺陷检测方法[J]. 浙江大学学报:工学版, 2020, 54 (3): 427- 434 FENG Yi-xiong, LI Kang-jie, GAO Yi-cong, et al The method of surface defect detection of shaft parts based on feature and morphology reconstruction[J]. Journal of Zhejiang University: Engineering Science, 2020, 54 (3): 427- 434
[6]
KYEONG K, KIM H Classification of mixed-type defect patterns in wafer bin maps using convolutional neural networks[J]. IEEE Transactions on Semiconductor Manufacturing, 2018, 31 (3): 395- 402
doi: 10.1109/TSM.2018.2841416
[7]
LEE H, KIM Y, KIM C O A deep learning model for robust wafer fault monitoring with sensor measurement noise[J]. IEEE Transactions on Semiconductor Manufacturing, 2017, 30 (2): 23- 31
[8]
SUDHA S, VIDHYALAKSHMI M, PAVITHRA K, et al. An automatic classification method for environment: Friendly waste segregation using deep learning [C]// 2016 IEEE Technological Innovations in ICT for Agriculture and Rural Development (TIAR). Chennai: IEEE, 2016: 65-70.
[9]
DELATTE D M, CRITES S T, GUTTENBERG N, et al Segmentation convolutional neural networks for automatic crater detection on Mars[J]. IEEE Journal of Selected Topics in Applied Earth Observations and Remote Sensing, 2019, 12 (8): 2944- 2957
doi: 10.1109/JSTARS.2019.2918302
[10]
JIN Y, HE F, LIU S, et al. Small scale crater detection based on deep learning with multi-temporal samples of high-resolution images [C]// 2019 10th International Workshop on the Analysis of Multitemporal Remote Sensing Images. Shanghai: IEEE, 2019: 1-4.
[11]
PALESTRA G, PETTINICCHIO A. Improved performance in facial expression recognition using 32 geometric features [C]∥International Conference on Image Analysis and Processing. Genova: ICIAR, 2015: 518-528.
[12]
ZHANG Z, JAISWAL P, RAI R FeatureNet: machining feature recognition based on 3D convolution neural network[J]. Computer-Aided Design, 2018, 101: 12- 22
doi: 10.1016/j.cad.2018.03.006
[13]
NATARAJAN V, HUNG T Y, VAIKUNDAM S, et al. Convolutional networks for voting-based anomaly classification in metal surface inspection [C]// IEEE International Conference on Industrial Technology. Toronto: IEEE, 2017.
[14]
魏域君. 顾及几何与拓扑特征的深度学习遥感影像道路提取[D]. 武汉: 武汉大学, 2020. WEI Yu-jun. Road extraction from remote sensed imagery based on deep learning considering geometric and topological properties [D]. Wuhan: Wuhan University, 2020.
[15]
韩丽, 朴京钰, 兰鹏燕, 等 结构感知深度学习的三维形状分类方法[J]. 计算机辅助设计与图形学学报, 2021, 33 (1): 29- 38 HAN Li, PIAO Jing-yu, LAN Peng-yan, et al 3D shape classification method based on shape-aware deep learning[J]. Journal of Computer-Aided Design and Computer Graphics, 2021, 33 (1): 29- 38
doi: 10.3724/SP.J.1089.2021.18280
[16]
李军军, 曹建农, 朱莹莹, 等 高分辨率遥感影像建筑区域局部几何特征提取[J]. 遥感学报, 2020, 24 (3): 233- 244 LI Jun-jun, CAO Jian-nong, ZHU Ying-ying, et al Built-up area detection from high resolution remote sensing images using geometric features[J]. Journal of Remote Sensing, 2020, 24 (3): 233- 244
[17]
SONG R, XIAO Z, LIN J, et al CIES: cloud-based intelligent evaluation service for video homework using CNN-LSTM network[J]. Journal of Cloud Computing, 2020, 9 (1): 1- 9
[18]
LIN J, ZHAO Y, LIU C, et al Abnormal video homework automatic detection system[J]. Journal of Ambient Intelligence and Humanized Computing, 2021, 12 (12): 10529- 10537
doi: 10.1007/s12652-020-02860-9
[19]
JING S, SANTOS O C, BOTICARIO J G, et al. Automatic grading of short answers for MOOC via semi-supervised document clustering [C]// International Conference on Educational Data Mining. Madrid: EDM, 2015: 554-555.
[20]
张旻. 智能图像识别在初中几何自动阅卷中的应用研究[D]. 北京: 北京工业大学, 2019. ZHANG Min. Research of application for intelligent image recognition in junior high school geometry automatic marking[D]. Beijing: Beijing University of Technology, 2019.
[21]
李万秋. 机械基础类客观题及图形改判题阅卷系统的关键技术研究[D]. 广州: 华南理工大学, 2019. LI Wan-qiu. Study on key techniques of marking system for objective questions and figure correction questions in basic mechanical courses[D]. Guangzhou: South China University of Technology, 2019.
[22]
沈银燕. 基于网络的建筑CAD考试系统的设计与实现[D]. 杭州: 浙江工业大学, 2009. SHEN Yin-yan. The design and implementation of the examination system of architectural CAD based on web [D]. Hangzhou: Zhejiang University of Technology, 2009.
[23]
易琳. CAD教学中考试及自动评卷系统的研究[D]. 成都: 电子科技大学, 2007. YI Lin. The research on the system of examination and automatic marking in cad teaching [D]. Chengdu: University of Electronic Science and Technology of China, 2007.
[24]
CANNY J A computational approach to edge detection[J]. IEEE Transactions on Pattern Analysis and Machine Intelligence, 1986, 8 (6): 679- 698
[25]
OTSU N A threshold selection method from gray-level histograms[J]. IEEE Transactions on Systems Man and Cybernetics, 2007, 9 (1): 62- 66
[26]
HUANG G, LIU Z, VAN D M, et al. Densely connected convolutional networks [C]// Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition. Honolulu: IEEE, 2017: 4700-4708.
[27]
SIMONYAN K, ZISSERMAN A Very deep convolutional networks for large-scale image recognition[J]. Computer Science, 2014, 34 (2): 1409- 1422
HU Li-sha, WANG Su-zhen, CHEN Yi-qiang, GAO Chen-long, HU Chun-yu, JIANG Xin-long, CHEN Zhen-yu, GAO Xing-yu. Fall detection algorithms based on wearable device: a review[J]. Journal of ZheJiang University (Engineering Science), 2018, 52(9): 1717-1728.
