Please wait a minute...
浙江大学学报(工学版)
Journal of ZheJiang University (Engineering Science)  2022, Vol. 56 Issue (5): 856-863, 889    DOI: 10.3785/j.issn.1008-973X.2022.05.002
    
Intelligent proofreading method of engineering drawing based on transfer learning
Yi-cong GAO1(),Yan-kun WANG1,Shao-mei FEI1,Qiong LIN2,*()
1. State Key Laboratory of Fluid Power and Mechatronic Systems, Zhejiang University, Hangzhou 310027, China
2. College of Mechanical Engineering, Zhejiang University of Technology, Hangzhou 310014, China
Download: HTML     PDF(2419KB) HTML
Export: BibTeX | EndNote (RIS)      

Abstract  

There are many problems in mechanical drawings, such as thousands of geometric features, confusing line styles and various manual drawing styles, which lead to low proofreading efficiency, false picking and missing inspection. Aiming at the above problem, an intelligent proofreading method of mechanical drawings based on transfer learning was presented. Firstly, the mechanical drawings were preprocessed, an improved threshold iterative algorithm was used to remove the background, noise and interference, and then the projections of mechanical drawings were obtained by segmentation operations. Secondly, by training the feature extractor of the source domain image, the network weight of the feature extractor was transferred to the mechanical drawing evaluation model and the knowledge transfer of similar domain was completed. Finally, the logistic regression classifier was trained and the intelligent evaluation model was established based on self-adjustment of neural network weight parameter. The standard elements of a mechanical drawing such as geometric feature, projection feature, line and section plane symbols were recognized. Experimental results show that the average proofreading time was 0.95 s, and the average judgement accuracy of engineering drawing was 98.82%. Compared with human proofreading of mechanical drawings, the proposed method has higher efficiency and accuracy.

Key words： intelligent proofreading      instrument tool drawing      transfer learning      mechanical drawing      machine learning     
Received: 08 June 2021      Published: 31 May 2022
CLC:  TN 911.73  
Fund:  国家自然科学基金资助项目(51975386)；浙江省自然科学基金资助项目(LS18G03006)；中国博士后科学基金资助项目（2021M690312）；浙江大学本科教学研究项目(zdjg21052)；湖州市自然科学基金资助项目(2019YZ09)
Corresponding Authors: Qiong LIN     E-mail: gaoyicong@zju.edu.cn;waglin@zjut.edu.cn
Service
E-mail this article
Add to my bookshelf
Add to citation manager
E-mail Alert
RSS
Articles by authors
Yi-cong GAO
Yan-kun WANG
Shao-mei FEI
Qiong LIN
Cite this article:

Yi-cong GAO,Yan-kun WANG,Shao-mei FEI,Qiong LIN. Intelligent proofreading method of engineering drawing based on transfer learning. Journal of ZheJiang University (Engineering Science), 2022, 56(5): 856-863, 889.

URL:

https://www.zjujournals.com/eng/10.3785/j.issn.1008-973X.2022.05.002     OR     https://www.zjujournals.com/eng/Y2022/V56/I5/856


基于迁移学习的机械制图智能评阅方法

针对机械图样几何特征种类多、线条线型易混淆、人工制图风格多样导致校对效率低、误检、漏检等问题，提出基于迁移学习的机械制图智能评阅方法. 对机械图样进行预处理，采用改进的阀值迭代算法去除背景、噪点和干扰，完成图样图像的分割，提取机械图样的特征投影图像. 通过训练源领域图片的特征提取器，将特征提取器的网络权值迁移到机械图样评阅模型中，完成相似领域的知识迁移. 训练逻辑回归分类器，建立基于神经网络权重参数自适应的智能评阅模型，对几何特征、投影特征、图线、剖面符号等机械图样的制图标准要素进行识别. 实验结果表明，所提出的机械制图智能评阅方法具有较高的错误识别率和鲁棒性能，单个测试样本平均评阅时间为0.95 s，机械图样的平均评阅正确率为98.82%；与人工评阅相比，所提方法能够在提高评阅效率的同时具有较高准确率.


关键词： 智能评阅,  仪器制图,  迁移学习,  机械图样,  机器学习 
Fig.1 Original image and preprocessing of mechanical drawings
Fig.2 Evaluation criteria of mechanical drawing
Fig.3 Feature segmentation of mechanical drawing
Fig.4 Example of data enhancement operation
Fig.5 Structure of DenseNet121 network
Fig.6 Flow chart of proposed intelligent proofreading method of engineering drawing based on transfer learning
Fig.7 Comparison of loss value and accuracy of transfer learning and non-transfer learning
得分
点编号 		ntra Acc/%
DenseNet121
（无迁移
学习） 		VGG16
（使用迁
移学习） 		ResNet18
（使用迁
移学习） 		DenseNet121
（使用迁
移学习）
1 80 81.11 95.56 94.44 94.44
150 88.10 95.56 97.78 97.78
2 80 77.78 90.85 89.63 91.64
150 81.25 97.48 93.71 98.11
3 80 75.00 82.89 76.32 84.21
150 75.68 83.67 86.49 89.19
4 80 94.32 95.45 93.18 97.73
150 95.83 95.83 97.22 98.61
5 80 88.89 95.15 96.12 95.15
150 93.07 97.03 97.03 97.03
6 80 91.21 96.04 94.06 98.90
150 97.80 100.00 100.00 100.00
7 80 82.91 91.45 90.60 94.02
150 84.42 98.29 97.44 97.44
Tab.1 Comparison of recognition results between non-transfer learning and transfer learning in four models
Fig.8 Thumbnails of review experiments
编号 实验内容 ns ntra ntes
1 线条补齐评阅 5 143 50
2 三视图绘制评阅 7 156 50
3 轴系判改评阅 10 152 50
Tab.2 Design of contrast experiments of manual proof reading and proposed intelligent method
编号 t/s Acc/%
人工 本研究方法 人工 本研究方法
1 224 35 100 98.40
2 298 50 100 98.86
3 472 58 100 99.20
平均值 331.3 47.7 100.0 98.82
Tab.3 Results of manual proofreading and proposed intelligent method
[1]   谭建荣, 张树有, 陆国栋. 图学基础教程[M]. 北京: 高等教育出版社, 2006.
[2]   费少梅, 陆国栋, 顾大强 时空融合知行耦合的机械大类课程教学新范式探索实践[J]. 高等工程教育研究, 2017, (6): 76- 80
FEI Shao-mei, LU Guo-dong, GU Da-qiang Exploration and practice of a new teaching paradigm of mechanical courses based on the integration of time and space, knowledge and practice[J]. Research on Higher Engineering Education, 2017, (6): 76- 80
[3]   SANCHEZ J, MONZON N, AGUSTIN S An analysis and implementation of the Harris corner detector[J]. Image Processing On Line, 2018, 8: 305- 328
doi: 10.5201/ipol.2018.229
[4]   冯毅雄, 李康杰, 高一聪, 等 面向视觉伺服的工业机器人轮廓曲线角点识别[J]. 浙江大学学报:工学版, 2020, 54 (8): 6- 13
FENG Yi-xiong, LI Kang-jie, GAO Yi-cong, et al Corner recognition of industrial robot contour curve for visual servo[J]. Journal of Zhejiang University: Engineering Science, 2020, 54 (8): 6- 13
[5]   冯毅雄, 李康杰, 高一聪, 等 基于特征与形貌重构的轴件表面缺陷检测方法[J]. 浙江大学学报:工学版, 2020, 54 (3): 427- 434
FENG Yi-xiong, LI Kang-jie, GAO Yi-cong, et al The method of surface defect detection of shaft parts based on feature and morphology reconstruction[J]. Journal of Zhejiang University: Engineering Science, 2020, 54 (3): 427- 434
[6]   KYEONG K, KIM H Classification of mixed-type defect patterns in wafer bin maps using convolutional neural networks[J]. IEEE Transactions on Semiconductor Manufacturing, 2018, 31 (3): 395- 402
doi: 10.1109/TSM.2018.2841416
[7]   LEE H, KIM Y, KIM C O A deep learning model for robust wafer fault monitoring with sensor measurement noise[J]. IEEE Transactions on Semiconductor Manufacturing, 2017, 30 (2): 23- 31
[8]   SUDHA S, VIDHYALAKSHMI M, PAVITHRA K, et al. An automatic classification method for environment: Friendly waste segregation using deep learning [C]// 2016 IEEE Technological Innovations in ICT for Agriculture and Rural Development (TIAR). Chennai: IEEE, 2016: 65-70.
[9]   DELATTE D M, CRITES S T, GUTTENBERG N, et al Segmentation convolutional neural networks for automatic crater detection on Mars[J]. IEEE Journal of Selected Topics in Applied Earth Observations and Remote Sensing, 2019, 12 (8): 2944- 2957
doi: 10.1109/JSTARS.2019.2918302
[10]   JIN Y, HE F, LIU S, et al. Small scale crater detection based on deep learning with multi-temporal samples of high-resolution images [C]// 2019 10th International Workshop on the Analysis of Multitemporal Remote Sensing Images. Shanghai: IEEE, 2019: 1-4.
[11]   PALESTRA G, PETTINICCHIO A. Improved performance in facial expression recognition using 32 geometric features [C]∥International Conference on Image Analysis and Processing. Genova: ICIAR, 2015: 518-528.
[12]   ZHANG Z, JAISWAL P, RAI R FeatureNet: machining feature recognition based on 3D convolution neural network[J]. Computer-Aided Design, 2018, 101: 12- 22
doi: 10.1016/j.cad.2018.03.006
[13]   NATARAJAN V, HUNG T Y, VAIKUNDAM S, et al. Convolutional networks for voting-based anomaly classification in metal surface inspection [C]// IEEE International Conference on Industrial Technology. Toronto: IEEE, 2017.
[14]   魏域君. 顾及几何与拓扑特征的深度学习遥感影像道路提取[D]. 武汉: 武汉大学, 2020.
WEI Yu-jun. Road extraction from remote sensed imagery based on deep learning considering geometric and topological properties [D]. Wuhan: Wuhan University, 2020.
[15]   韩丽, 朴京钰, 兰鹏燕, 等 结构感知深度学习的三维形状分类方法[J]. 计算机辅助设计与图形学学报, 2021, 33 (1): 29- 38
HAN Li, PIAO Jing-yu, LAN Peng-yan, et al 3D shape classification method based on shape-aware deep learning[J]. Journal of Computer-Aided Design and Computer Graphics, 2021, 33 (1): 29- 38
doi: 10.3724/SP.J.1089.2021.18280
[16]   李军军, 曹建农, 朱莹莹, 等 高分辨率遥感影像建筑区域局部几何特征提取[J]. 遥感学报, 2020, 24 (3): 233- 244
LI Jun-jun, CAO Jian-nong, ZHU Ying-ying, et al Built-up area detection from high resolution remote sensing images using geometric features[J]. Journal of Remote Sensing, 2020, 24 (3): 233- 244
[17]   SONG R, XIAO Z, LIN J, et al CIES: cloud-based intelligent evaluation service for video homework using CNN-LSTM network[J]. Journal of Cloud Computing, 2020, 9 (1): 1- 9
[18]   LIN J, ZHAO Y, LIU C, et al Abnormal video homework automatic detection system[J]. Journal of Ambient Intelligence and Humanized Computing, 2021, 12 (12): 10529- 10537
doi: 10.1007/s12652-020-02860-9
[19]   JING S, SANTOS O C, BOTICARIO J G, et al. Automatic grading of short answers for MOOC via semi-supervised document clustering [C]// International Conference on Educational Data Mining. Madrid: EDM, 2015: 554-555.
[20]   张旻. 智能图像识别在初中几何自动阅卷中的应用研究[D]. 北京: 北京工业大学, 2019.
ZHANG Min. Research of application for intelligent image recognition in junior high school geometry automatic marking[D]. Beijing: Beijing University of Technology, 2019.
[21]   李万秋. 机械基础类客观题及图形改判题阅卷系统的关键技术研究[D]. 广州: 华南理工大学, 2019.
LI Wan-qiu. Study on key techniques of marking system for objective questions and figure correction questions in basic mechanical courses[D]. Guangzhou: South China University of Technology, 2019.
[22]   沈银燕. 基于网络的建筑CAD考试系统的设计与实现[D]. 杭州: 浙江工业大学, 2009.
SHEN Yin-yan. The design and implementation of the examination system of architectural CAD based on web [D]. Hangzhou: Zhejiang University of Technology, 2009.
[23]   易琳. CAD教学中考试及自动评卷系统的研究[D]. 成都: 电子科技大学, 2007.
YI Lin. The research on the system of examination and automatic marking in cad teaching [D]. Chengdu: University of Electronic Science and Technology of China, 2007.
[24]   CANNY J A computational approach to edge detection[J]. IEEE Transactions on Pattern Analysis and Machine Intelligence, 1986, 8 (6): 679- 698
[25]   OTSU N A threshold selection method from gray-level histograms[J]. IEEE Transactions on Systems Man and Cybernetics, 2007, 9 (1): 62- 66
[26]   HUANG G, LIU Z, VAN D M, et al. Densely connected convolutional networks [C]// Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition. Honolulu: IEEE, 2017: 4700-4708.
[27]   SIMONYAN K, ZISSERMAN A Very deep convolutional networks for large-scale image recognition[J]. Computer Science, 2014, 34 (2): 1409- 1422
[1] Fa-ming HUANG,Li-han PAN,Chi YAO,Chuang-bing ZHOU,Qing-hui JIANG,Zhi-lu CHANG. Landslide susceptibility prediction modelling based on semi-supervised machine learning[J]. Journal of ZheJiang University (Engineering Science), 2021, 55(9): 1705-1713.
[2] Zhi-chao CHEN,Hai-ning JIAO,Jie YANG,Hua-fu ZENG. Garbage image classification algorithm based on improved MobileNet v2[J]. Journal of ZheJiang University (Engineering Science), 2021, 55(8): 1490-1499.
[3] Jia-hao REN,Hai-ou WANG,Jiang-kuan XING,Kun LUO,Jian-ren FAN. Lower-dimensional approximation models of tangential strain rate of turbulent flames[J]. Journal of ZheJiang University (Engineering Science), 2021, 55(6): 1128-1134.
[4] You ZHAN,Qiang LI,Xiao-tian MA,Chen-ping WANG,Yan-jun QIU. Macro and micro texture based prediction of pavement surface friction[J]. Journal of ZheJiang University (Engineering Science), 2021, 55(4): 684-694.
[5] Yong YU,Jing-yuan XUE,Sheng DAI,Qiang-wei BAO,Gang ZHAO. Quality prediction and process parameter optimization method for machining parts[J]. Journal of ZheJiang University (Engineering Science), 2021, 55(3): 441-447.
[6] Qiao-hong CHEN,YI CHEN,Wen-shu Li,Yu-bo JIA. Clothing image classification based on multi-scale SE-Xception[J]. Journal of ZheJiang University (Engineering Science), 2020, 54(9): 1727-1735.
[7] Zhuang KANG,Jie YANG,Hao-qi GUO. Automatic garbage classification system based on machine vision[J]. Journal of ZheJiang University (Engineering Science), 2020, 54(7): 1272-1280.
[8] Zong-li SHEN,Jian-bo YU. Wafer map defect recognition based on transfer learning and deep forest[J]. Journal of ZheJiang University (Engineering Science), 2020, 54(6): 1228-1239.
[9] Hui-fang WANG,Chen-yu ZHANG. Prediction of voltage stability margin in power system based on extreme gradient boosting algorithm[J]. Journal of ZheJiang University (Engineering Science), 2020, 54(3): 606-613.
[10] Le XIE,Xi-dan HENG,Yang LIU,Qi-long JIANG,Dong LIU. Transformer fault diagnosis based on linear discriminant analysis and step-by-step machine learning[J]. Journal of ZheJiang University (Engineering Science), 2020, 54(11): 2266-2272.
[11] Xiao-feng FU,Li NIU,Zhuo-qun HU,Jian-jun LI,Qing WU. Deep micro-expression spotting network training based on concept of transition frame[J]. Journal of ZheJiang University (Engineering Science), 2020, 54(11): 2128-2137.
[12] Zhi-yuan WAN,Jia-heng TAO,Jia-kun LIANG,Zhen-gong CAI,Cheng CHANG,Lin QIAO,Qiao-ni ZHOU. Large-scale empirical study on machine learning related questions on Stack Overflow[J]. Journal of ZheJiang University (Engineering Science), 2019, 53(5): 819-828.
[13] Dong-xiang KE,Li-min PAN,Sen-lin LUO,Han-qing ZHANG. Android malicious behavior recognition and classification method based on random forest algorithm[J]. Journal of ZheJiang University (Engineering Science), 2019, 53(10): 2013-2023.
[14] HU Li-sha, WANG Su-zhen, CHEN Yi-qiang, GAO Chen-long, HU Chun-yu, JIANG Xin-long, CHEN Zhen-yu, GAO Xing-yu. Fall detection algorithms based on wearable device: a review[J]. Journal of ZheJiang University (Engineering Science), 2018, 52(9): 1717-1728.
[15] WANG Hong-kai, CHEN Zhong-hua, ZHOU Zong-wei, LI Ying-ci, LU Pei-ou, WANG Wen-zhi, LIU Wan-yu, YU Li-juan. Evaluation of machine learning classifiers for diagnosing mediastinal lymph node metastasis of lung cancer from PET/CT images[J]. Journal of ZheJiang University (Engineering Science), 2018, 52(4): 788-797.