Please wait a minute...
浙江大学学报(工学版)
浙江大学学报(工学版)  2023, Vol. 57 Issue (10): 1998-2010    DOI: 10.3785/j.issn.1008-973X.2023.10.009
计算机技术、自动化技术     
基于改进生成对抗网络的图像数据增强方法
詹燕(),胡蝶,汤洪涛,鲁建厦,谭健,刘长睿
浙江工业大学 机械工程学院，浙江 杭州 310023
Image data enhancement method based on improved generative adversarial network
Yan ZHAN(),Die HU,Hong-tao TANG,Jian-sha LU,Jian TAN,Chang-rui LIU
College of Mechanical Engineering, Zhejiang University of Technology, Hangzhou 310023, China
 全文: PDF(5142 KB)   HTML
摘要：

为了提高机器学习模型的精确度，提出基于数据分布拟合、生成式对抗神经网络和图像超分辨率重建的图像数据增强方法. 该方法将最大似然估计和采样算法生成的符合原始数据分布的二维噪声用于对抗训练，克服了在生成模型中传统图像噪声输入随意的问题；采用逐层训练方式生成高分辨率图像，改进高分辨率图像映射困难、参数冗余的缺点. 以轴承滚子表面灰度图像数据增强为例，验证所提方法的有效性. 研究结果表明，所提方法生成的图像质量更优，相比传统方法生成的图像峰值信噪比提高13.07%，结构相似性提高32.40%，弗雷歇初始距离降低37.58%，且数据增强后的模型平均精确度提升7.89%.
关键词： 图像数据增强;  分布拟合;  采样算法;  生成式对抗网络;  图像超分辨率重建    
Abstract:

An image data enhancement method based on data distribution fitting, generative adversarial neural network and image super-resolution reconstruction was proposed to improve the accuracy of machine learning model. The maximum likelihood estimation and sampling algorithm were used to generate two-dimensional noise conforming to the original data distribution for counter-training. The problem of random noise input in traditional image generation models was overcome. Layer by layer training method was used to generate high-resolution images to correct the shortcomings of difficult mapping to high-resolution images. The effectiveness of the proposed method was verified by taking the gray image data enhancement of bearing roller surface. The results showed that the image quality generated by the proposed method was superior. Compared with the image generated by the traditional method, the image peak signal-to-noise ratio was increased by 13.07%, the structural similarity was increased by 32.40%, Fréchet inception distance was reduced by 37.58%, and the average accuracy of the model after data enhancement was increased by 7.89%.
Key words: image data enhancement    distribution fitting    sampling algorithm    generative adversarial neural network    image super-resolution reconstruction
收稿日期: 2022-12-13 出版日期: 2023-10-18
CLC:  TP 391.4  
基金资助: 浙江省科技计划资助项目[重点研发（尖兵）项目]（2023C01063）
作者简介: 詹燕（1976—），女，副教授，从事图像处理及智能制造研究. orcid.org/0000-0002-6861-8005. E-mail： yzhan@zjut.edu.cn
服务　　
把本文推荐给朋友
加入引用管理器
E-mail Alert
作者相关文章　　
詹燕
胡蝶
汤洪涛
鲁建厦
谭健
刘长睿

引用本文:

詹燕,胡蝶,汤洪涛,鲁建厦,谭健,刘长睿. 基于改进生成对抗网络的图像数据增强方法[J]. 浙江大学学报(工学版), 2023, 57(10): 1998-2010.

Yan ZHAN,Die HU,Hong-tao TANG,Jian-sha LU,Jian TAN,Chang-rui LIU. Image data enhancement method based on improved generative adversarial network. Journal of ZheJiang University (Engineering Science), 2023, 57(10): 1998-2010.

链接本文:

https://www.zjujournals.com/eng/CN/10.3785/j.issn.1008-973X.2023.10.009    或    https://www.zjujournals.com/eng/CN/Y2023/V57/I10/1998

图 1  基于传统GAN的图像数据增强方法
图 2  基于生成对抗神经网络超分辨率重建的图像数据增强方法流程图
图 3  生成随机噪声流程图
图 4  Metropolis-Hastings抽样算法流程图
图 5  WGAN-GP网络架构图
图 6  WGAN-GP+SR模型整体结构图
图 7  降采样后图像数据及其灰度直方图
拟合分布名称 估计参数 SSE
α β ε
卡方分布 2.848 45.687 2.848 0.004 551
伽马分布 ?20.172 8.656 9.378 0.004 411
高斯分布 65.852 31.368 0 0.004 014
T分布 66.366 17.124 2.734 0.002 338
拉普拉斯分布 68.000 19.693 0 0.002 049
韦布尔分布 69.000 20.150 0.926 0.002 051
柯西分布 69.070 11.861 0 0.002 038
表 1  轴承滚子端面灰度直方图拟合结果
图 8  轴承滚子端面灰度直方图拟合效果
图 9  轴承滚子各表面灰度直方图的最优分布拟合效果
图像名称 估计参数 SSE
α β ε
倒角面 8.305 0.475 2.000 0.046284
侧面（峰1） 8.360 4.882 0 0.007293
侧面（峰2） 141.441 21.260 0 0.004029
侧面（峰3） 254.950 1.754 0 0.023585
表 2  轴承滚子倒角面及侧面灰度直方图的拟合结果
图 10  Box-Muller变换生成的端面一维随机噪声分布
图 11  MH采样算法采样结果
图 12  采样数据分布与原始数据分布的对比图
图 13  各模型输出结果对比图
图 14  CGAN损失函数值变化曲线图
图 15  WGAN-GP损失函数值变化曲线图
图 16  SRCNN损失函数值变化曲线图
模型 PSNR/dB SSIM FID
侧面 倒角面 端面 侧面 倒角面 端面 侧面 倒角面 端面
CGAN 48.043 3 54.327 9 52.459 2 0.184 2 0.288 9 0.248 0 472.899 4 343.297 9 372.268 5
WGAN-GP 57.523 0 62.487 9 53.688 9 0.634 2 0.831 7 0.396 4 363.993 2 166.245 8 207.444 2
WGAN-GP+SR 63.543 6 68.508 5 64.347 9 0.807 0 0.941 1 0.717 6 224.5837 132.936 6 102.986 2
表 3  各模型生成图像质量评价表
图 17  不同标准差噪声及对应生成图像
噪声标准差 $\sigma $ PSNR/dB SSIM FID
0.168 6 56.580 9 0.599 2 312.831 4
0.215 5 56.375 6 0.371 3 406.542 9
0.784 0 55.263 4 0.165 7 430.592 1
表 4  各模型生成图像质量评价表
图 18  轴承滚子数据集不同采样方式训练过程FID曲线图
图 19  WGAN-GP+SR与WGAN-GP图像对比
图 20  Yolov5目标检测模型训练过程MAP曲线图
图 21  轴承滚子缺陷检测对比
1 康守强, 胡明武, 王玉静, 等 基于特征迁移学习的变工况下滚动轴承故障诊断方法[J]. 中国电机工程学报, 2019, 39 (3): 764- 772
KANG Shou-qiang, HU Ming-wu, WANG Yu-jing, et al Fault diagnosis method of a rolling bearing under variable working conditions based on feature transfer learning[J]. Proceedings of the CSEE, 2019, 39 (3): 764- 772
doi: 10.13334/J.0258-8013.PCSEE.180130
2 肖雄, 肖宇雄, 张勇军, 等 基于二维灰度图的数据增强方法在电机轴承故障诊断的应用研究[J]. 中国电机工程学报, 2021, 41 (2): 738- 749
XIAO Xiong, XIAO Yu-xiong, ZHANG Yong-jun, et al Research on the application of the data augmentation method based on 2D gray pixel images in the fault diagnosis of motor bearing[J]. Proceedings of the CSEE, 2021, 41 (2): 738- 749
doi: 10.13334/j.0258-8013.pcsee.200834
3 PAN S J, YANG Q A survey on transfer learning[J]. IEEE Transactions on Knowledge and Data Engineering, 2009, 22 (10): 1345- 1359
4 SALAKHUTDINOV R, LAROCHELLE H. Efficient learning of deep Boltzmann machines [C]// Proceedings of the 13th International Conference on Artificial Intelligence and Statistics. Sardinia: JMLR, 2010: 693-700.
5 BENGIO Y, LAUFER E, ALAIN G, et al. Deep generative stochastic networks trainable by backprop [C]// International Conference on Machine Learning. Beijing: PMLR, 2014: 226-234.
6 KINGMA D P, WELLING M. Auto-encoding variational bayes [EB/OL]. [2013-12-20]. https://arxiv.org/abs/1312.6114.
7 VANDEN OORD A, KALCHBRENNER N, ESPEHOLT L, et al. Conditional image generation with pixel-cnn decoders [C]// 30th Conference on Neural Information Processing Systems. Barcelona: CA, 2016: 4797-4805.
8 VAN OORD A, KALCHBRENNER N, KAVUKCUOGLU K. Pixel recurrent neural networks [C]// International Conference on Machine Learning. New York: JMLR, 2016: 1747-1756.
9 GOODFELLOW I, POUGET-ABADIE J, MIRZA M, et al Generative adversarial networks[J]. Communications of the ACM, 2020, 63 (11): 139- 144
doi: 10.1145/3422622
10 MIRZA M, OSINDERO S. Conditional generative adversarial nets [EB/OL]. [2014-11-06]. https://arxiv.org/abs/1411.1784.
11 RADFORD A, METZ L, CHINTALA S. Unsupervised representation learning with deep convolutional generative adversarial networks [EB/OL]. [2015-11-19]. https://arxiv.org/abs/1511.06434.
12 ARJOVSKY M, CHINTALA S, BOTTOU L. Wasserstein generative adversarial networks [C]// International Conference on Machine Learning. Sydney: PMLR, 2017: 214-223.
13 GULRAJANI I, AHMED F, ARJOVSKY M, et al. Improved training of wasserstein gans [EB/OL]. [2017-03-31]. https://arxiv.org/abs/1704.00028.
14 BERTHELOT D, SCHUMM T, METZ L. Boundary equilibrium generative adversarial networks [EB/OL]. [2017-03-21]. https://arxiv.org/abs/1703.10717.
15 KARRAS T, AILA T, LAINE S, et al. Progressive growing of gans for improved quality, stability, and variation [C]// International Conference on Learning Representations. Vancouver: JMLR, 2018: 26.
16 KARRAS T, LAINE S, AILA T. A style-based generator architecture for generative adversarial networks [C]// CVF Conference on Computer Vision and Pattern Recognition. Long Beach: IEEE, 2019: 4396-4405.
17 KARRAS T, LAINE S, AITTALA M, et al. Analyzing and improving the image quality of style GAN [C]// CVF Conference on Computer Vision and Pattern Recognition. Seattle: IEEE, 2020: 8107-8116.
18 KARRAS T, AITTALA M, LAINE S, et al Alias-free generative adversarial networks[J]. Advances in Neural Information Processing Systems, 2021, 34: 852- 863
19 DONG C, LOY C C, HE K, et al Image super-resolution using deep convolutional networks[J]. IEEE Transactions on Pattern Analysis and Machine Intelligence, 2015, 38 (2): 295- 307
20 DEON A F, MENYAEV Y A Twister generator of random normal numbers by box-muller model[J]. International Journal of Trends in Computer Science, 2020, 16 (1): 1- 13
21 何新林, 戚宗锋, 李建勋 基于隐变量后验生成对抗网络的不平衡学习[J]. 上海交通大学学报, 2021, 55 (5): 557- 565
HE Xin-lin, QI Zong-feng, LI Jian-xun Unbalanced learning of generative adversarial network based on latent posterior[J]. Journal of Shanghai Jiaotong University, 2021, 55 (5): 557- 565
doi: 10.16183/j.cnki.jsjtu.2019.264
22 ARJOVSKY M, BOTTOU L. Towards principled methods for training generative adversarial networks [EB/OL]. [2017-01-17]. https://arxiv.org/abs/1701.04862.
[1] 吕鑫栋,李娇,邓真楠,冯浩,崔欣桐,邓红霞. 基于改进Transformer的结构化图像超分辨网络[J]. 浙江大学学报(工学版), 2023, 57(5): 865-874.
[2] 温佩芝,陈君谋,肖雁南,温雅媛,黄文明. 基于生成式对抗网络和多级小波包卷积网络的水下图像增强算法[J]. 浙江大学学报(工学版), 2022, 56(2): 213-224.
[3] 王万良,杨胜兰,赵燕伟,李卓蓉. 基于条件边界平衡生成对抗网络的河流表面流速估测[J]. 浙江大学学报(工学版), 2019, 53(11): 2118-2128.