|
|
Fast adversarial training method based on discrete cosine transform |
Xiaomiao WANG1( ),Yujin ZHANG1,*( ),Tao ZHANG2,Jin TIAN1,Fei WU1 |
1. School of Electronic and Electrical Engineering, Shanghai University of Engineering Science, Shanghai 201620, China 2. School of Computer Science and Engineering, Changshu Institute of Technology, Changshu 215500, China |
|
|
Abstract A fast adversarial training method based on discrete cosine transform (DCT) was proposed from the perspective of the frequency domain in order to enhance the robustness of deep neural network. An adversarial initialization generation module was introduced, which adaptively generated initialization information based on the system’s robustness, allowing for more accurate capture of image features and effectively avoiding catastrophic overfitting. Random spectral transformations were applied to the samples, transforming them from the spatial domain to the frequency domain, which improved the model’s transferability and generalization ability by controlling spectral saliency. The effectiveness of the proposed method was validated on the CIFAR-10 and CIFAR-100 datasets. The experimental results show that the robust accuracy of the proposed method on CIFAR-10 improved by 2% to 9% compared to existing methods, and improved by 1% to 9% on CIFAR-100 by using ResNet18 as the target network and facing PGD-10 attacks. Similar effects were achieved when facing PGD-20, PGD-50, C&W and other attacks, as well as when applied to more complex model architectures. The proposed method not only avoids catastrophic overfitting but also effectively enhances system robustness.
|
Received: 03 July 2023
Published: 23 October 2024
|
|
Fund: 国家自然科学基金资助项目(62072057);上海市自然科学基金资助项目(17ZR1411900);中国高校产学研创新基金资助项目(2021ZYB01003). |
Corresponding Authors:
Yujin ZHANG
E-mail: m320121342@sues.edu.cn;yjzhang@sues.edu.cn
|
基于离散余弦变换的快速对抗训练方法
为了提升深度神经网络的鲁棒性,从频域的角度提出基于离散余弦变换(DCT)的快速对抗训练方法. 引入对抗初始化生成模块,根据系统的鲁棒性自适应地生成初始化信息,可以更精准地捕捉到图像特征,有效避免灾难性过拟合. 对样本进行随机谱变换,将样本从空间域变换至频谱域,通过控制频谱显著性提高模型的迁移与泛化能力. 在CIFAR-10与CIFAR-100数据集上验证提出方法的有效性. 实验结果表明,在以ResNet18为目标网络,面对PGD-10攻击时,本文方法在CIFAR-10上的鲁棒精度较现有方法提升了2%~9%,在CIFAR-100上提升了1%~9%. 在面对PGD-20、PGD-50、C&W等其他攻击以及架构更复杂的模型时,均取得了类似的效果. 提出方法在避免灾难性过拟合现象的同时,有效提高了系统的鲁棒性.
关键词:
对抗样本,
快速对抗训练,
离散余弦变换(DCT),
鲁棒性,
样本初始化
|
|
[1] |
金鑫, 庄建军, 徐子恒 轻量化YOLOv5s网络车底危险物识别算法[J]. 浙江大学学报: 工学版, 2023, 57 (8): 1516- 1526 JIN Xin, ZHUANG Jianjun, XU Ziheng Lightweight YOLOv5s network-based algorithm for identifying hazardous objects under vehicles[J]. Journal of Zhejiang University: Engineering Science, 2023, 57 (8): 1516- 1526
|
|
|
[2] |
熊帆, 陈田, 卞佰成, 等 基于卷积循环神经网络的芯片表面字符识别[J]. 浙江大学学报: 工学版, 2023, 57 (5): 948- 956 XIONG Fan, CHEN Tian, BIAN Baicheng, et al Chip surface character recognition based on convolutional recurrent neural network[J]. Journal of Zhejiang University: Engineering Science, 2023, 57 (5): 948- 956
|
|
|
[3] |
刘春娟, 乔泽, 闫浩文, 等 基于多尺度互注意力的遥感图像语义分割网络[J]. 浙江大学学报: 工学版, 2023, 57 (7): 1335- 1344 LIU Chunjuan, QIAO Ze, YAN Haowen, et al Semantic segmentation network for remote sensing image based on multi-scale mutual attention[J]. Journal of Zhejiang University: Engineering Science, 2023, 57 (7): 1335- 1344
|
|
|
[4] |
杨长春, 叶赞挺, 刘半藤, 等 基于多源信息融合的医学图像分割方法[J]. 浙江大学学报: 工学版, 2023, 57 (2): 226- 234 YANG Changchun, YE Zanting, LIU Banteng, et al Medical image segmentation method based on multi-source information fusion[J]. Journal of Zhejiang University: Engineering Science, 2023, 57 (2): 226- 234
|
|
|
[5] |
宋秀兰, 董兆航, 单杭冠, 等 基于时空融合的多头注意力车辆轨迹预测[J]. 浙江大学学报: 工学版, 2023, 57 (8): 1636- 1643 SONG Xiulan, DONG Zhaohang, SHAN Hangguan, et al Vehicle trajectory prediction based on temporal-spatial multi-head attention mechanism[J]. Journal of Zhejiang University: Engineering Science, 2023, 57 (8): 1636- 1643
|
|
|
[6] |
SZEGEDY C, ZAREMBA W, SUTSKEVER I, et al. Intriguing properties of neural networks [C]// 2nd International Conference on Learning Representations. Banff: [s. n. ], 2014.
|
|
|
[7] |
MADRY A, MAKELOV A, SCHMIDT L, et al. Towards deep learning models resistant to adversarial attacks [C]// International Conference on Learning Representations. Vancouver: [s. n.], 2018.
|
|
|
[8] |
WANG Y, MA X, BAILEY J, et al. On the convergence and robustness of adversarial training [C]// International Conference on Machine Learning . Long Beach: International Machine Learning Society, 2019: 6586-6595.
|
|
|
[9] |
GOODFELLOW J, SHLENS J, SZEGEDY C. Explaining and harnessing adversarial examples [C]// International Conference on Learning Representation . San Diego: [s. n.], 2015.
|
|
|
[10] |
WONG E, RICE L, KOLTER J. Z. Fast is better than free: revisiting adversarial training [C]// International Conference on Learning Representations . Addis Ababa, Ethiopia: [s. n.], 2020.
|
|
|
[11] |
ANDRIUSHCHENKO M, FLAMMARION N. Understanding and improving fast adversarial training [C]// Neural Information Processing Systems . [S. l. ]: Curran Associates, Inc, 2020: 16048-16059.
|
|
|
[12] |
KIM H, LEE W, LEE J. Understanding catastrophic overfitting in single-step adversarial training [C]// Proceedings of the AAAI Conference on Artificial Intelligence . Vancouver: AAAI Press, 2021: 8119-8127.
|
|
|
[13] |
SHAFAHI A, NAJIBI M, GHIASI A, et al. Adversarial training for free! [C]// Neural Information Processing Systems . Vancouver: Curran Associates, Inc. , 2019: 3353-3364.
|
|
|
[14] |
SRIRAMANAN G, ADDEPALLI S, BABURAJ A, et al. Towards efficient and effective adversarial training [C]// Neural Information Processing Systems . [S. l. ]: Curran Associates, Inc. , 2021: 11821-11833.
|
|
|
[15] |
IOFFE S, SZEGEDY C. Batch normalization: accelerating deep network training by reducing internal covariate shift [C]// International Conference on Machine Learning . Lille: MIT Press, 2015: 448-456.
|
|
|
[16] |
AGARAP F. Deep learning using rectified linear units (ReLU) [EB/OL]. [2023-06-20]. https://arxiv.org/abs/1803.08375.
|
|
|
[17] |
MIYATO T, KATAOKA T, KOYAMAM M, et al. Spectral normalization for generative adversarial networks [C]// International Conference on Learning Representations . Vancouver: [s. n. ], 2018.
|
|
|
[18] |
WANG H, WU X, HUANG Z, et al. High-frequency component helps explain the generalization of convolutional neural networks [C]// IEEE Conference on Computer Vision and Pattern Recognition . Seattle: IEEE, 2020: 8681–8691.
|
|
|
[19] |
AHMED N, NATARAJAN T, RAO K R. Discrete cosine transform[J]. IEEE Transactions on Computers, 1974, 23 (1): 90- 93
|
|
|
[20] |
SELVARAJU R. R. COGSWELL M, DAS A, et al. Grad-CAM: visual explanations from deep networks via gradient-based localization [C]// IEEE International Conference on Computer Vision. Venice: IEEE, 2017: 618-626.
|
|
|
[21] |
KRIZHEVSKY A, HINTON G. Learning multiple layers of features from tiny images [D]. Toronto: University of Toronto, 2009.
|
|
|
[22] |
CARLINI N, WAGNER D. A. Towards evaluating the robustness of neural networks [C]// IEEE Symposium on Security and Privacy . San Jose: IEEE, 2017: 39–57.
|
|
|
[23] |
REBUFFI S A, GOWAL S, CALIAN D A, et al. Fixing data augmentation to improve adversarial robustness [EB/OL]. [2023-06-20]. https://arxiv.org/abs/2103.01946.
|
|
|
[24] |
HO J, JAIN A, ABBEEL P. Denoising diffusion probabilistic models [C]// Advances in Neural Information Processing Systems . [S. l. ]: Curran Associates, Inc. , 2020: 6840-6851.
|
|
|
[25] |
ZAGORUYKO S, KOMODAKIS N. Wide residual networks [C]// Proceedings of the British Machine Vision Conference . York: BMVA Press, 2016: 87.1-87.12.
|
|
|
[26] |
HE K, ZHANG X, REN S, et al. Deep residual learning for image recognition [C]// IEEE Conference on Computer Vision and Pattern Recognition . Las Vegas: IEEE Computer Society, 2016: 770–778.
|
|
|
[27] |
HE K, ZHANG X, REN S, et al. Identity mappings in deep residual networks [C]// European Conference on Computer Vision . Amsterdam: Springer Verlag, 2016: 630–645.
|
|
|
|
Viewed |
|
|
|
Full text
|
|
|
|
|
Abstract
|
|
|
|
|
Cited |
|
|
|
|
|
Shared |
|
|
|
|
|
Discussed |
|
|
|
|