|
|
|
| Calligraphy generation algorithm based on improved generative adversarial network |
Yun-hong LI( ),Jiao-jiao DUAN,Xue-ping SU,Lei-tao ZHANG,Hui-kang YU,Xing-rui LIU |
| School of Electronics and Information, Xi’an Polytechnic University, Xi’an 710048, China |
|
|
|
Abstract An improved zi2zi generative adversarial network for generating calligraphic characters was proposed to solve the problems of missing strokes, wrong glyph structure, blurred images and poor quality in generating fonts by generative adversarial networks. The residual block with a convolution kernel of 1 was introduced into the encoder to improve the ability of the generator to extract detailed features of calligraphic fonts. The context-aware attention structure was increased to extract the stylistic features of calligraphic fonts. The spectral normalization was used in the discriminator to enhance model’s stability, which avoided pattern collapse due to unstable model training. The minimum absolute error L1 norm was used to constraint the font edge features, which made the font outline clearer. Then two styles of calligraphy characters were generated. The test results of the target style datasets of Yan Zhenqing Regular Script and Zhao Mengfu Running Script showed that the subjective and objective evaluation results of the proposed algorithm were better than the comparison algorithm compared with zi2zi. The peak signal-to-noise ratio was increased by 1.58 and 1.76 dB respectively, the structural similarity was increased by 5.66% and 6.91% respectively, and the perceptual similarity was reduced by 4.21% and 6.20% respectively.
|
|
Received: 02 August 2022
Published: 17 July 2023
|
|
|
| Fund: 国家自然科学基金资助项目（61902301）；陕西省科技厅自然科学基础研究重点资助项目（2022JZ-35） |
基于改进生成对抗网络的书法字生成算法
针对生成对抗网络生成字体存在笔画缺失、字形结构错乱、图像模糊与质量差的问题，提出改进zi2zi生成对抗网络的书法字生成算法. 在编码器中引入卷积核为1的残差块，提高生成器提取书法字体细节特征的能力，通过增加上下文感知注意力结构提取书法字体的风格特征. 在判别器中利用谱归一化增强模型的稳定性，避免因模型训练不稳定而带来的模式崩塌. 采用最小绝对误差L1范数约束生成字体边缘特征，使得字体轮廓更加清晰，最终生成2种风格的书法字. 颜真卿楷书与赵孟頫行书目标风格数据集的测试结果表明，提出算法的主观客观评价结果均优于对比算法，与zi2zi相比，峰值信噪比分别提高了1.58、1.76 dB，结构相似性分别提高了5.66%、6.91%，感知相似性分别降低了4.21%、6.20%.
关键词：
书法字生成,
深度学习,
生成对抗网络,
上下文感知注意力,
边缘损失
|
|
| [1] |
程若然, 赵晓丽, 周浩军, 等 基于深度学习的中文字体风格转换研究综述[J]. 浙江大学学报: 工学版, 2022, 56 (3): 510- 519
CHENG Ruo-ran, ZHAO Xiao-li, ZHOU Hao-jun, et al Review of Chinese font style conversion based on deep learning[J]. Journal of Zhejiang University: Engineering Science, 2022, 56 (3): 510- 519
|
|
|
| [2] |
曾锦山, 陈琪, 王明文 基于田字格变换的自监督汉字字体生成[J]. 中国科学: 信息科学, 2022, 52 (1): 145- 159
ZENG Jin-shan, CHEN Qi, WANG Ming-wen Self-supervised font generation of Chinese characters based on Tian Zige transformation[J]. Science of China: Information Science, 2022, 52 (1): 145- 159
|
|
|
| [3] |
黄子君, 陈琪, 罗文兵 基于深度学习的汉字生成方法[J]. 计算机工程与应用, 2021, 57 (17): 29- 36
HUANG Zi-jun, CHEN Qi, LUO Wen-bing Chinese character generation method based on deep learning[J]. Computer Engineering and Application, 2021, 57 (17): 29- 36
doi: 10.3778/j.issn.1002-8331.2103-0297
|
|
|
| [4] |
TIAN Y. Rewrite [CP/OL]. [2022-08-02]. https://github.com/kaonashi-tyc/Rewrite.
|
|
|
| [5] |
ISOLA P, ZHU J Y, ZHOU T, et al. Image-to-image translation with conditional adversarial networks [C]// Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition. Los Alamitos: IEEE, 2017: 5967-5976.
|
|
|
| [6] |
TIAN Y. Zi2zi [CP/OL]. [2022-08-02]. https://github.com/kaonashi-tyc/zi2zi.
|
|
|
| [7] |
JIANG Y, LIAN Z, TANG Y, et al. DCFont: an end-to-end deep Chinese font generation system [C]// SIGGRAPH Asia 2017 Technical Briefs. New York: ACM, 2017: 1-4.
|
|
|
| [8] |
CHANG B, ZHANG Q, PAN S, et al. Generating handwritten Chinese characters using CycleGAN [C]// 2018 IEEE Winter Conference on Applications of Computer Vision. Lake Tahoe: IEEE, 2018: 199-207.
|
|
|
| [9] |
ZHU J Y, PARK T, ISOLA P, et al. Unpaired image-to-image translation using cycle-consistent adversarial networks [C]// 2017 IEEE International Conference on Computer Vision. Venice: IEEE, 2017: 2242-2251.
|
|
|
| [10] |
REN C, LYU S, ZHAN H, et al SAFont: automatic font synthesis using self-attention mechanisms[J]. Australia Journal of Intelligent Information Processing Systems, 2019, 16 (2): 19- 25
|
|
|
| [11] |
WU S J, YANG C Y, HSU J Y. CalliGAN: style and structure-aware Chinese calligraphy character generator [EB/OL]. (2020-05-26) [2022-08-02]. https://arxiv.org/abs/2005.12500.
|
|
|
| [12] |
PARK S, CHUN S, CHA J, et al. Few-shot font generation with localized style representations and factorization [C]// Proceedings of the AAAI Conference on Artificial Intelligence. Vancouver: AAAI, 2021, 35(3): 2393-2402.
|
|
|
| [13] |
XIE Y, CHEN X, SUN L, et al. DGFont: deformable generative networks for unsupervised font generation [C]// Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition. Nashville: IEEE, 2021: 5130-5140.
|
|
|
| [14] |
PARK S, CHUN S, CHA J, et al. Multiple heads are better than one: few-shot font generation with multiple localized experts [C]// Proceedings of the IEEE/CVF International Conference on Computer Vision. Montreal: IEEE, 2021: 13900-13909.
|
|
|
| [15] |
KONG Y, LUO C, MA W, et al. Look closer to supervise better: one-shot font generation via component-based discriminator [C]// Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition. New Orleans: IEEE, 2022: 13482-13491.
|
|
|
| [16] |
ODENA A, OLAH C, SHLENS J. Conditional image synthesis with auxiliary classifier GANs [C]// Proceedings of the 34th International Conference on Machine Learning. Sydney: ACM, 2017, 70: 2642–2651.
|
|
|
| [17] |
TAIGMAN Y, POLYAK A, WOLF L. Unsupervised cross-domain image generation [EB/OL]. (2016-10-07) [2022-08-02]. https://arxiv.org/abs/1611.02200.
|
|
|
| [18] |
WANG Z, BOVIK A C, SHEIKH H R, et al Image quality assessment: from error visibility to structural similarity[J]. IEEE Transactions on Image Processing, 2004, 13 (4): 600- 612
doi: 10.1109/TIP.2003.819861
|
|
|
| [19] |
ZHANG R, ISOLA P, EFROS A A, et al. The unreasonable effectiveness of deep features as a perceptual metric [C]// Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition. Los Alamitos: IEEE, 2018: 586-595.
|
|
|
|
Viewed |
|
|
|
Full text
|
|
|
|
|
Abstract
|
|
|
|
|
Cited |
|
|
|
|
| |
Shared |
|
|
|
|
| |
Discussed |
|
|
|
|
