|
|
|
| Image semantic communication based on CSI-guided attention and gated generation repair |
Yi PENG( ),Heng LI,Qingqing YANG*( ),Hui LI,Jianming WANG |
| Faculty of Information Engineering and Automation, Kunming University of Science and Technology, Kunming 650031, China |
|
|
|
Abstract A Swin Transformer-based image semantic communication scheme that integrated channel state information (CSI) awareness with generative adversarial network (GAN) assistance was proposed aiming at the issues of texture loss and edge blurring in reconstructed images encountered by existing deep joint source-channel coding schemes for image semantic communication in order to improve the perceptual quality and robustness of image transmission under typical SNR conditions. A CSI-guided self-attention mechanism was designed at the encoder. A scalar CSI was mapped into an additive bias matrix through a multilayer perceptron and incorporated into the self-attention computation in order to dynamically reshape the feature extraction topology. A CSI-gated generative restoration module was developed at the decoder. CSI was nonlinearly mapped to generate a gating factor for dynamically adjusting the restoration weight of the GAN on residual features. The experimental results demonstrate that the proposed scheme achieves superior peak signal-to-noise ratio (PSNR) and multi-scale structural similarity (MS-SSIM) performance across datasets with different resolutions compared with conventional separated source-channel coding and existing mainstream semantic communication schemes. The proposed method provides an effective solution for image semantic transmission in complex channel environments.
|
|
Received: 04 December 2025
Published: 21 October 2026
|
|
|
| Fund: 国家自然科学基金资助项目(62461030);云南省基础研究重点资助项目(202401AS070105);中央引导地方科技发展专项资金资助项目(202607AD040003). |
|
Corresponding Authors:
Qingqing YANG
E-mail: 12309214@kust.edu.cn;20090119@kust.edu.cn
|
基于CSI导向注意力与门控生成修复的图像语义通信
针对现有深度联合信源信道编码方案在图像语义通信中存在重建纹理丢失与边缘模糊的问题,提出融合信道状态信息(CSI)感知与生成对抗网络(GAN)辅助的Swin Transformer图像语义通信方案,以提升典型信噪比下图像传输的感知质量与鲁棒性. 在编码端,设计CSI导向自注意力机制,通过多层感知机将标量CSI映射为加性偏置矩阵并注入到自注意力计算中,动态重塑特征提取拓扑. 在解码端,构建CSI门控生成式修复模块,利用CSI经过非线性映射生成门控因子,自适应调节GAN对残差特征的修复权重. 实验表明,所提方案在不同分辨率数据集上的峰值信噪比(PSNR)与多尺度结构相似性(MS-SSIM)指标均优于传统分离式编码及现有主流语义通信方案,为复杂信道环境下的图像语义传输提供了有效的解决方案.
关键词:
语义通信,
Swin Transformer,
信道状态信息,
自适应调节,
生成对抗网络
|
|
| [1] |
Tong W, Zhu P. 6G: the next horizon [M]. Cambridge: Cambridge University Press, 2021.
|
|
|
| [2] |
Letaief K B, Chen Wei, Shi Yuanming, et al The roadmap to 6G: AI empowered wireless networks[J]. IEEE Communications Magazine, 2019, 57 (8): 84- 90
doi: 10.1109/MCOM.2019.1900271
|
|
|
| [3] |
Wang Y, Han H, Feng Y, et al Semantic communication empowered 6g networks: techniques, applications, and challenges[J]. IEEE Access, 2025, 13: 28293- 28314
doi: 10.1109/ACCESS.2025.3532797
|
|
|
| [4] |
栾宏之, 李书才, 李屹, 等 5G-A车联网确定性网络中的语义通信系统[J]. 浙江大学学报(工学版), 2025, 59 (5): 964- 972 Luan Hongzhi, Li Shucai, Li Yi, et al Semantic communication system in deterministic networking for5G-advanced vehicular network[J]. Journal of Zhejiang University (Engineering Science), 2025, 59 (5): 964- 972
|
|
|
| [5] |
Shannon C E A mathematical theory of communication[J]. The Bell System Technical Journal, 1948, 27 (3): 379- 423
doi: 10.1002/j.1538-7305.1948.tb01338.x
|
|
|
| [6] |
Zhang P, Xu W, Gao H, et al Toward wisdom-evolutionary and primitive-concise 6G: a new paradigm of semantic communication networks[J]. Engineering, 2022, 8: 60- 73
doi: 10.1016/j.eng.2021.11.003
|
|
|
| [7] |
Fang Y, He Q, Ding D, et al. Deep-learning based gaussian mixture likelihoods joint source-channel coding for image transmission [C]//2023 International Conference on Wireless Communications and Signal Processing. Hangzhou: IEEE, 2023: 19-24.
|
|
|
| [8] |
Gündüz D, Qin Z, Aguerri I E, et al Beyond transmitting bits: context, semantics, and task-oriented communications[J]. IEEE Journal on Selected Areas in Communications, 2023, 41 (1): 5- 41
doi: 10.1109/JSAC.2022.3223408
|
|
|
| [9] |
Bourtsoulatze E, Kurka D B, Gündüz D Deep joint source-channel coding for wireless image transmission[J]. IEEE Transactions on Cognitive Communications and Networking, 2019, 5 (3): 567- 579
doi: 10.1109/TCCN.2019.2919300
|
|
|
| [10] |
Dai J, Wang S, Tan K, et al Nonlinear transform source-channel coding for semantic communications[J]. IEEE Journal on Selected Areas in Communications, 2022, 40 (8): 2300- 2316
doi: 10.1109/JSAC.2022.3180802
|
|
|
| [11] |
Park T, Hong E, Jeon Y, et al. Robust deep joint source channel coding for task-oriented semantic communications [PP/OL]. arXiv (2025-03-17)[2025-11-15]. https://arxiv.org/abs/2503.12907.
|
|
|
| [12] |
Hu Q, Zhang G, Qin Z, et al Robust semantic communications with masked VQ-VAE enabled codebook[J]. IEEE Transactions on Communications, 2023, 22 (12): 8707- 8722
|
|
|
| [13] |
Kurka D B, Gündüz D DeepJSCC-f: deep joint source-channel coding of images with feedback[J]. IEEE Journal on Selected Areas in Information Theory, 2020, 1 (1): 178- 193
doi: 10.1109/JSAIT.2020.2987203
|
|
|
| [14] |
Dosovitskiy A, Beyer L, Kolesnikov A, et al. An image is worth 16×16 words: Transformers for image recognition at scale [PP/OL]. arXiv (2020-10-22)[2025-11-15]. https://arxiv.org/abs/2010.11929.
|
|
|
| [15] |
Liu Z, Lin Y, Cao Y, et al. Swin transformer: hierarchical vision transformer using shifted windows [C]// IEEE/CVF International Conference on Computer Vision. Montreal: IEEE, 2021: 10012-10022.
|
|
|
| [16] |
张振利, 胡新凯, 李凡, 等 基于 CNN 和 Efficient Transformer 的多尺度遥感图像语义分割算法[J]. 浙江大学学报(工学版), 2025, 59 (4): 778- 786 Zhang Zhenli, Hu Xinkai, Li Fan, et al Semantic segmentation algorithm for multiscale remote sensing images based on CNN and Efficient Transformer[J]. Journal of Zhejiang University (Engineering Science), 2025, 59 (4): 778- 786
|
|
|
| [17] |
刘春娟, 乔泽, 闫浩文, 等 基于多尺度互注意力的遥感图像语义分割网络[J]. 浙江大学学报(工学版), 2023, 57 (07): 1335- 1344 Liu Chunjuan, Qiao Ze, Yan Haowen, et al Semantic segmentation network for remote sensing image based on multi-scale mutual attention[J]. Journal of Zhejiang University (Engineering Science), 2023, 57 (07): 1335- 1344
|
|
|
| [18] |
梁龙学, 贺成龙, 吴小所, 等 全局信息提取与重建的遥感图像语义分割网络[J]. 浙江大学学报(工学版), 2024, 58 (11): 2270- 2279 Liang Longxue, He Chenglong, Wu Xiaosuo, et al Remote sensing image semantic segmentation network based on global information extraction and reconstruction[J]. Journal of Zhejiang University (Engineering Science), 2024, 58 (11): 2270- 2279
|
|
|
| [19] |
Yang K, Wang S, Dai J, et al. WITT: a wireless image transmission transformer for semantic communications [C]// 2023 IEEE International Conference on Acoustics, Speech and Signal Processing. Rhodes: IEEE, 2023: 1-5.
|
|
|
| [20] |
Nguyen L X, Tun Y L, Tun Y K, et al Swin transformer-based dynamic semantic communication for multi-user with different computing capacity[J]. IEEE Transactions on Vehicular Technology, 2024, 73 (6): 8957- 8972
doi: 10.1109/TVT.2024.3362328
|
|
|
| [21] |
Blau Y, Michaeli T. The perception-distortion tradeoff [C]// Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition. Salt Lake City: IEEE, 2018: 6228-6237.
|
|
|
| [22] |
Ledig C, Theis L, Huszár F, et al. Photo-realistic single image super-resolution using a generative adversarial network [C]// Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition. Honolulu: IEEE, 2017: 4681–4690.
|
|
|
| [23] |
Agustsson E, Minnen D, Toderici G, et al Generative adversarial networks for extreme learned image compression[J]. IEEE Transactions on Image Processing, 2020, 29: 2752- 2763
|
|
|
| [24] |
Zhang G, Hu Q, Cai Y, et al. Adaptive CSI feedback for deep learning-enabled image transmission [C]// IEEE International Conference on Communications. Rome: IEEE, 2023: 4163-4168.
|
|
|
| [25] |
Song M, Ma N, Dong C, et al Deep joint source-channel coding for wireless image transmission with adaptive models[J]. Electronics, 2023, 12 (22): 4637
doi: 10.3390/electronics12224637
|
|
|
| [26] |
Liang C, Du H, Sun Y, et al Generative AI-driven semantic communication networks: architecture, technologies and applications[J]. IEEE Transactions on Cognitive Communications and Networking, 2024, 11 (1): 27- 47
doi: 10.1109/tccn.2024.3435524
|
|
|
| [27] |
Grassucci E, Barbarossa S, Comminiello D. Generative semantic communication: diffusion models beyond bit recovery [PP/OL]. arXiv (2023-06-07)[2025-11-15]. https://arxiv.org/abs/2306.04321.
|
|
|
| [28] |
Erdem E, Gündüz D, Dragotti P L, et al Generative joint source-channel coding for semantic image transmission[J]. IEEE Journal on Selected Areas in Communications, 2023, 41 (8): 2645- 2657
doi: 10.1109/JSAC.2023.3288243
|
|
|
| [29] |
杨冰, 周家辉, 姚金良, 等 基于多模态语义信息的文本生成图像方法[J]. 浙江大学学报(工学版), 2026, 60 (2): 1- 10 Yang Bing, Zhou Jiahui, Yao Jinliang, et al Text-to-image generation method based on multimodal semantic information[J]. Journal of Zhejiang University (Engineering Science), 2026, 60 (2): 1- 10
|
|
|
| [30] |
Liang W, Ma N, Shen L, et al. An image semantic communication system based on Swin Transformer and generative adversarial networks [C]// 2024 IEEE Globecom Workshops. Cape Town: IEEE, 2024: 1-6.
|
|
|
| [31] |
Richardson T, Kudekar S Design of low-density parity check codes for 5G new radio[J]. IEEE Communications Magazine, 2018, 56 (3): 28- 34
doi: 10.1109/MCOM.2018.1700839
|
|
|
|
Viewed |
|
|
|
Full text
|
|
|
|
|
Abstract
|
|
|
|
|
Cited |
|
|
|
|
| |
Shared |
|
|
|
|
| |
Discussed |
|
|
|
|