Please wait a minute...
浙江大学学报(工学版)  2026, Vol. 60 Issue (12): 2694-2703    DOI: 10.3785/j.issn.1008-973X.2026.12.015
通信与电子工程     
基于CSI导向注意力与门控生成修复的图像语义通信
彭艺(),李恒,杨青青*(),李辉,王健明
昆明理工大学 信息工程与自动化学院,云南 昆明 650031
Image semantic communication based on CSI-guided attention and gated generation repair
Yi PENG(),Heng LI,Qingqing YANG*(),Hui LI,Jianming WANG
Faculty of Information Engineering and Automation, Kunming University of Science and Technology, Kunming 650031, China
 全文: PDF(1392 KB)   HTML
摘要:

针对现有深度联合信源信道编码方案在图像语义通信中存在重建纹理丢失与边缘模糊的问题,提出融合信道状态信息(CSI)感知与生成对抗网络(GAN)辅助的Swin Transformer图像语义通信方案,以提升典型信噪比下图像传输的感知质量与鲁棒性. 在编码端,设计CSI导向自注意力机制,通过多层感知机将标量CSI映射为加性偏置矩阵并注入到自注意力计算中,动态重塑特征提取拓扑. 在解码端,构建CSI门控生成式修复模块,利用CSI经过非线性映射生成门控因子,自适应调节GAN对残差特征的修复权重. 实验表明,所提方案在不同分辨率数据集上的峰值信噪比(PSNR)与多尺度结构相似性(MS-SSIM)指标均优于传统分离式编码及现有主流语义通信方案,为复杂信道环境下的图像语义传输提供了有效的解决方案.

关键词: 语义通信;  Swin Transformer;  信道状态信息;  自适应调节;  生成对抗网络    
Abstract:

A Swin Transformer-based image semantic communication scheme that integrated channel state information (CSI) awareness with generative adversarial network (GAN) assistance was proposed aiming at the issues of texture loss and edge blurring in reconstructed images encountered by existing deep joint source-channel coding schemes for image semantic communication in order to improve the perceptual quality and robustness of image transmission under typical SNR conditions. A CSI-guided self-attention mechanism was designed at the encoder. A scalar CSI was mapped into an additive bias matrix through a multilayer perceptron and incorporated into the self-attention computation in order to dynamically reshape the feature extraction topology. A CSI-gated generative restoration module was developed at the decoder. CSI was nonlinearly mapped to generate a gating factor for dynamically adjusting the restoration weight of the GAN on residual features. The experimental results demonstrate that the proposed scheme achieves superior peak signal-to-noise ratio (PSNR) and multi-scale structural similarity (MS-SSIM) performance across datasets with different resolutions compared with conventional separated source-channel coding and existing mainstream semantic communication schemes. The proposed method provides an effective solution for image semantic transmission in complex channel environments.

Key words: semantic communication    Swin Transformer    channel state information    adaptive adjustment    generative adversarial network
收稿日期: 2025-12-04 出版日期: 2026-10-21
CLC:  TN 919  
基金资助: 国家自然科学基金资助项目(62461030);云南省基础研究重点资助项目(202401AS070105);中央引导地方科技发展专项资金资助项目(202607AD040003).
通讯作者: 杨青青     E-mail: 12309214@kust.edu.cn;20090119@kust.edu.cn
作者简介: 彭艺(1975—),女,教授,博士,从事无线通信、认知无线电技术、协作通信研究. orcid.org/0009-0004-3389-0254. E-mail:12309214@kust.edu.cn
服务  
把本文推荐给朋友
加入引用管理器
E-mail Alert
作者相关文章  
彭艺
李恒
杨青青
李辉
王健明

引用本文:

彭艺,李恒,杨青青,李辉,王健明. 基于CSI导向注意力与门控生成修复的图像语义通信[J]. 浙江大学学报(工学版), 2026, 60(12): 2694-2703.

Yi PENG,Heng LI,Qingqing YANG,Hui LI,Jianming WANG. Image semantic communication based on CSI-guided attention and gated generation repair. Journal of ZheJiang University (Engineering Science), 2026, 60(12): 2694-2703.

链接本文:

https://www.zjujournals.com/eng/CN/10.3785/j.issn.1008-973X.2026.12.015    或    https://www.zjujournals.com/eng/CN/Y2026/V60/I12/2694

图 1  基于Swin Transformer的图像语义传输系统
图 2  2个连续的Swin Transformer块
图 3  CSI感知双端自适应机制
图 4  CGSA机制结构
图 5  CGGR模块的结构
图 6  不同传输方案重建图像的PSNR随信噪比的变化曲线
图 7  不同传输方案重建图像的MS-SSIM随信噪比的变化曲线
图 8  不同方案传输Kodak数据集典型场景的视觉对比
方案变体描述
CAG-ST本研究提出的完整方案,同时包含CGSA机制和CGGR模块.
CAG-ST(CGSA)在基线基础上,仅在编码端引入CSI导向自注意力机制.
CAG-ST(CGGR,$ \alpha $+GAN)在基线基础上,仅在解码端引入完整的CGGR模块.
CAG-ST(CGGR,$ \alpha $)CGGR模块移除GAN仅保留CSI门控机制,且编码端不含CGSA机制.
CAG-ST(CGGR,GAN)CGGR模块移除CSI门控机制仅保留GAN,且编码端不含CGSA机制.
表 1  CAG-ST方案的不同变体
图 9  不同CAG-ST变体的重建图像PSNR随信噪比的变化曲线    
图 10  不同CAG-ST变体的重建图像MS-SSIM随信噪比的变化曲线
传输方案Np/106FLOPs/109ten/mstde/ms
Deep JSCC15.8153.4348.2667.48
WITT28.233.617.519.39
CAG-ST(CGSA)37.5341.379.199.72
CAG-ST45.4775.989.1915.41
表 2  不同传输方案的复杂度测试
1 Tong W, Zhu P. 6G: the next horizon [M]. Cambridge: Cambridge University Press, 2021.
2 Letaief K B, Chen Wei, Shi Yuanming, et al The roadmap to 6G: AI empowered wireless networks[J]. IEEE Communications Magazine, 2019, 57 (8): 84- 90
doi: 10.1109/MCOM.2019.1900271
3 Wang Y, Han H, Feng Y, et al Semantic communication empowered 6g networks: techniques, applications, and challenges[J]. IEEE Access, 2025, 13: 28293- 28314
doi: 10.1109/ACCESS.2025.3532797
4 栾宏之, 李书才, 李屹, 等 5G-A车联网确定性网络中的语义通信系统[J]. 浙江大学学报(工学版), 2025, 59 (5): 964- 972
Luan Hongzhi, Li Shucai, Li Yi, et al Semantic communication system in deterministic networking for5G-advanced vehicular network[J]. Journal of Zhejiang University (Engineering Science), 2025, 59 (5): 964- 972
5 Shannon C E A mathematical theory of communication[J]. The Bell System Technical Journal, 1948, 27 (3): 379- 423
doi: 10.1002/j.1538-7305.1948.tb01338.x
6 Zhang P, Xu W, Gao H, et al Toward wisdom-evolutionary and primitive-concise 6G: a new paradigm of semantic communication networks[J]. Engineering, 2022, 8: 60- 73
doi: 10.1016/j.eng.2021.11.003
7 Fang Y, He Q, Ding D, et al. Deep-learning based gaussian mixture likelihoods joint source-channel coding for image transmission [C]//2023 International Conference on Wireless Communications and Signal Processing. Hangzhou: IEEE, 2023: 19-24.
8 Gündüz D, Qin Z, Aguerri I E, et al Beyond transmitting bits: context, semantics, and task-oriented communications[J]. IEEE Journal on Selected Areas in Communications, 2023, 41 (1): 5- 41
doi: 10.1109/JSAC.2022.3223408
9 Bourtsoulatze E, Kurka D B, Gündüz D Deep joint source-channel coding for wireless image transmission[J]. IEEE Transactions on Cognitive Communications and Networking, 2019, 5 (3): 567- 579
doi: 10.1109/TCCN.2019.2919300
10 Dai J, Wang S, Tan K, et al Nonlinear transform source-channel coding for semantic communications[J]. IEEE Journal on Selected Areas in Communications, 2022, 40 (8): 2300- 2316
doi: 10.1109/JSAC.2022.3180802
11 Park T, Hong E, Jeon Y, et al. Robust deep joint source channel coding for task-oriented semantic communications [PP/OL]. arXiv (2025-03-17)[2025-11-15]. https://arxiv.org/abs/2503.12907.
12 Hu Q, Zhang G, Qin Z, et al Robust semantic communications with masked VQ-VAE enabled codebook[J]. IEEE Transactions on Communications, 2023, 22 (12): 8707- 8722
13 Kurka D B, Gündüz D DeepJSCC-f: deep joint source-channel coding of images with feedback[J]. IEEE Journal on Selected Areas in Information Theory, 2020, 1 (1): 178- 193
doi: 10.1109/JSAIT.2020.2987203
14 Dosovitskiy A, Beyer L, Kolesnikov A, et al. An image is worth 16×16 words: Transformers for image recognition at scale [PP/OL]. arXiv (2020-10-22)[2025-11-15]. https://arxiv.org/abs/2010.11929.
15 Liu Z, Lin Y, Cao Y, et al. Swin transformer: hierarchical vision transformer using shifted windows [C]// IEEE/CVF International Conference on Computer Vision. Montreal: IEEE, 2021: 10012-10022.
16 张振利, 胡新凯, 李凡, 等 基于 CNN 和 Efficient Transformer 的多尺度遥感图像语义分割算法[J]. 浙江大学学报(工学版), 2025, 59 (4): 778- 786
Zhang Zhenli, Hu Xinkai, Li Fan, et al Semantic segmentation algorithm for multiscale remote sensing images based on CNN and Efficient Transformer[J]. Journal of Zhejiang University (Engineering Science), 2025, 59 (4): 778- 786
17 刘春娟, 乔泽, 闫浩文, 等 基于多尺度互注意力的遥感图像语义分割网络[J]. 浙江大学学报(工学版), 2023, 57 (07): 1335- 1344
Liu Chunjuan, Qiao Ze, Yan Haowen, et al Semantic segmentation network for remote sensing image based on multi-scale mutual attention[J]. Journal of Zhejiang University (Engineering Science), 2023, 57 (07): 1335- 1344
18 梁龙学, 贺成龙, 吴小所, 等 全局信息提取与重建的遥感图像语义分割网络[J]. 浙江大学学报(工学版), 2024, 58 (11): 2270- 2279
Liang Longxue, He Chenglong, Wu Xiaosuo, et al Remote sensing image semantic segmentation network based on global information extraction and reconstruction[J]. Journal of Zhejiang University (Engineering Science), 2024, 58 (11): 2270- 2279
19 Yang K, Wang S, Dai J, et al. WITT: a wireless image transmission transformer for semantic communications [C]// 2023 IEEE International Conference on Acoustics, Speech and Signal Processing. Rhodes: IEEE, 2023: 1-5.
20 Nguyen L X, Tun Y L, Tun Y K, et al Swin transformer-based dynamic semantic communication for multi-user with different computing capacity[J]. IEEE Transactions on Vehicular Technology, 2024, 73 (6): 8957- 8972
doi: 10.1109/TVT.2024.3362328
21 Blau Y, Michaeli T. The perception-distortion tradeoff [C]// Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition. Salt Lake City: IEEE, 2018: 6228-6237.
22 Ledig C, Theis L, Huszár F, et al. Photo-realistic single image super-resolution using a generative adversarial network [C]// Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition. Honolulu: IEEE, 2017: 4681–4690.
23 Agustsson E, Minnen D, Toderici G, et al Generative adversarial networks for extreme learned image compression[J]. IEEE Transactions on Image Processing, 2020, 29: 2752- 2763
24 Zhang G, Hu Q, Cai Y, et al. Adaptive CSI feedback for deep learning-enabled image transmission [C]// IEEE International Conference on Communications. Rome: IEEE, 2023: 4163-4168.
25 Song M, Ma N, Dong C, et al Deep joint source-channel coding for wireless image transmission with adaptive models[J]. Electronics, 2023, 12 (22): 4637
doi: 10.3390/electronics12224637
26 Liang C, Du H, Sun Y, et al Generative AI-driven semantic communication networks: architecture, technologies and applications[J]. IEEE Transactions on Cognitive Communications and Networking, 2024, 11 (1): 27- 47
doi: 10.1109/tccn.2024.3435524
27 Grassucci E, Barbarossa S, Comminiello D. Generative semantic communication: diffusion models beyond bit recovery [PP/OL]. arXiv (2023-06-07)[2025-11-15]. https://arxiv.org/abs/2306.04321.
28 Erdem E, Gündüz D, Dragotti P L, et al Generative joint source-channel coding for semantic image transmission[J]. IEEE Journal on Selected Areas in Communications, 2023, 41 (8): 2645- 2657
doi: 10.1109/JSAC.2023.3288243
29 杨冰, 周家辉, 姚金良, 等 基于多模态语义信息的文本生成图像方法[J]. 浙江大学学报(工学版), 2026, 60 (2): 1- 10
Yang Bing, Zhou Jiahui, Yao Jinliang, et al Text-to-image generation method based on multimodal semantic information[J]. Journal of Zhejiang University (Engineering Science), 2026, 60 (2): 1- 10
30 Liang W, Ma N, Shen L, et al. An image semantic communication system based on Swin Transformer and generative adversarial networks [C]// 2024 IEEE Globecom Workshops. Cape Town: IEEE, 2024: 1-6.
31 Richardson T, Kudekar S Design of low-density parity check codes for 5G new radio[J]. IEEE Communications Magazine, 2018, 56 (3): 28- 34
doi: 10.1109/MCOM.2018.1700839
[1] 李云红,张琪琪,陈锦妮,陈伟重,苏雪平,梁成名. 基于生成对抗网络和坐标注意力机制的文本生成图像算法[J]. 浙江大学学报(工学版), 2026, 60(6): 1213-1220.
[2] 陈可嘉,曾威文,王朕卿. 基于条件生成对抗和泊松融合的图像数据增强方法[J]. 浙江大学学报(工学版), 2026, 60(12): 2592-2601.
[3] 宋莉,宛袁玉,金苍宏,宋明黎. 基于对抗图生成的动态蒙特卡洛强化学习[J]. 浙江大学学报(工学版), 2026, 60(12): 2613-2622.
[4] 彭艺,李恒,杨青青,李辉,王健明. 基于Swin Transformer的图像多级语义MIMO传输[J]. 浙江大学学报(工学版), 2026, 60(11): 2462-2471.
[5] 栾宏之,李书才,李屹,钱传杰,梅琼. 5G-A车联网确定性网络中的语义通信系统[J]. 浙江大学学报(工学版), 2025, 59(5): 964-972.
[6] 李勇,刘志强,田茂幸,贾松霖. 基于全局相关语义重要性的语义压缩算法[J]. 浙江大学学报(工学版), 2025, 59(4): 795-803.
[7] 杨燕,贾存鹏. 代理注意力下域特征交互的高效图像去雾算法[J]. 浙江大学学报(工学版), 2025, 59(12): 2527-2538.
[8] 王天飞,周文俊,项圣,贺宇航,彭博. 基于异常特征对抗学习的工业图像异常检测方法[J]. 浙江大学学报(工学版), 2025, 59(12): 2566-2575.
[9] 杨磊,何鹏举,丑幸幸. 基于TimeGAN数据增强的复杂过程故障分类方法[J]. 浙江大学学报(工学版), 2024, 58(9): 1768-1780.
[10] 曹寅,秦俊平,高彤,马千里,任家琪. 基于生成对抗网络的文本两阶段生成高质量图像方法[J]. 浙江大学学报(工学版), 2024, 58(4): 674-683.
[11] 曹寅,秦俊平,马千里,孙昊,闫凯,王磊,任家琪. 文本生成图像研究综述[J]. 浙江大学学报(工学版), 2024, 58(2): 219-238.
[12] 李灿林,张文娇,邵志文,马利庄,王新玥. 基于Trans-nightSeg的夜间道路场景语义分割方法[J]. 浙江大学学报(工学版), 2024, 58(2): 294-303.
[13] 李海烽,张雪英,段淑斐,贾海蓉,Huizhi Liang . 融合生成对抗网络与时间卷积网络的普通话情感识别[J]. 浙江大学学报(工学版), 2023, 57(9): 1865-1875.
[14] 李云红,段姣姣,苏雪平,张蕾涛,于惠康,刘杏瑞. 基于改进生成对抗网络的书法字生成算法[J]. 浙江大学学报(工学版), 2023, 57(7): 1326-1334.
[15] 曹晓璐,卢富男,朱翔,翁立波,卢书芳,高飞. 基于草图的兼容性服装生成方法[J]. 浙江大学学报(工学版), 2023, 57(5): 939-947.