基于Swin Transformer的图像多级语义MIMO传输

doi:10.3785/j.issn.1008-973X.2026.11.015

浙江大学学报(工学版)

2026, Vol. 60

Issue (11): 2462-2471 DOI: 10.3785/j.issn.1008-973X.2026.11.015

计算机技术

基于Swin Transformer的图像多级语义MIMO传输

彭艺1,2(

),李恒1,2,杨青青1,2,李辉1,2,王健明1,2

1. 昆明理工大学信息工程与自动化学院，云南昆明 650031
2. 昆明理工大学云南省计算机重点实验室，云南昆明 650500

Multilevel semantic MIMO transmission of images based on Swin Transformer

Yi PENG1,2(

),Heng LI1,2,Qingqing YANG1,2,Hui LI1,2,Jianming WANG1,2

1. Faculty of Information Engineering and Automation, Kunming University of Science and Technology, Kunming 650031, China
2. Yunnan Provincial Key Laboratory of Computer Science, Kunming University of Science and Technology, Kunming 650500, China

全文: PDF(1267 KB) HTML

摘要：

语义通信在低信噪比（SNR）环境下易丢失细节，MIMO系统应用时接收端语义信息的模糊与畸变问题加剧. 为此，提出MIMO系统下基于Swin Transformer的图像多级语义特征自适应融合的传输方案. Swin Transformer作为骨干网络提取像素语义特征，引入多级语义特征自适应融合机制，联合像素级细节与标签级文本语义信息，生成紧凑且全局关联的语义表征，增强低SNR条件下图像重建能力. 通过Transformer预编码技术增强语义-信道联合优化，抑制多用户干扰并提升低SNR下的语义保真度. 实验结果表明，4×4 MIMO场景下SNR为0 dB时，所提方案较JPEG2000、BPG和Deep JSCC在峰值信噪比（PSNR）上分别提升3.06倍、4.55倍和62.97%，在结构相似性（SSIM）上分别提升2.79倍、4.46倍和42.59%，表现出更优的重建图像质量.

关键词： 语义通信; Swin Transformer; MIMO系统; 多级语义特征融合; 图像重建

Abstract:

Semantic communication tends to lose details in low signal-to-noise ratio (SNR) environments, and the application of MIMO systems further increases the ambiguity and distortion of semantic information at the receiver. Therefore, a transmission scheme for image multilevel semantic feature adaptive fusion based on Swin Transformer was proposed for MIMO systems. The Swin Transformer was employed as the backbone network to extract pixel-level semantic features. A multilevel semantic feature adaptive fusion mechanism was introduced to integrate pixel-level details and label-level text semantic information, generating a compact and globally correlated semantic representation to enhance image reconstruction under low SNR conditions. On this basis, Transformer precoding technology was applied to improve semantic-channel joint optimization, suppress multi-user interference, and enhance semantic fidelity under low SNR. Experimental results show that in a 4×4 MIMO scenario at an SNR of 0 dB, the proposed scheme outperforms JPEG2000, BPG, and Deep JSCC by 3.06 times, 4.55 times, and 62.97% in peak signal-to-noise ratio (PSNR), and by 2.79 times, 4.46 times, and 42.59% in structural similarity index measure (SSIM), respectively, demonstrating superior reconstructed image quality.

Key words: semantic communication Swin Transformer MIMO system multilevel semantic feature fusion image reconstruction

收稿日期: 2025-09-06 出版日期: 2026-10-16

CLC:

TN 919.85

基金资助: 国家自然科学基金资助项目（62461030）；云南省基础研究重点项目（202401AS070105）；云南省基础研究面上项目（202401AT070415）；中央引导地方科技发展专项资金项目（202607AD040003）.

作者简介: 彭艺（1975—），女，教授，博士，从事无线通信，认知无线电技术、协作通信研究. orcid.org/0009-0004-3389-0254. E-mail：12309214@kust.edu.cn

	服务
	把本文推荐给朋友
	加入引用管理器
	E-mail Alert
	作者相关文章
	彭艺
	李恒
	杨青青
	李辉
	王健明

引用本文:

彭艺,李恒,杨青青,李辉,王健明. 基于Swin Transformer的图像多级语义MIMO传输[J]. 浙江大学学报(工学版), 2026, 60(11): 2462-2471.

Yi PENG,Heng LI,Qingqing YANG,Hui LI,Jianming WANG. Multilevel semantic MIMO transmission of images based on Swin Transformer. Journal of ZheJiang University (Engineering Science), 2026, 60(11): 2462-2471.

链接本文:

https://www.zjujournals.com/eng/CN/10.3785/j.issn.1008-973X.2026.11.015 或 https://www.zjujournals.com/eng/CN/Y2026/V60/I11/2462

图 1 传统数字通信方案

图 2 图像语义传输方案及语义知识库映射

图 3 基于神经网络的图像语义通信系统

图 4 图像语义通信MIMO系统

图 5 语义编码器结构及连续Swin Transformer块结构

图 6 多级语义特征融合机制

图 7 Transformer预编码器和解码器的结构

图 8 不同传输方案的图像重建质量对比

图 9 信道带宽比对不同传输方案图像重建质量的影响

图 10 多级语义特征融合对图像重建质量的影响

表 1 方案复杂度与实时性的评估

16	XIE B, WU Y, SHI Y, et al Robust image semantic coding with learnable CSI fusion masking over MIMO fading channels[J]. IEEE Transactions on Wireless Communications, 2024, 23 (10): 14155- 14170 doi: 10.1109/TWC.2024.3409735
17	HUANG D, GAO F, TAO X, et al Toward semantic communications: deep learning-based image semantic coding[J]. IEEE Journal on Selected Areas in Communications, 2023, 41 (1): 55- 71 doi: 10.1109/JSAC.2022.3221999
18	SHI Y, SHAO S, WU Y, et al Excess distortion exponent analysis for semantic-aware MIMO communication systems[J]. IEEE Transactions on Wireless Communications, 2023, 22 (9): 5927- 5940 doi: 10.1109/TWC.2023.3238463
19	CAI C, YUAN X, ZHANG Y A End-to-end learning for task-oriented semantic communications over MIMO channels: an information-theoretic framework[J]. IEEE Journal on Selected Areas in Communications, 2025, 43 (4): 1292- 1307 doi: 10.1109/JSAC.2025.3531575
20	WANG S, DAI J, LIANG Z, et al Wireless deep video semantic transmission[J]. IEEE Journal on Selected Areas in Communications, 2023, 41 (1): 214- 229 doi: 10.1109/JSAC.2022.3221977
21	BYERS G J, TAKAWIRA F Spatially and temporally correlated MIMO channels: modeling and capacity analysis[J]. IEEE Transactions on Vehicular Technology, 2004, 53 (3): 634- 643 doi: 10.1109/TVT.2004.825766
22	LIU X, XU Q Adaptive attention-based high-level semantic introduction for image caption[J]. ACM Transactions on Multimedia Computing, Communications, and Applications, 2020, 16 (4): 1- 22 doi: 10.1145/3409388
23	LI D, ZHAO Y, CUI R, et al. Research on image text generation based on word2vec visual vocabulary attention [C]// Proceedings of the Asia-Pacific Conference on Communications Technology and Computer Science. Shenyang: IEEE, 2021: 344–348.
24	ZHAI X, WANG X, MUSTAFA B, et al. LiT: zero-shot transfer with locked-image text tuning [C]// Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition. New Orleans: IEEE, 2022: 18102–18112.
25	TATARIA H, SHAFI M. Simple framework of ZF precoding analysis for full-dimensional massive MIMO systems [C]// Proceedings of the IEEE 94th Vehicular Technology Conference. Norman: IEEE, 2021: 1–5.
1	YANG K, WANG S, DAI J, et al SwinJSCC: taming Swin Transformer for deep joint source-channel coding[J]. IEEE Transactions on Cognitive Communications and Networking, 2025, 11 (1): 90- 104 doi: 10.1109/TCCN.2024.3424842
2	YANG K, WANG S, DAI J, et al. WITT: a wireless image transmission transformer for semantic communications [C]// Proceedings of the ICASSP 2023—2023 IEEE International Conference on Acoustics, Speech and Signal Processing. Rhodes Island: IEEE, 2023: 1–5.
3	张平, 戴金晟, 张育铭, 等面向语义通信的非线性变换编码[J]. 通信学报, 2023, 44 (4): 1- 14 ZHANG Ping, DAI Jinsheng, ZHANG Yuming, et al Nonlinear transform coding for semantic communications[J]. Journal on Communications, 2023, 44 (4): 1- 14 doi: 10.11959/j.issn.1000-436x.2023087
4	陈建侨, 马楠, 许晓东, 等面向语义通信的信道知识库构建与信道处理研究综述[J]. 无线电通信技术, 2024, 50 (3): 519- 527 CHEN Jianqiao, MA Nan, XU Xiaodong, et al Recent research on channel knowledge base construction and channel processing for semantic communication: a survey[J]. Radio Communications Technology, 2024, 50 (3): 519- 527 doi: 10.3969/j.issn.1003-3114.2024.03.013
5	HUANG D, TAO X, GAO F, et al. Deep learning-based image semantic coding for semantic communications [C]// Proceedings of the IEEE Global Communications Conference. Madrid: IEEE, 2022: 1–6.
6	NGUYEN L X, TUN Y L, TUN Y K, et al Swin Transformer-based dynamic semantic communication for multi-user with different computing capacity[J]. IEEE Transactions on Vehicular Technology, 2024, 73 (6): 8957- 8972 doi: 10.1109/TVT.2024.3362328
7	BOURTSOULATZE E, BURTH KURKA D, GÜNDÜZ D Deep joint source-channel coding for wireless image transmission[J]. IEEE Transactions on Cognitive Communications and Networking, 2019, 5 (3): 567- 579 doi: 10.1109/TCCN.2019.2919300
8	WANG Y, GAO Z, ZHENG D, et al Transformer-empowered 6G intelligent networks: from massive MIMO processing to semantic communication[J]. IEEE Wireless Communications, 2023, 30 (6): 127- 135 doi: 10.1109/MWC.008.2200157
9	DOSOVITSKIY A, BEYER L, KOLESNIKOV A, et al. An image is worth 16x16 words: transformers for image recognition at scale [EB/OL]. (2021–06–03)[2025–05–15]. https://arxiv.org/pdf/2010.11929.
10	HAN K, WANG Y, CHEN H, et al A survey on vision transformer[J]. IEEE Transactions on Pattern Analysis and Machine Intelligence, 2023, 45 (1): 87- 110 doi: 10.1109/TPAMI.2022.3152247
11	VASWANI A, SHAZEER N, PARMAR N, et al. Attention is all you need [C]// Proceedings of the 31st International Conference on Neural Information Processing Systems. [S.l.]: Curran Associates Inc. , 2017: 6000–6010.
12	LIU Z, LIN Y, CAO Y, et al. Swin Transformer: hierarchical vision transformer using shifted windows [C]// Proceedings of the IEEE/CVF International Conference on Computer Vision. Montreal: IEEE, 2022: 9992–10002.
13	LIANG H, BAO Z, AN W, et al. Semantic importance-aware based for multi-user communication over MIMO fading channels [EB/OL]. (2023–12–26)[2025–05–15]. https://arxiv.org/pdf/2312.16057.
14	PENG K, MENG Y, WU D, et al. Multi-task semantic communication for image transmission over MIMO channel [C]// Proceedings of the 16th International Conference on Wireless Communications and Signal Processing. Hefei: IEEE, 2025: 1468–1473.

[1]	段继忠,李海源. 基于变分模型和Transformer的多尺度并行磁共振成像重建[J]. 浙江大学学报(工学版), 2025, 59(9): 1826-1837.
[2]	栾宏之,李书才,李屹,钱传杰,梅琼. 5G-A车联网确定性网络中的语义通信系统[J]. 浙江大学学报(工学版), 2025, 59(5): 964-972.
[3]	李勇,刘志强,田茂幸,贾松霖. 基于全局相关语义重要性的语义压缩算法[J]. 浙江大学学报(工学版), 2025, 59(4): 795-803.
[4]	杨燕,贾存鹏. 代理注意力下域特征交互的高效图像去雾算法[J]. 浙江大学学报(工学版), 2025, 59(12): 2527-2538.
[5]	刘近贞,陈飞,熊慧. 多尺度残差网络模型的开放式电阻抗成像算法[J]. 浙江大学学报(工学版), 2022, 56(9): 1789-1795.
[6]	何立,庞善民. 结合年龄监督和人脸先验的语音-人脸图像重建[J]. 浙江大学学报(工学版), 2022, 56(5): 1006-1016.
[7]	董巍, 沈会良. 双向纹理函数稀疏采集与重建[J]. 浙江大学学报(工学版), 2016, 50(12): 2364-2370.
[8]	王嵩, 上田之雄, 山下丰, 刘华锋. 自适应双重点阵DOT图像重建[J]. J4, 2013, 47(1): 102-108.
[9]	张俊超, 岳茂雄, 刘华锋. 结构先验约束的动态PET图像重建[J]. J4, 2012, 46(6): 961-966.
[10]	朱正为,周建江. 基于MMSE-T的合成孔径雷达图像超分辨率重建[J]. J4, 2011, 45(9): 1576-1581.

Viewed

Full text

Abstract

Cited

Shared

Discussed