|
|
|
| Multilevel semantic MIMO transmission of images based on Swin Transformer |
Yi PENG1,2( ),Heng LI1,2,Qingqing YANG1,2,Hui LI1,2,Jianming WANG1,2 |
1. Faculty of Information Engineering and Automation, Kunming University of Science and Technology, Kunming 650031, China
2. Yunnan Provincial Key Laboratory of Computer Science, Kunming University of Science and Technology, Kunming 650500, China |
|
|
|
Abstract Semantic communication tends to lose details in low signal-to-noise ratio (SNR) environments, and the application of MIMO systems further increases the ambiguity and distortion of semantic information at the receiver. Therefore, a transmission scheme for image multilevel semantic feature adaptive fusion based on Swin Transformer was proposed for MIMO systems. The Swin Transformer was employed as the backbone network to extract pixel-level semantic features. A multilevel semantic feature adaptive fusion mechanism was introduced to integrate pixel-level details and label-level text semantic information, generating a compact and globally correlated semantic representation to enhance image reconstruction under low SNR conditions. On this basis, Transformer precoding technology was applied to improve semantic-channel joint optimization, suppress multi-user interference, and enhance semantic fidelity under low SNR. Experimental results show that in a 4×4 MIMO scenario at an SNR of 0 dB, the proposed scheme outperforms JPEG2000, BPG, and Deep JSCC by 3.06 times, 4.55 times, and 62.97% in peak signal-to-noise ratio (PSNR), and by 2.79 times, 4.46 times, and 42.59% in structural similarity index measure (SSIM), respectively, demonstrating superior reconstructed image quality.
|
|
Received: 06 September 2025
Published: 16 October 2026
|
|
|
| Fund: 国家自然科学基金资助项目（62461030）；云南省基础研究重点项目（202401AS070105）；云南省基础研究面上项目（202401AT070415）；中央引导地方科技发展专项资金项目（202607AD040003）. |
基于Swin Transformer的图像多级语义MIMO传输
语义通信在低信噪比（SNR）环境下易丢失细节，MIMO系统应用时接收端语义信息的模糊与畸变问题加剧. 为此，提出MIMO系统下基于Swin Transformer的图像多级语义特征自适应融合的传输方案. Swin Transformer作为骨干网络提取像素语义特征，引入多级语义特征自适应融合机制，联合像素级细节与标签级文本语义信息，生成紧凑且全局关联的语义表征，增强低SNR条件下图像重建能力. 通过Transformer预编码技术增强语义-信道联合优化，抑制多用户干扰并提升低SNR下的语义保真度. 实验结果表明，4×4 MIMO场景下SNR为0 dB时，所提方案较JPEG2000、BPG和Deep JSCC在峰值信噪比（PSNR）上分别提升3.06倍、4.55倍和62.97%，在结构相似性（SSIM）上分别提升2.79倍、4.46倍和42.59%，表现出更优的重建图像质量.
关键词：
语义通信,
Swin Transformer,
MIMO系统,
多级语义特征融合,
图像重建
|
|
| [16] |
XIE B, WU Y, SHI Y, et al Robust image semantic coding with learnable CSI fusion masking over MIMO fading channels[J]. IEEE Transactions on Wireless Communications, 2024, 23 (10): 14155- 14170
doi: 10.1109/TWC.2024.3409735
|
|
|
| [17] |
HUANG D, GAO F, TAO X, et al Toward semantic communications: deep learning-based image semantic coding[J]. IEEE Journal on Selected Areas in Communications, 2023, 41 (1): 55- 71
doi: 10.1109/JSAC.2022.3221999
|
|
|
| [18] |
SHI Y, SHAO S, WU Y, et al Excess distortion exponent analysis for semantic-aware MIMO communication systems[J]. IEEE Transactions on Wireless Communications, 2023, 22 (9): 5927- 5940
doi: 10.1109/TWC.2023.3238463
|
|
|
| [19] |
CAI C, YUAN X, ZHANG Y A End-to-end learning for task-oriented semantic communications over MIMO channels: an information-theoretic framework[J]. IEEE Journal on Selected Areas in Communications, 2025, 43 (4): 1292- 1307
doi: 10.1109/JSAC.2025.3531575
|
|
|
| [20] |
WANG S, DAI J, LIANG Z, et al Wireless deep video semantic transmission[J]. IEEE Journal on Selected Areas in Communications, 2023, 41 (1): 214- 229
doi: 10.1109/JSAC.2022.3221977
|
|
|
| [21] |
BYERS G J, TAKAWIRA F Spatially and temporally correlated MIMO channels: modeling and capacity analysis[J]. IEEE Transactions on Vehicular Technology, 2004, 53 (3): 634- 643
doi: 10.1109/TVT.2004.825766
|
|
|
| [22] |
LIU X, XU Q Adaptive attention-based high-level semantic introduction for image caption[J]. ACM Transactions on Multimedia Computing, Communications, and Applications, 2020, 16 (4): 1- 22
doi: 10.1145/3409388
|
|
|
| [23] |
LI D, ZHAO Y, CUI R, et al. Research on image text generation based on word2vec visual vocabulary attention [C]// Proceedings of the Asia-Pacific Conference on Communications Technology and Computer Science. Shenyang: IEEE, 2021: 344–348.
|
|
|
| [24] |
ZHAI X, WANG X, MUSTAFA B, et al. LiT: zero-shot transfer with locked-image text tuning [C]// Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition. New Orleans: IEEE, 2022: 18102–18112.
|
|
|
| [25] |
TATARIA H, SHAFI M. Simple framework of ZF precoding analysis for full-dimensional massive MIMO systems [C]// Proceedings of the IEEE 94th Vehicular Technology Conference. Norman: IEEE, 2021: 1–5.
|
|
|
| [1] |
YANG K, WANG S, DAI J, et al SwinJSCC: taming Swin Transformer for deep joint source-channel coding[J]. IEEE Transactions on Cognitive Communications and Networking, 2025, 11 (1): 90- 104
doi: 10.1109/TCCN.2024.3424842
|
|
|
| [2] |
YANG K, WANG S, DAI J, et al. WITT: a wireless image transmission transformer for semantic communications [C]// Proceedings of the ICASSP 2023—2023 IEEE International Conference on Acoustics, Speech and Signal Processing. Rhodes Island: IEEE, 2023: 1–5.
|
|
|
| [3] |
张平, 戴金晟, 张育铭, 等 面向语义通信的非线性变换编码[J]. 通信学报, 2023, 44 (4): 1- 14
ZHANG Ping, DAI Jinsheng, ZHANG Yuming, et al Nonlinear transform coding for semantic communications[J]. Journal on Communications, 2023, 44 (4): 1- 14
doi: 10.11959/j.issn.1000-436x.2023087
|
|
|
| [4] |
陈建侨, 马楠, 许晓东, 等 面向语义通信的信道知识库构建与信道处理研究综述[J]. 无线电通信技术, 2024, 50 (3): 519- 527
CHEN Jianqiao, MA Nan, XU Xiaodong, et al Recent research on channel knowledge base construction and channel processing for semantic communication: a survey[J]. Radio Communications Technology, 2024, 50 (3): 519- 527
doi: 10.3969/j.issn.1003-3114.2024.03.013
|
|
|
| [5] |
HUANG D, TAO X, GAO F, et al. Deep learning-based image semantic coding for semantic communications [C]// Proceedings of the IEEE Global Communications Conference. Madrid: IEEE, 2022: 1–6.
|
|
|
| [6] |
NGUYEN L X, TUN Y L, TUN Y K, et al Swin Transformer-based dynamic semantic communication for multi-user with different computing capacity[J]. IEEE Transactions on Vehicular Technology, 2024, 73 (6): 8957- 8972
doi: 10.1109/TVT.2024.3362328
|
|
|
| [7] |
BOURTSOULATZE E, BURTH KURKA D, GÜNDÜZ D Deep joint source-channel coding for wireless image transmission[J]. IEEE Transactions on Cognitive Communications and Networking, 2019, 5 (3): 567- 579
doi: 10.1109/TCCN.2019.2919300
|
|
|
| [8] |
WANG Y, GAO Z, ZHENG D, et al Transformer-empowered 6G intelligent networks: from massive MIMO processing to semantic communication[J]. IEEE Wireless Communications, 2023, 30 (6): 127- 135
doi: 10.1109/MWC.008.2200157
|
|
|
| [9] |
DOSOVITSKIY A, BEYER L, KOLESNIKOV A, et al. An image is worth 16x16 words: transformers for image recognition at scale [EB/OL]. (2021–06–03)[2025–05–15]. https://arxiv.org/pdf/2010.11929.
|
|
|
| [10] |
HAN K, WANG Y, CHEN H, et al A survey on vision transformer[J]. IEEE Transactions on Pattern Analysis and Machine Intelligence, 2023, 45 (1): 87- 110
doi: 10.1109/TPAMI.2022.3152247
|
|
|
| [11] |
VASWANI A, SHAZEER N, PARMAR N, et al. Attention is all you need [C]// Proceedings of the 31st International Conference on Neural Information Processing Systems. [S.l.]: Curran Associates Inc. , 2017: 6000–6010.
|
|
|
| [12] |
LIU Z, LIN Y, CAO Y, et al. Swin Transformer: hierarchical vision transformer using shifted windows [C]// Proceedings of the IEEE/CVF International Conference on Computer Vision. Montreal: IEEE, 2022: 9992–10002.
|
|
|
| [13] |
LIANG H, BAO Z, AN W, et al. Semantic importance-aware based for multi-user communication over MIMO fading channels [EB/OL]. (2023–12–26)[2025–05–15]. https://arxiv.org/pdf/2312.16057.
|
|
|
| [14] |
PENG K, MENG Y, WU D, et al. Multi-task semantic communication for image transmission over MIMO channel [C]// Proceedings of the 16th International Conference on Wireless Communications and Signal Processing. Hefei: IEEE, 2025: 1468–1473.
|
|
|
|
Viewed |
|
|
|
Full text
|
|
|
|
|
Abstract
|
|
|
|
|
Cited |
|
|
|
|
| |
Shared |
|
|
|
|
| |
Discussed |
|
|
|
|
