|
|
|
| Medical image fusion based on dual-stage attention mechanism and invertible transformation |
Yu SHEN( ),Zhen YUAN( ),Jiaying LIU,Shan BAI,Zhenkai QIANG |
| School of Electronic and Information Engineering, Lanzhou Jiaotong University, Lanzhou 730070, China |
|
|
|
Abstract A new fusion method based on a dual-stage attention mechanism (DSAM) and invertible transformation was proposed aiming at the problem that existing medical image fusion methods suffer from the loss of global information, insufficient complementary information integration, and poor contrast and clarity of fused images. A DSAM was introduced into the low-frequency and high-frequency branches after wavelet decomposition in order to enhance shallow global features and deep local features, respectively. An IBRC Block was employed to achieve bidirectional and invertible integration of complementary information. A multi-modal feature weighted fusion unit (MM_FWFU) was used to complete feature aggregation. An edge detail enhancement network (EDEN) was applied to improve the detail clarity and edge contrast of the fused images. The experimental results demonstrated that the proposed method achieved improvements of 1.99%, 13.75%, 0.34% and 0.10% in standard deviation (SD), average gradient (AG), correlation coefficient (CC) and multi-scale structural similarity index measure (MS-SSIM), respectively, compared with the best-performing results among nine state-of-the-art methods. The proposed method effectively preserved deep local features from the source images, producing fused images with clear contours, high contrast and rich detail information, and exhibited strong generalization capability.
|
|
Received: 16 September 2025
Published: 21 October 2026
|
|
|
| Fund: 国家自然科学基金资助项目(62241106,61861025); 甘肃省重点研发计划资助项目(甘科计[2024]10号-24YFGA037);甘肃省科技专员专项资助项目(甘科计[2023]18号-23CXGA0008);“智慧天路”建设重大专项资助项目(2023QZzhtl1102);兰州局集团公司科技研究开发计划资助项目(LZJKY2025107-1);中国国家铁路集团有限公司重点课题(N2023X050);兰州交通大学重点研发资助项目(LZJTU-ZDYF2305). |
基于双阶段注意力机制与可逆变换的医学图像融合
针对现有医学图像融合方法中全局信息易丢失、互补信息融合不充分及融合图像对比度与清晰度差的问题,提出基于双阶段注意力机制(DSAM)与可逆变换的新型融合方法. 在小波分解后的低频与高频分支中引入双阶段注意力模块,分别用于增强浅层全局特征与深层局部特征. 采用可逆瓶颈残差耦合模块(IBRC Block)实现双向可逆的互补信息整合. 通过多模态特征加权融合单元(MM_FWFU)完成特征整合. 通过边缘细节增强网络(EDEN)提升融合图像的细节清晰度和边缘对比度. 实验结果表明,所提方法在标准差(SD)、平均梯度(AG)、相关系数(CC)和多尺度结构相似性(MS-SSIM)等指标上较9种主流方法中的最优值分别提升了1.99%、13.75%、0.34%和0.10%. 利用所提方法,能够有效地保留源图像的深层局部特征,融合结果的轮廓清晰,对比度高,细节信息充分,且具有较强的泛化能力.
关键词:
医学图像融合,
双阶段注意力机制(DSAM),
可逆瓶颈残差,
边缘细节增强
|
|
| [1] |
沈瑜, 白珊, 魏子易, 等 基于跨模态感知和空频交叉的医学图像融合[J]. 中国激光, 2025, 52 (9): 0810001 Shen Yu, Bai Shan, Wei Ziyi, et al Medical image fusion network for Cross-Modality perception and Spatial-Frequency interaction[J]. Chinese Journal of Lasers, 2025, 52 (9): 0810001
|
|
|
| [2] |
沈瑜, 刘佳英, 闫佳荣, 等 基于局部与全局信息协同的双分支多模态医学图像融合[J]. 光学学报, 2025, 45 (8): 117- 131 Shen Yu, Liu Jiaying, Yan Jiarong, et al Dual-branch multimodal medical image fusion based on local and global information collaboration[J]. Acta Optica Sinica, 2025, 45 (8): 117- 131
|
|
|
| [3] |
Qiang Z, Shen Y, Yuan Y, et al DWSFusion: dual weight supervision for lightweight infrared and visible image fusion[J]. Pattern Recognition, 2026, 179: 113520
doi: 10.1016/j.patcog.2026.113520
|
|
|
| [4] |
Satyanarayana V, Mohanaiah P Multimodal medical image fusion in NSST domain in coupled neural systems[J]. International Journal of Computational Intelligence Systems, 2025, 18 (1): 111
doi: 10.1007/s44196-025-00841-4
|
|
|
| [5] |
沈瑜, 强振凯, 魏子易, 等 基于深度学习的红外与可见光图像融合综述: 发展与展望[J]. 控制与决策, 2025, 40 (6): 1793- 1806 Shen Yu, Qiang Zhenkai, Wei Ziyi, et al Review of infrared and visible image fusion based on deep learning: developments and prospects[J]. Control and Decision, 2025, 40 (6): 1793- 1806
doi: 10.13195/j.kzyjc.2024.1037
|
|
|
| [6] |
Wang Y M, Wang C Y, Liu K Y, et al CNN-based cross-modality fusion for enhanced breast cancer detection using mammography and ultrasound[J]. Tomography, 2024, 10 (12): 2038- 2057
doi: 10.3390/tomography10120145
|
|
|
| [7] |
Xie X Y, Zhang X Z, Tang X L, et al MACTFusion: lightweight cross transformer for adaptive multimodal medical image fusion[J]. IEEE Journal of Biomedical and Health Informatics, 2025, 29 (5): 3317- 3328
doi: 10.1109/JBHI.2024.3391620
|
|
|
| [8] |
Wang Y H, Liu R R, Li Z H, et al Variable augmented network for invertible modality synthesis and fusion[J]. IEEE Journal of Biomedical and Health Informatics, 2023, 27 (6): 2898- 2909
doi: 10.1109/JBHI.2023.3257544
|
|
|
| [9] |
Zhao Z X, Bai H W, Zhang J S, et al. CDDFuse: correlation-driven dual-branch feature decomposition for multi-modality image fusion [C]//Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition. Vancouver: IEEE, 2023: 5906–5916.
|
|
|
| [10] |
He D, He X P, Xiang H L, et al Wavelet transform-based two-stream convolutional networks for face anti-spoofing[J]. Journal of Electronic Imaging, 2023, 32 (1): 013015
doi: 10.1117/1.jei.32.1.013015
|
|
|
| [11] |
Wang Z, Simoncelli E P, Bovik A C. Multiscale structural similarity for image quality assessment [C]//Proceedings of the 37th Asilomar Conference on Signals, Systems and Computers. Pacific Grove: IEEE, 2004: 1398–1402.
|
|
|
| [12] |
Ma J Y, Tang L F, Fan F, et al SwinFusion: cross-domain long-range learning for general image fusion via swin transformer[J]. IEEE/CAA Journal of Automatica Sinica, 2022, 9 (7): 1200- 1217
doi: 10.1109/JAS.2022.105686
|
|
|
| [13] |
Li B, Peng H, Luo X H, et al Medical image fusion method based on coupled neural P systems in nonsubsampled shearlet transform domain[J]. International Journal of Neural Systems, 2021, 31 (1): 2050050
doi: 10.1142/S0129065720500501
|
|
|
| [14] |
Kong W W, Zhang L J, Lei Y Novel fusion method for visible light and infrared images based on NSST-SF-PCNN[J]. Infrared Physics and Technology, 2014, 65: 103- 112
doi: 10.1016/j.infrared.2014.04.003
|
|
|
| [15] |
Li H, Su D, Cai Q, et al BSAFusion: a bidirectional stepwise feature alignment network for unaligned medical image fusion[J]. Proceedings of the AAAI Conference on Artificial Intelligence, 2025, 39 (5): 4725- 4733
doi: 10.1609/aaai.v39i5.32499
|
|
|
| [16] |
Wang X X, Fang L X, Zhao J L, et al MMAE: a universal image fusion method via mask attention mechanism[J]. Pattern Recognition, 2025, 158: 111041
doi: 10.1016/j.patcog.2024.111041
|
|
|
| [17] |
Sun Y C, Dong M L, Yu M X, et al MBHFuse: a multi- branch heterogeneous global and local infrared and visible image fusion with differential convolutional amplification features[J]. Optics and Laser Technology, 2025, 181: 111666
doi: 10.1016/j.optlastec.2024.111666
|
|
|
| [18] |
Zhao Z, Bai H, Zhang J, et al. Equivariant multi-modality image fusion [C]//Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition. Seattle: IEEE, 2024: 25912–25921.
|
|
|
| [19] |
Li H, Wu X J CrossFuse: a novel cross attention mechanism based infrared and visible image fusion approach[J]. Information Fusion, 2024, 103: 102147
doi: 10.1016/j.inffus.2023.102147
|
|
|
| [20] |
Tang W, He F Z FATFusion: a functional–anatomical transformer for medical image fusion[J]. Information Processing and Management, 2024, 61 (4): 103687
doi: 10.1016/j.ipm.2024.103687
|
|
|
|
Viewed |
|
|
|
Full text
|
|
|
|
|
Abstract
|
|
|
|
|
Cited |
|
|
|
|
| |
Shared |
|
|
|
|
| |
Discussed |
|
|
|
|