|
|
Multi-scale parallel magnetic resonance imaging reconstruction based on variational model and Transformer |
Jizhong DUAN( ),Haiyuan LI |
Faculty of Information Engineering and Automation, Kunming University of Science and Technology, Kunming 650504, China |
|
|
Abstract A multi-scale parallel MRI reconstruction model based on a variational model and Transformer (VNTM) was proposed, to enhance the quality of reconstructed MR images from undersampled multi-coil MR data. First, undersampled multi-coil k-space data were used to estimate sensitivity maps, with an intermediate-stage enhancement strategy applied to improve the accuracy of these maps. Next, the undersampled multi-coil k-space data and estimated sensitivity maps were input into a variational model for reconstruction. In the variational model, resolution was reduced through a pre-processing module to reduce computational load; multi-scale features were then effectively fused through a multi-scale U-shaped network with the Transformer. Finally, a post-processing module was applied to restore resolution, and data consistency operations were performed on the output to ensure fidelity. Extensive quantitative and qualitative experiments were conducted on publicly available datasets to validate the effectiveness of the proposed method. The experimental results indicate that the proposed reconstruction model achieves superior reconstruction quality and more stable performance in terms of peak signal-to-noise ratio, structural similarity, and visual effects. In addition, a series of ablation studies and robustness evaluations with varying auto-calibration signal (ACS) region sizes were carried out, confirming that VNTM maintained consistently high reconstruction performance under diverse conditions.
|
Received: 29 August 2024
Published: 25 August 2025
|
|
Fund: 国家自然科学基金地区科学基金资助项目(61861023);云南省基础研究计划资助项目(202301AT070452). |
基于变分模型和Transformer的多尺度并行磁共振成像重建
为了提高欠采样并行磁共振成像的图像重建质量,提出基于变分模型和Transformer的多尺度并行磁共振成像重建模型(VNTM). 该模型利用欠采样多线圈k空间数据来估计灵敏度图,并利用中期增强策略以提高灵敏度图的准确性. 将欠采样多线圈k空间数据和估计的灵敏度图输入变分模型进行重建,在变分模型中,通过前处理模块对图像数据进行降分辨率处理,以减少计算负担. 通过具有Transformer的多尺度U型网络,实现多尺度特征的有效融合. 使用后处理模块恢复分辨率,并对输出数据进行数据一致性操作以确保保真度. 在公开数据集进行大量定量和定性实验以验证所提方法的有效性. 结果表明,在峰值信噪比、结构相似度和视觉效果方面,所提出的重建模型均表现出更优的重建质量和更稳定的重建性能. 多组消融实验和不同自校准信号(ACS)区域大小的鲁棒性实验,验证了VNTM在不同条件下均能保持良好的重建性能.
关键词:
并行磁共振成像,
深度学习,
图像重建,
灵敏度图,
小波变换
|
|
[1] |
PRUESSMANN K P Encoding and reconstruction in parallel MRI[J]. NMR in Biomedicine, 2006, 19 (3): 288- 299
doi: 10.1002/nbm.1042
|
|
|
[2] |
LARKMAN D J, NUNES R G Parallel magnetic resonance imaging[J]. Physics in Medicine and Biology, 2007, 52 (7): R15
doi: 10.1088/0031-9155/52/7/R01
|
|
|
[3] |
LUSTIG M, PAULY J M SPIRiT: iterative self-consistent parallel imaging reconstruction from arbitrary k-space[J]. Magnetic Resonance in Medicine, 2010, 64 (2): 457- 471
doi: 10.1002/mrm.22428
|
|
|
[4] |
DONOHO D L Compressed sensing[J]. IEEE Transactions on Information Theory, 2006, 52 (4): 1289- 1306
doi: 10.1109/TIT.2006.871582
|
|
|
[5] |
LUSTIG M, DONOHO D, PAULY J M Sparse MRI: the application of compressed sensing for rapid MR imaging[J]. Magnetic Resonance in Medicine, 2007, 58 (6): 1182- 1195
doi: 10.1002/mrm.21391
|
|
|
[6] |
PRUESSMANN K P, WEIGER M, SCHEIDEGGER M B, et al SENSE: sensitivity encoding for fast MRI[J]. Magnetic Resonance in Medicine, 1999, 42 (5): 952- 962
doi: 10.1002/(SICI)1522-2594(199911)42:5<952::AID-MRM16>3.0.CO;2-S
|
|
|
[7] |
UECKER M, LAI P, MURPHY M J, et al ESPIRiT: an eigenvalue approach to autocalibrating parallel MRI: where SENSE meets GRAPPA[J]. Magnetic Resonance in Medicine, 2014, 71 (3): 990- 1001
doi: 10.1002/mrm.24751
|
|
|
[8] |
LUSTIG M, DONOHO D L, SANTOS J M, et al Compressed sensing MRI[J]. IEEE Signal Processing Magazine, 2008, 25 (2): 72- 82
doi: 10.1109/MSP.2007.914728
|
|
|
[9] |
RUDIN L I, OSHER S, FATEMI E Nonlinear total variation based noise removal algorithms[J]. Physica D: Nonlinear Phenomena, 1992, 60 (1/2/3/4): 259- 268
|
|
|
[10] |
RAVISHANKAR S, BRESLER Y. Sparsifying transform learning for compressed sensing MRI [C]// IEEE 10th International Symposium on Biomedical Imaging. San Francisco: IEEE, 2013: 17–20.
|
|
|
[11] |
ZHANG K, ZUO W, CHEN Y, et al Beyond a Gaussian denoiser: residual learning of deep CNN for image denoising[J]. IEEE Transactions on Image Processing, 2017, 26 (7): 3142- 3155
doi: 10.1109/TIP.2017.2662206
|
|
|
[12] |
LIU G, REDA F A, SHIH K J, et al. Image inpainting for irregular holes using partial convolutions [C]// European Conference on Computer Vision. Cham: Springer International Publishing, 2018: 89–105.
|
|
|
[13] |
LIM B, SON S, KIM H, et al. Enhanced deep residual networks for single image super-resolution [C]// Computer Vision and Pattern Recognition Workshops. Honolulu: IEEE, 2017: 1132–1140.
|
|
|
[14] |
WU Z, LIAO W, YAN C, et al Deep learning based MRI reconstruction with transformer[J]. Computer Methods and Programs in Biomedicine, 2023, 233: 107452
doi: 10.1016/j.cmpb.2023.107452
|
|
|
[15] |
HAMMERNIK K, KLATZER T, KOBLER E, et al Learning a variational network for reconstruction of accelerated MRI data[J]. Magnetic Resonance in Medicine, 2018, 79 (6): 3055- 3071
doi: 10.1002/mrm.26977
|
|
|
[16] |
SRIRAM A, ZBONTAR J, MURRELL T, et al. End-to-end variational networks for accelerated MRI reconstruction [C]// Medical Image Computing and Computer Assisted Intervention. Cham: Springer, 2020: 64–73.
|
|
|
[17] |
YIASEMIS G, SONKE J J, SÁNCHEZ C, et al. Recurrent variational network: a deep learning inverse problem solver applied to the task of accelerated MRI reconstruction [C] // IEEE/CVF Conference on Computer Vision and Pattern Recognition. New Orleans: IEEE, 2022: 722–731.
|
|
|
[18] |
GUO P, MEI Y, ZHOU J, et al ReconFormer: accelerated MRI reconstruction using recurrent transformer[J]. IEEE Transactions on Medical Imaging, 2024, 43 (1): 582- 593
doi: 10.1109/TMI.2023.3314747
|
|
|
[19] |
HUANG J, XING X, GAO Z, et al. Swin deformable attention U-Net transformer (SDAUT) for explainable fast MRI [C]// Medical Image Computing and Computer Assisted Intervention. Cham: Springer, 2022: 538–548.
|
|
|
[20] |
HUANG J, FANG Y, WU Y, et al Swin transformer for fast MRI[J]. Neurocomputing, 2024, 493: 281- 304
|
|
|
[21] |
WANG B, LIAN Y, XIONG X, et al DCT-net: dual-domain cross-fusion transformer network for MRI reconstruction[J]. Magnetic Resonance Imaging, 2024, 107: 69- 79
doi: 10.1016/j.mri.2024.01.007
|
|
|
[22] |
LIU X, XU W, YE X. The ill-posed problem and regularization in parallel magnetic resonance imaging [C]// 3rd International Conference on Bioinformatics and Biomedical Engineering. Beijing: IEEE, 2009: 1–4.
|
|
|
[23] |
ROEMER P B, EDELSTEIN W A, HAYES C E, et al The NMR phased array[J]. Magnetic Resonance in Medicine, 1990, 16 (2): 192- 225
doi: 10.1002/mrm.1910160203
|
|
|
[24] |
LIANG J, CAO J, SUN G, et al. SwinIR: image restoration using swin transformer [C]// International Conference on Computer Vision Workshops. Montreal: IEEE, 2021: 1833–1844.
|
|
|
[25] |
FABIAN Z, TINAZ B, SOLTANOLKOTABI M. HUMUS-Net: hybrid unrolled multi-scale network architecture for accelerated MRI reconstruction [EB/OL]. (2023-03-17)[2024-08-01]. https://arxiv.org/abs/2203.08213.
|
|
|
[26] |
LAI Z, QU X, LIU Y, et al Image reconstruction of compressed sensing MRI using graph-based redundant wavelet transform[J]. Medical Image Analysis, 2016, 27: 93- 104
doi: 10.1016/j.media.2015.05.012
|
|
|
[27] |
KNOLL F, ZBONTAR J, SRIRAM A, et al fastMRI: a publicly available raw k-space and DICOM dataset of knee images for accelerated MR image reconstruction using machine learning[J]. Radiology Artificial Intelligence, 2020, 2 (1): e190007
doi: 10.1148/ryai.2020190007
|
|
|
[28] |
PASZKE A, GROSS S, MASSA F, et al. PyTorch: an imperative style, high-performance deep learning library [EB/OL]. (2019-12-03)[2024-08-01]. https://arxiv.org/abs/1912.01703.
|
|
|
[29] |
HORÉ A, ZIOU D. Image quality metrics: psnr vs. SSIM [C]// 20th International Conference on Pattern Recognition. Istanbul: IEEE, 2010: 2366–2369.
|
|
|
[30] |
WANG Z, BOVIK A C, SHEIKH H R, et al Image quality assessment: from error visibility to structural similarity[J]. IEEE Transactions on Image Processing, 2004, 13 (4): 600- 612
doi: 10.1109/TIP.2003.819861
|
|
|
[31] |
PRAMANIK A, AGGARWAL H K, JACOB M Deep generalization of structured low-rank algorithms (deep-SLR)[J]. IEEE Transactions on Medical Imaging, 2020, 39 (12): 4186- 4197
doi: 10.1109/TMI.2020.3014581
|
|
|
[32] |
WANG S, CHENG H, YING L, et al Deep complexMRI: exploiting deep residual network for fast parallel MR imaging with complex convolution[J]. Magnetic Resonance Imaging, 2020, 68: 136- 147
doi: 10.1016/j.mri.2020.02.002
|
|
|
|
Viewed |
|
|
|
Full text
|
|
|
|
|
Abstract
|
|
|
|
|
Cited |
|
|
|
|
|
Shared |
|
|
|
|
|
Discussed |
|
|
|
|