Magnetic Resonance Image Reconstruction of Multi-scale Residual Unet Fused with Attention Mechanism

doi:10.11938/cjmr20223040

Abstract

Abstract:

This paper integrates the attention mechanism and multi-scale residual convolution to construct a Unet network, aiming at improving the quality of magnetic resonance image (MRI) reconstructed from under-sampled k-space data. To enhance the feature representation ability of the network and prevent gradient disappearance and degradation during network training, multi-scale residual convolution was embedded in the encoding path of the Unet network to extract different scale feature information of MRI. Moreover, to accurately recover the detailed texture features of MRI, the convolution attention module was embedded in the jump connection part between the encoding and decoding path of the Unet network to respond to the key information, such as details and textures in different degrees. Experiments showed that the proposed network could effectively reconstruct high-quality MRIs with clear texture and without overlapping artifacts from the under-sampled k-space data.

Key words: magnetic resonance imaging, image reconstruction, Unet, attention mechanism, deep learning

CLC Number:

TP391.41

Li Yijie, YANG Xinyu, YANG Xiaomei. Magnetic Resonance Image Reconstruction of Multi-scale Residual Unet Fused with Attention Mechanism[J]. Chinese Journal of Magnetic Resonance, 2023, 40(3): 307-319.

Figures/Tables 15

Fig. 1

Fig. 2

Fig. 3

Fig. 4

Fig. 5

Fig. 6

Fig. 7

Fig. 8

Table 1

Table 2

Fig. 9

Fig. 10

Table 3

Fig. 11

Table 4

References 34

[1]	LIU Y Y, YANG Y X, ZHU Q Y, et al. Accelerating T₁_ρ dispersion imaging with multiple relaxation signal compensation[J]. Chinese J Magn Reson, 2022, 39(3): 243-257.
	刘元元, 杨育昕, 朱庆永, 等. 基于多弛豫信号补偿的快速磁共振T₁_ρ散布成像[J]. 波谱学杂志, 2022, 39(3): 243-257.
[2]	DONOHO D L. Compressed sensing[J]. IEEE Trans Inform Theory, 2006, 52(4): 1289-1306. doi: 10.1109/TIT.2006.871582
[3]	刘晓晖. 基于压缩感知的磁共振图像稀疏重建算法研究[D]. 广州: 南方医科大学, 2018.
[4]	MICHAEL L, DONOHO D L, PAULY J M. Sparse MRI: the application of compressed sensing for rapid MR imaging[J]. Magn Reson Med, 2007, 58(6): 1182-1195. doi: 10.1002/mrm.21391 pmid: 17969013
[5]	GOPI V P, PALANISAMY P, WAHID K A, et al. Multiple regularization based MRI reconstruction[J]. Signal Process, 2014, 103: 103-113. doi: 10.1016/j.sigpro.2013.11.001
[6]	YU Y, HONG M, LIU F, et al. Compressed sensing MRI using singular value decomposition based sparsity basis[C]// Annual International Conference of the IEEE Engineering in Medicine and Biology Society. Boston, USA, IEEE, 2011: 5734-5737.
[7]	HUANG J, ZHANG S, METAXAS D. Efficient MR image reconstruction for compressed MR imaging[J]. Med Image Anal, 2011, 15: 670-679. doi: 10.1016/j.media.2011.06.001 pmid: 21742542
[8]	MA S, YIN W, ZHANG Y. An efficient algorithm for compressed MR imaging using total variation and wavelets[C]// IEEE Conference on Computer Vision and Pattern Recognition, Anchorage, AK, USA: IEEE, 2008: 1-8.
[9]	CHEN Z, FU Y, XIANG Y, et al. A novel low-rank model for MRI using theredundant wavelet tight frame[J]. Neurocomputing, 2018: S0925231218301310.
[10]	RAVISHANKAR S, BRESLER Y. MR image reconstruction from highly undersampled k-space data by dictionary learning[J]. IEEE T Med Imaging, 2010, 30(5): 1028-1041. doi: 10.1109/TMI.2010.2090538
[11]	IKRAM S, ZUBAIR S, SHAH J A, et al. Enhancing MR image reconstruction using block dictionary learning[J]. IEEE Access, 2019, 7: 158434-158444. doi: 10.1109/ACCESS.2019.2949917
[12]	QU X, CAO X, GUO D, et al. Combined sparsifying transforms for compressed sensing MRI[J]. Electron Lett, 2010, 46(2): 121-123. doi: 10.1049/el.2010.1845
[13]	KNOLL F, BREDIES K, POCK T, et al. Second order total generalized variation (tgv) for MRI[J]. Magn Reson Med, 2011, 65(2): 480-491. doi: 10.1002/mrm.22595 pmid: 21264937
[14]	YANG X M, MEI Y B, HU X Y, et al. Compressed sensing MRI by integrating deep denoiser and weighted schatten P-norm minimization[J]. IEEE Signal Proc Lett, 2022, 29: 21-25. doi: 10.1109/LSP.2021.3122338
[15]	JIANG M F, YUAN Z H, YANG X, et al. Accelerating CS-MRI reconstruction with fine-tuning wasserstein generative adversarial network[J]. IEEE Access, 2019, 7: 152347-152357. doi: 10.1109/Access.6287639
[16]	LI G, LV J, WANG C. A modified generative adversarial network using spatial and channel-wise attention for CS-MRI reconstruction[J]. IEEE Access, 2021, 9: 83185-83198. doi: 10.1109/ACCESS.2021.3086839
[17]	OLAF R, PHILIPP F, THOMAS B. U-net: convolutional networks for biomedical image segmentation[C]// Proceedings of the 2015 Medical Imaging Computing and Computer-Assisted Intervention. Munich, Germany: Springer, Cham, 2015: 234-241.
[18]	LIAN Q S, FU L P, CHEN S Z, et al. A compressed sensing algorithm based on multi-scale residual reconstruction network[J]. Acta Automatica Sinica, 2019(11): 2082-2091.
	练秋生, 富利鹏, 陈书贞, 等. 基于多尺度残差网络的压缩感知重构算法[J]. 自动化学报, 2019, 45(11): 2082-2091
[19]	DAI Z X, LI J X, ZHANG X D, et al. Super-resolution reconstruction of MRI based on DNGAN[J]. Computer Science, 2022, 49(7): 113-119.
	戴朝霞, 李锦欣, 张向东, 等. 基于DNGAN的磁共振图像超分辨率重建算法[J]. 计算机科学, 2022, 49(07): 113-119.
[20]	CHAN S H, WANG X, ELGENDY O A. Plug-and-Play ADMM for image restoration: fixed-point convergence and applications[J]. IEEE T Compu Imag, 2017, 3(1): 84-98.
[21]	ZHANG J, GHANEM B. ISTA-Net: Interpretable optimization-inspired deep network for image compressive sensing[C]// IEEE/CVF Conference on Computer Vision and Pattern Recognition, Salt Lake City, USA: IEEE, 2018: 1828-1837.
[22]	LIU Y, LIU Q, ZHANG M, et al. IFR-Net: Iterative feature refinement network for compressed sensing MRI[J]. IEEE T Comput Imag, 6, 434-446.
[23]	SHAH K D, PATEL D K, PATEL H A, et al. EMED-UNet: An efficient multi-encoder-decoder based UNet for chest X-ray segmentation[C]// 2022 IEEE Region 10 Symposium (TENSYMP), Mumbai, India, IEEE, 2022: 1-6.
[24]	HONG L T T, THANH N C, LONG T Q, CRF-EfficientUNet: An improved UNet framework for polyp segmentation in colonoscopy images with combined asymmetric loss function and CRF-RNN layer[J]. IEEE Access, 2021, 9: 156987-157001. doi: 10.1109/ACCESS.2021.3129480
[25]	RAMZI Z, CIUCIU P, STARCK J L. Benchmarking MRI reconstruction neural networks on large public datasets[J]. Appl Sci, 2020, 10, 1816. doi: 10.3390/app10051816
[26]	JETHI A K, MURUGESAN B, RAM K, et al. Dual-encoder-U-net for fast MRI reconstruction[C]// 2020 IEEE 17th International Symposium on Biomedical Imaging Workshops (ISBI Workshops), Iowa City, USA: IEEE, 2020: 1-4.
[27]	ZHOU Z, SIDDIQUEE M M R, TAJBAKHSH N, et al. UNet++: Redesigning skip connections to exploit multiscale features in image segmentation[J]. IEEE T Med Imaging, 39(6): 1856-1867. doi: 10.1109/TMI.42
[28]	HE K M, ZHANG X Y, REN S Q, et al. Deep residual learning for image recognition[C]// 2016 IEEE Conference on Computer Vision and Pattern Recognition (CVPR), Las Vegas, USA: IEEE, 2016: 770-778.
[29]	ZBONTAR J, KNOLL F, SRIRAM A, et al. fastMRI: An open dataset and benchmarks for accelerated MRI[J]. arXiv preprint arXiv:1811.08839, 2018.
[30]	LI Y Y, LI L, LI X S, et al. 3D Dynamic MRI with Homotopic l₀ Minimization Reconstruction[J]. Chinese J Magn Reson, 2022, 39(01): 20-32.
	李嫣嫣, 李律, 李雪松, 等. 基于同伦l₀范数最小化重建的三维动态磁共振成像[J]. 波谱学杂志, 2022, 39(01): 20-32.
[31]	CHENG J, WANG H, YING L, et al. Model learning: Primal dual networks for fastMR imaging[C]// Medical Image Computing and Computer Assisted Intervention - MICCAI 2019, Springer, Cham, 2019: 21-29.
[32]	SCHLEMPER J, CABALLERO J, HAJNAL J V, et al. A deep cascade of convolutional neural networks for MR image reconstruction[C]// International conference on information processing in medical imaging, Springer, Cham, 2017: 647-658.
[33]	EO T, JUN Y, KIM T, et al. KIKI-net: cross-domain convolutional neural networks for reconstructing undersampled magnetic resonance images[J]. Magn Reson Med, 2018, 80(5): 2188-2201. doi: 10.1002/mrm.27201 pmid: 29624729
[34]	RAMANARAYANAN S, MURUGESAN B, RAM K, et al. DC-WCNN: A deep cascade of wavelet based convolutional neural networks for MR image reconstruction[C]// 2020 IEEE 17th International Symposium on Biomedical Imaging, Iowa City, USA: IEEE, 2020: 1069-1073.

网络模型	PSNR			SSIM
网络模型	×2	×3	×4	×2	×3	×4
PD-net	32.5438±1.0981	30.9821±1.1230	30.6958±1.0923	0.7736±0.0812	0.6802±0.0800	0.6345±0.0813
Cascade-net	31.5583±1.1120	30.1253±1.096	29.8148±1.0220	0.7651±0.0795	0.6752±0.0820	0.6340±0.0760
KIKI-net	30.7728±1.4592	28.6321±1.3201	29.7129±1.5329	0.7532±0.0821	0.6457±0.0832	0.6274±0.0846
DC-WCNN	32.8576±1.0786	30.9972±1.2356	30.7214±1.0643	0.7821±0.0831	0.6853±0.0801	0.6399±0.0842
AttMRes-Unet	33.2185±1.2759	31.7255±1.1794	30.9175±1.2376	0.7862±0.0862	0.6972±0.0859	0.6497±0.0865

网络模型	参数量	训练时间/s	重建时间/ms
PD-net	318280	9024	487.5
Cascade-net	424570	3597	70.1
KIKI-net	890504	6302	186.8
DC-WCNN	534080	4809	32.9
AttMRes-Unet	404714	2301	21.1

网络模型	PSNR			SSIM
网络模型	×2	×3	×4	×2	×3	×4
Unet	32.5193±1.0901	31.3347±1.0899	30.8267±1.0921	0.7771±0.0761	0.6829±0.0770	0.6475±0.0769
Att-Unet	32.9367±1.1096	31.5930±1.2019	30.9052±1.1099	0.7777±0.0799	0.6887±0.0811	0.6485±0.0787
MRes-Unet	33.0847±0.9998	31.6665±1.0621	30.9085±1.2260	0.7849±0.0814	0.6959±0.0816	0.6445±0.0813
SAMRes-Unet	33.1018±1.3921	31.6770±1.2998	30.9095±1.2975	0.7852±0.0824	0.6953±0.0827	0.6466±0.0825
CAMRes-Unet	33.1115±1.3001	31.6801±1.2645	30.9106±1.2301	0.7856±0.0831	0.6960±0.0829	0.6476±0.0833
AttMRes-Unet	33.2185±1.2759	31.7255±1.1794	30.9175±1.2376	0.7862±0.0862	0.6972±0.0859	0.6497±0.0865

网络模型	参数量	训练时间/s	重建时间/ms
Unet	485065	2498	19.4
Att-Unet	490730	2506	20.6
MRes-Unet	399049	2240	19.9
SAMRes-Unet	399058	2253	20.1
CAMRes-Unet	404705	2289	20.2
AttMRes-Unet	404714	2301	21.1