用于超快时空编码MRI的Transformer超分辨率重建算法研究

doi:10.11938/cjmr20243110

波谱学杂志 ›› 2024, Vol. 41 ›› Issue (4): 454-468.doi: 10.11938/cjmr20243110

用于超快时空编码MRI的Transformer超分辨率重建算法研究

宁欣宙¹, 黄臻¹, 陈西曲¹, 刘鑫杰²^,³, 陈罡²^,³, 张志²^,³, 鲍庆嘉²^,³^,^*(), 刘朝阳²^,³^,⁴^,^#()

1.武汉轻工大学电气与电子工程学院，湖北武汉 430023
2.波谱与原子分子物理国家重点实验室，武汉磁共振中心（中国科学院精密测量科学与技术创新研究院），湖北武汉 430071
3.中国科学院大学，北京 100049
4.湖北光谷实验室，湖北武汉 430074

收稿日期:2024-04-16 出版日期:2024-12-05 在线发表日期:2024-05-16
通讯作者: * Tel: 027-87199686, E-mail: qingjia.bao@apm.ac.cn;# Tel: 027-87198790, E-mail: chyliu@apm.ac.cn.
基金资助:
国家重点研发计划(2023YFE0113300);国家重点研发计划(2022YFF0707000);国家自然科学基金项目(22327901);中国科学院B类战略性先导科技专项(XDB0540300);湖北省科技创新人才及服务专项(2023EHA003);中国科学院磁共振技术联盟科研仪器设备研制项目(2021GZL001);中国科学院精密测量科学与技术创新研究院交叉培养项目(S21S4101);中国科学院科研仪器开发项目(YJKYYQ 20190032)

Research on Transformer Super-Resolution Reconstruction Algorithm for Ultrafast Spatiotemporal Encoding Magnetic Resonance Imaging

NING Xinzhou¹, HUANG Zhen¹, CHEN Xiqu¹, LIU Xinjie²^,³, CHEN Gang²^,³, ZHANG Zhi²^,³, BAO Qingjia²^,³^,^*(), LIU Chaoyang²^,³^,⁴^,^#()

1. School of Electrical & Electronic Engineering, Wuhan Polytechnic University, Wuhan 430023, China
2. State Key Laboratory of Magnetic Resonance and Atomic and Molecular Physics, National Center for Magnetic Resonance in Wuhan (Innovation Academy for Precision Measurement Science and Technology, Chinese Academy of Sciences), Wuhan 430071, China
3. University of Chinese Academy of Sciences, Beijing 100049, China
4. Optics Valley Laboratory, Wuhan 430074, China

Received:2024-04-16 Published:2024-12-05 Online:2024-05-16
Contact: * Tel: 027-87199686, E-mail: qingjia.bao@apm.ac.cn;# Tel: 027-87198790, E-mail: chyliu@apm.ac.cn.

摘要/Abstract

摘要：

时空编码（SPEN）磁共振成像（MRI）是一种超快MRI技术，通过该技术采集获得的原始图像空间分辨率较低，需要基于序列物理原理进行超分辨率重建以提高其原始图像的分辨率，而现有的基于深度学习SPEN超分辨率重建算法在提取图像像素长距离依赖关系上的能力有限.为了解决此问题，本文提出了一种基于Transformer的SPEN MRI超分辨率重建算法.该算法采用编码器-解码器结构，并引入Transformer模块以提取特征图的局部上下文信息和长距离依赖关系.实验结果表明，本文所提的重建算法可以在不增加额外采样点的情况下从SPEN低分辨率图像中重建出高空间分辨率、无混叠伪影的超分辨率图像.与现有的超分辨率算法相比，本文提出的算法在临床前以及临床数据集上都取得了更好的重建效果.

关键词: 超快磁共振成像, 时空编码, 深度学习, 超分辨率, 图像重建

Abstract:

Spatio-temporal encoding (SPEN) magnetic resonance imaging (MRI) is an ultrafast MRI technique. However, resolution of the original image acquired with SPEN is relatively low, requiring super-resolution reconstruction based on sequence physics principles to improve spatial resolution. As the existing SPEN super-resolution reconstruction algorithms based on deep learning have confined abilities to capture long-range dependencies, this paper proposes a transformer-based SPEN MRI super-resolution reconstruction algorithm. An encoder-decoder structure is adopted, and a transformer module is introduced to extract local context information and long-range dependencies of feature maps. Experimental results show that the proposed reconstruction method can reconstruct a super-resolution image with high spatial resolution and no aliasing artifacts from the low-resolution SPEN image without adding additional sampling points. Compared to the existing super-resolution methods, the proposed method achieves better results on both clinical and preclinical datasets.

Key words: ultrafast MRI, spatio-temporal encoding(SPEN), deep learning, super-resolution, image reconstruction

中图分类号:

O482.53

宁欣宙, 黄臻, 陈西曲, 刘鑫杰, 陈罡, 张志, 鲍庆嘉, 刘朝阳. 用于超快时空编码MRI的Transformer超分辨率重建算法研究[J]. 波谱学杂志, 2024, 41(4): 454-468.

NING Xinzhou, HUANG Zhen, CHEN Xiqu, LIU Xinjie, CHEN Gang, ZHANG Zhi, BAO Qingjia, LIU Chaoyang. Research on Transformer Super-Resolution Reconstruction Algorithm for Ultrafast Spatiotemporal Encoding Magnetic Resonance Imaging[J]. Chinese Journal of Magnetic Resonance, 2024, 41(4): 454-468.

图/表 9

图1

图2

图3

图4

图5

图6

图7

图8

表1

参考文献 38

[1]	EMAM M M, HOUSSEIN E H, GHONIEM R M. A modified reptile search algorithm for global optimization and image segmentation: Case study brain MRI images[J]. Comput Biol Med, 2023, 152: 106404.
[2]	WALTER S S, FRITZ B, KIJOWSKI R, et al. 2D versus 3D MRI of osteoarthritis in clinical practice and research[J]. Skeletal Radiol, 2023, 52(11): 2211-2224. doi: 10.1007/s00256-023-04309-4 pmid: 36907953
[3]	TSAO J. Ultrafast imaging: principles, pitfalls, solutions, and applications[J]. J Magn Reson Imaging, 2010, 32(2): 252-266. doi: 10.1002/jmri.22239 pmid: 20677249
[4]	GIL R, FERNANDES F F, SHEMESH N. Neuroplasticity-driven timing modulations revealed by ultrafast functional magnetic resonance imaging[J]. NeuroImage, 2021, 225: 117446.
[5]	HIRSCH F W, FRAHM J, SORGE I, et al. Real-time magnetic resonance imaging in pediatric radiology—new approach to movement and moving children[J]. Pediatr Radiol, 2021, 51: 840-846.
[6]	MILON A, FLAMENT V, GUENICHE Y, et al. How to optimize breast MRI protocol? The value of combined analysis of ultrafast and diffusion-weighted MRI sequences[J]. Diagn Interv Imaging, 2023, 104(6): 284-291.
[7]	LIAO C, CAO X, CHO J, et al. Highly efficient MRI through multi-shot echo planar imaging[C]// Wavelets and Sparsity XVIII, 2019, 11138: 353-365.
[8]	DUONG S T, PHUNG S L, BOUZERDOUM A, et al. An unsupervised deep learning technique for susceptibility artifact correction in reversed phase-encoding EPI images[J]. Magn Reson Imaging, 2020, 71: 1-10. doi: S0730-725X(19)30732-5 pmid: 32407764
[9]	SHROT Y, FRYDMAN L. Spatially encoded NMR and the acquisition of 2D magnetic resonance images within a single scan[J]. J Magn Reson, 2005, 172(2): 179-190. pmid: 15649744
[10]	CAI S H, CAI C B, CHEN Z. Spatiotemporally encoded nuclear magnetic resonance and its applications[J]. J Xiamen University(Natural Science), 2021, 60(3): 542-554.
	蔡淑惠, 蔡聪波, 陈忠. 时空编码核磁共振方法及其应用[J]. 厦门大学学报(自然科学版), 2021, 60(3): 542-554.
[11]	TAL A, FRYDMAN L. Spatial encoding and the single-scan acquisition of high definition MR images in inhomogeneous fields[J]. J Magn Reson, 2006, 182(2): 179-194. pmid: 16843690
[12]	TAL A, FRYDMAN L. Single-scan multidimensional magnetic resonance[J]. Prog Nucl Magn Reson Spectrosc, 2010, 57(3): 241-292.
[13]	BEN-ELIEZER N, IRANI M, FRYDMAN L. Super-resolved spatially encoded single-scan 2D MRI[J]. Magn Reson Med, 2010, 63(6): 1594-1600.
[14]	CHEN Y, LI J, QU X, et al. Partial Fourier transform reconstruction for single-shot MRI with linear frequency-swept excitation[J]. Magn Reson Med, 2013, 69(5): 1326-1336. doi: 10.1002/mrm.24366 pmid: 22706702
[15]	CAI C, DONG J, CAI S, et al. An efficient de-convolution reconstruction method for spatiotemporal-encoding single-scan 2D MRI[J]. J Magn Reson, 2013, 228: 136-147. doi: 10.1016/j.jmr.2012.12.020 pmid: 23433507
[16]	CHEN L, LI J, ZHANG M, et al. Super-resolved enhancing and edge deghosting (SEED) for spatiotemporally encoded single-shot MRI[J]. Med Image Anal, 2015, 23(1): 1-14. doi: 10.1016/j.media.2015.03.004 pmid: 25910683
[17]	ZHONG S, CHEN M, WEI X, et al. Understanding aliasing effects and their removal in SPEN MRI: A k-space perspective[J]. Magn Reson Med, 2023, 90(1): 166-176. doi: 10.1002/mrm.29638 pmid: 36961093
[18]	LI Y J, YANG X Y, YANG X M. Magnetic resonance image reconstruction of multi-scale residual unet fused with attention mechanism[J]. Chinese J Magn Reson, 2023, 40(3):307-319.
	李奕洁, 杨馨雨, 杨晓梅. 融合注意力机制的多尺度残差Unet的磁共振图像重建[J]. 波谱学杂志, 2023, 40(3): 307-319.
[19]	LU Q Q, LIAN Z F, LI J L, et al. Magnetic resonance R₂^* parameter mapping of liver based on self-supervised deep neural network[J]. Chinese J Magn Reson, 2023, 40(3):258-269.
	陆琪琪, 连梓锋, 李嘉龙. 等. 基于自监督网络的肝脏磁共振R₂^*参数图像重建[J]. 波谱学杂志, 2023, 40(3): 258-269.
[20]	DE LEEUW DEN BOUTER M L, IPPOLITO G, O’REILLY T P, et al. Deep learning-based single image super-resolution for low-field MR brain images[J]. Sci Rep, 2022, 12(1): 6362. doi: 10.1038/s41598-022-10298-6 pmid: 35430586
[21]	SONG L, WANG Q, LIU T, et al. Deep robust residual network for super-resolution of 2D fetal brain MRI[J]. Sci Rep, 2022, 12(1): 406. doi: 10.1038/s41598-021-03979-1 pmid: 35013383
[22]	XIANG P C, CAI C B, WANG J C, et al. Super-resolved reconstruction method for spatiotemporally encoded magnetic resonance imaging based on deep neural network>[J]. Acta Phys Sin, 2022, 71(5):363-371.
	向鹏程, 蔡聪波, 王杰超. 等. 基于深度神经网络的时空编码磁共振成像超分辨率重建方法[J]. 物理学报, 2022, 71(5): 363-371.
[23]	CHEN X, WANG W, HUANG J, et al. Ultrafast water-fat separation using deep learning-based single-shot MRI[J]. Magn Reson Med, 2022, 87(6): 2811-2825.
[24]	CHOI S R, LEE M. Transformer architecture and attention mechanisms in genome data analysis: a comprehensive review[J]. Biology, 2023, 12(7): 1033.
[25]	RAHALI A, AKHLOUFI M A. End-to-end transformer-based models in textual-based NLP[J]. AI, 2023, 4(1): 54-110.
[26]	YIN Y, TANG Z, WENG H. Application of visual transformer in renal image analysis[J]. BioMed Eng OnLine, 2024, 23(1): 27. doi: 10.1186/s12938-024-01209-z pmid: 38439100
[27]	ZU B, CAO T, LI Y, et al. SwinT-SRNet: Swin transformer with image super-resolution reconstruction network for pollen images classification[J]. Eng Appl Artif Intell, 2024, 133: 108041.
[28]	DOSOVITSKIY A, BEYER L, KOLESNIKOV A, et al. An image is worth 16x16 words: Transformers for image recognition at scale[C]// International Conference on Learning Representations, 2021.
[29]	LIU Z, LIN Y, CAO Y, et al. Swin transformer: hierarchical vision transformer using shifted windows[C]// IEEE/CVF International Conference on Computer Vision, 2021: 9992-10002.
[30]	CHEN H, WANG Y, GUO T, et al. Pre-trained image processing transformer[C]// IEEE/CVF Conference on Computer Vision and Pattern Recognition, 2021: 12294-12305.
[31]	LIANG J, CAO J, SUN G, et al. SwinIR: Image restoration using swin transformer[C]// IEEE/CVF International Conference on Computer Vision Workshops, 2021: 833-1844.
[32]	ZAMIR S W, ARORA A, KHAN S, et al. Restormer: Efficient transformer for high-resolution image restoration[C]// IEEE/CVF Conference on Computer Vision and Pattern Recognition, 2022: 5718-5729.
[33]	WANG Z, CUN X, BAO J, et al. Uformer: A general u-shaped transformer for image restoration[C]// IEEE/CVF Conference on Computer Vision and Pattern Recognition, 2022: 17662-17672.
[34]	SCHMIDT R, SEGINER A, FRYDMAN L. Interleaved multishot imaging by spatiotemporal encoding: A fast, self-referenced method for high-definition diffusion and functional MRI[J]. Magn Reson Med, 2016, 75(5): 1935-1948. doi: 10.1002/mrm.25742 pmid: 26108165
[35]	VASWANI A, SHAZEER N, PARMAR N, et al. Attention is all you need[C]// Neural Information Processing Systems, 2017: 6000-6010.
[36]	WU H, XIAO B, CODELLA N, et al. Cvt: Introducing convolutions to vision transformers[C]// IEEE/CVF International Conference on Computer Vision, 2021: 22-31.
[37]	YUAN K, GUO S, LIU Z, et al. Incorporating convolution designs into visual transformers[C]// IEEE/CVF International Conference on Computer Vision, 2021: 559-568.
[38]	VAN ESSEN D C, SMITH S M, BARCH D M, et al. The WU-Minn human connectome project: an overview[J]. NeuroImage, 2013, 80: 62-79. doi: 10.1016/j.neuroimage.2013.05.041 pmid: 23684880

方法	PSNR/dB	SSIM	FLOPS	模型参数/MB
传统迭代重建	31.08	0.8928	-	-
U-Net重建	38.58	0.9812	5.48	5.3
Transformer (C=12)	38.85	0.9801	4.42	3.0
Transformer (C=24)	45.22	0.9928	16.14	11.6
Transformer (C=48)	53.31	0.9983	61.48	46.2
Transformer (T=1)	42.34	0.9877	8.71	6.8
Transformer (T=2)	45.22	0.9928	16.14	11.6
Transformer (T=3)	45.41	0.9928	23.57	16.4
Transformer (Without GDL)	42.83	0.9901	16.14	11.6
Transformer (With GDL)	45.22	0.9928	16.14	11.6

用于超快时空编码MRI的Transformer超分辨率重建算法研究

Research on Transformer Super-Resolution Reconstruction Algorithm for Ultrafast Spatiotemporal Encoding Magnetic Resonance Imaging

RichHTML

PDF

可视化

摘要/Abstract

引用本文

使用本文

图/表 9

参考文献 38

相关文章 15

编辑推荐

Metrics

本文评价

[1]	黄敏, 朱俊琳, 考宇辰, 周到, 唐奇伶. 基于物理模型的ISTAVS-Net多线圈MRI图像重建[J]. 波谱学杂志, 2024, 41(4): 418-429.
[2]	杨黎明, 王远军. 扩散张量图像去噪算法研究进展[J]. 波谱学杂志, 2024, 41(3): 341-361.
[3]	戴俊龙, 何聪, 武杰, 边云. 融合双解码和全局注意力上采样模块的胰腺囊性肿瘤分割网络[J]. 波谱学杂志, 2024, 41(2): 151-161.
[4]	杨钰, 陈博, 吴柳滨, 林恩平, 黄玉清, 陈忠. Laplace NMR谱图重建——从经典正则化到深度学习[J]. 波谱学杂志, 2024, 41(2): 191-208.
[5]	常博, 孙灏芸, 高清宇, 王丽嘉. 传统方法和深度学习用于不同模态心脏医学图像的分割研究进展[J]. 波谱学杂志, 2024, 41(2): 224-244.
[6]	徐真顺, 袁小涵, 黄子珩, 邵成伟, 武杰, 边云. 基于深度学习的胰腺黏液性和浆液性囊性肿瘤的多源特征分类模型[J]. 波谱学杂志, 2024, 41(1): 19-29.
[7]	赖嘉雯, 汪宇玲, 蔡晓宇, 周丽华. 基于CNN-SVM的多维度信息融合半月板撕裂分类方法[J]. 波谱学杂志, 2023, 40(4): 423-434.
[8]	王慧, 王甜甜, 王丽嘉. 基于心脏磁共振电影图像的压缩激励残差U形网络左心肌分割[J]. 波谱学杂志, 2023, 40(4): 435-447.
[9]	李奕洁, 杨馨雨, 杨晓梅. 融合注意力机制的多尺度残差Unet的磁共振图像重建[J]. 波谱学杂志, 2023, 40(3): 307-319.
[10]	陆琪琪, 连梓锋, 李嘉龙, 斯文彬, 麦兆华, 冯衍秋. 基于自监督网络的肝脏磁共振R₂^*参数图像重建[J]. 波谱学杂志, 2023, 40(3): 258-269.
[11]	张嘉骏, 鲁宇澄, 鲍奕仿, 李郁欣, 耿辰, 胡伏原, 戴亚康. 基于DBCNet的TOF-MRA中脑动脉树区域自动分割方法[J]. 波谱学杂志, 2023, 40(3): 320-331.
[12]	田慧, 武杰, 边云, 张志伟, 邵成伟. 基于DenseNet结合迁移学习的胰腺囊性肿瘤分类方法[J]. 波谱学杂志, 2023, 40(3): 270-279.
[13]	钱程一,王远军. 基于深度学习的阿尔兹海默症影像学分类研究进展[J]. 波谱学杂志, 2023, 40(2): 220-238.
[14]	黄敏,李思怡,陈军波,周到. 磁共振指纹成像技术及临床应用的进展[J]. 波谱学杂志, 2023, 40(2): 207-219.
[15]	施伟成,金朝阳,叶铮. 基于PCAU-Net的快速多通道磁共振成像方法[J]. 波谱学杂志, 2023, 40(1): 39-51.