融合多重自注意力和可变形卷积的多模态脑胶质瘤分割

doi:10.11938/cjmr20233059

摘要/Abstract

摘要：

脑胶质瘤的磁共振图像分割对于脑肿瘤的诊断、手术规划以及放疗等治疗方案的确定具有非常重要的意义．针对现有脑肿瘤分割算法分割精度不高边缘分割不精确，易出现假阳性的问题，本文提出一种基于多重自注意力和可变形卷积的Unet改进模型．模型将原始Unet框架的标准卷积替换为残差模块，以防止模型训练过程中出现梯度消失；通过在瓶颈层加入基于Transformer的多重自注意力模块来提取局部特征和全局上下文信息，以更好地挖掘像素间的相关性；在跨层连接处采用可变形卷积来增强模型对形状感知的敏感性，以提升肿瘤边缘特征的提取能力．实验结果表明，所提算法的分割结果评价指标高于使用同样数据集的其他对比模型，而且对肿瘤边缘的分割更加精确．这表明本文算法是一种有效的脑胶质瘤自动分割算法．

关键词: 图像分割, 脑胶质瘤, Unet, Transformer自注意力, 可变形卷积

Abstract:

The magnetic resonance image segmentation of glioma is significant in disease diagnosis, surgical planning, and determination of treatment plans such as radiotherapy. In response to the problem of low segmentation accuracy, inaccurate edge segmentation, and prone to false positives in existing brain tumor segmentation algorithms, an improved Unet model based on multi-head self-attention and deformable convolution is proposed. The model replaces the standard convolution of the original Unet framework with residual modules to prevent vanishing gradient during model training. Multi-head self-attention modules based on Transformer are added in the bottleneck layer to extract local features and global context information for better exploration of correlations between pixels. Deformable convolution is used at cross-layer connections to enhance the model's sensitivity to shape perception and improve the ability to extract tumor edge features. Experimental results show that the segmentation evaluation metrics of the proposed algorithm are higher than those of other comparative literature and models using the same dataset, with more precise segmentation of tumor edges. This indicates that the algorithm proposed in this paper is an effective automatic glioma segmentation algorithm.

Key words: image segmentation, glioma, Unet, Transformer self-attention, deformable convolution

中图分类号:

赵欣, 张鑫, 李鑫杰, 王洪凯. 融合多重自注意力和可变形卷积的多模态脑胶质瘤分割[J]. 波谱学杂志, 2023, 40(3): 280-292.

ZHAO Xin, ZHANG Xin, LI Xinjie, WANG Hongkai. Multimodal Glioma Segmentation with Fusion of Multiple Self-attention and Deformable Convolutions[J]. Chinese Journal of Magnetic Resonance, 2023, 40(3): 280-292.

图/表 12

图1

图2

图3

图4

图5

图6

表1

图7

图8

表2

表3

表4

参考文献 25

[1]	RONNEBERGER T, FISCHER P, BROX T. U-net: Convolutional networks for biomedical image segmentation[C]// International Conference on Medical image computing and computer-assisted intervention. Springer, 2015: 234-241.
[2]	REHMAN M U, CHO S B, KIM J, et al. Brainseg-net: Brain tumor mr image segmentation via enhanced encoder-decoder network[J]. Diagnostics, 2021, 11(2): 169. doi: 10.3390/diagnostics11020169
[3]	ZHAO L, MA J, SHAO Y, et al. MM-UNet: A multimodality brain tumor segmentation network in MRI images[J]. Front Oncol, 12: 950706. doi: 10.3389/fonc.2022.950706. doi: 10.3389/fonc.2022.950706
[4]	SHENG N, LIU D, ZHANG J, et al. Second-order ResU-Net for automatic MRI brain tumor segmentation[J]. Math Biosci Eng, 2021, 18(5): 4943-4960. doi: 10.3934/mbe.2021251 pmid: 34517471
[5]	HAN Y, SONG J M, XUE A Y, et al. Triple attention segmentation network for brain tumor images[J]. Chin J Biomed Eng, 2022, 41(1): 57-63.
	韩阳, 宋金淼, 薛安懿, 等. 基于三重注意力的脑肿瘤图像分割网络[J]. 中国生物医学工程学报, 2022, 41(1): 57-63.
[6]	ZHOU Z, RAHMAN SIDDIQUEE M M, TAJBAKHSH N, et al. Unet++: A nested u-net architecture for medical image segmentation[C]// Deep Learning in Medical Image Analysis and Multimodal Learning for Clinical Decision Support:4th International Workshop, DLMIA 2018, and 8th International Workshop, ML-CDS 2018, Held in Conjunction with MICCAI 2018, Granada, Spain, Proceedings 4. Springer International Publishing, 2018: 3-11.
[7]	HUANG H, LIN L, TONG R, et al. Unet 3+: A full-scale connected unet for medical image segmentation[C]// ICASSP 2020-2020 IEEE International Conference on Acoustics, Speech and Signal Processing (ICASSP). IEEE, 2020: 1055-1059.
[8]	QIN C, WU Y, LIAO W, et al. Improved U-Net3+ with stage residual for brain tumor segmentation[J]. BMC Medical Imaging, 2022, 22(1): 1-15. doi: 10.1186/s12880-021-00730-0
[9]	CHILD R, GRAY S, RADFORD A, et al. Generating long sequences with sparse transformers[J]. arXiv preprint arXiv:1904.10509, 2019.
[10]	YANG F, YANG H, FU J, et al. Learning texture transformer network for image super-resolution[C]// Proceedings of the IEEE/CVF conference on computer vision and pattern recognition. 2020: 5791-5800.
[11]	DOSOVITSKIY A, BEYER L, KOLESNIKOV A, et al. An image is worth 16×16 words: Transformers for image recognition at scale[J]. arXiv preprint arXiv:2010.11929, 2020.
[12]	TOUVRON H, CORD M, DOUZE M, et al. Training data-efficient image transformers & distillation through attention[C]// International conference on machine learning. PMLR, 2021: 10347-10357.
[13]	LIU Z, LIN Y, CAO Y, et al. Swin transformer: Hierarchical vision transformer using shifted windows[C]// Proceedings of the IEEE/CVF international conference on computer vision. 2021: 10012-10022.
[14]	GRAHAM B, EL-NOUBY A, TOUVRON H, et al. Levit: a vision transformer in convnet's clothing for faster inference[C]// Proceedings of the IEEE/CVF international conference on computer vision. 2021: 12259-12269.
[15]	CHEN J, LU Y, YU Q, et al. Transunet: Transformers make strong encoders for medical image segmentation[J]. arXiv preprint arXiv:2102.04306, 2021.
[16]	WANG W, CHEN C, DING M, et al. Transbts: Multimodal brain tumor segmentation using transformer[C]// Medical Image Computing and Computer Assisted Intervention-MICCAI 2021: 24th International Conference, Strasbourg, France, Proceedings, Part I 24. Springer International Publishing, 2021: 109-119.
[17]	HUANG L, CHEN L, ZHANG B, et al. A transformer-based generative adversarial network for brain tumor segmentation[J]. arXiv preprint arXiv:2207.14134, 2022.
[18]	HO J, KALCHBRENNER N, WEISSENBORN D, et al. Axial attention in multidimensional transformers[J]. arXiv preprint arXiv:1912.12180, 2019.
[19]	GUO M H, LIU Z N, MU T J, et al. Beyond self-attention: External attention using two linear layers for visual tasks[J]. IEEE Transactions on Pattern Anal, 2023, 45(1): 5436-5447.
[20]	DAI J, QI H, XIONG Y, et al. Deformable convolutional networks[C]// Proceedings of the IEEE international conference on computer vision. 2017: 764-773.
[21]	HOU A, WU L, SUN H, et al. Brain Segmentation Based on UNet++ with Weighted Parameters and Convolutional Neural Network[C]// IEEE International Conference on Advances in Electrical Engineering and Computer Applications (AEECA). IEEE, 2021: 644-648.
[22]	CHEN J, LU Y, YU Q, et al. Transunet: Transformers make strong encoders for medical image segmentation[J]. arXiv preprint arXiv:2102.04306, 2021.
[23]	SHAKER A, YAN W Y, LAROCQUE P E. Automatic land-water classification using multispectral airborne LiDAR data for near-shore and river environments[J]. ISPRS J Photogramm, 2019, 152: 94-108. doi: 10.1016/j.isprsjprs.2019.04.005
[24]	ZHOU Z, SIDDIQUEE M M R, TAJBAKHSH N, et al. Unet++: Redesigning skip connections to exploit multiscale features in image segmentation[J]. IEEE T Med Maging, 2019, 39(6): 1856-1867.
[25]	VASWANI A, SHAZEER N, PARMAR N, et al. Attention is all you need[C]// 31st Conference on Neural Information Processing Systems, Long Beach, CA, USA, 2017: 5998-6008.

网络	Dice/%				Hausdorff_95				PPV/%				Sensitivity/%
网络	WT	TC	ET	MV	WT	TC	ET	MV	WT	TC	ET	MV	WT	TC	ET	MV
Unet	83.24	84.57	76.84	81.55	2.6167	1.6551	2.7735	2.3488	85.57	85.34	78.52	83.14	85.94	90.58	80.55	85.69
DeepResUnet	85.00	84.80	78.74	82.85	2.5781	1.6267	2.7588	2.3212	87.67	88.50	80.79	85.65	86.55	91.24	81.92	86.57
Unet++	85.76	86.44	77.88	83.36	2.5874	1.6902	2.7558	2.3445	86.70	86.30	79.75	84.25	87.13	92.51	82.16	87.27
TransUnet	86.73	86.83	79.09	84.22	2.6562	1.5829	2.7395	2.3262	86.57	88.39	80.06	85.00	87.87	92.34	83.34	87.85
本文方法	88.15	87.98	80.46	85.33	2.5637	1.5323	2.6623	2.2528	87.75	88.98	79.89	85.54	88.22	92.16	83.66	88.01

文献来源	Dice/%	Hausdorff_95/mm	PPV/%	Sensitivity/%
文献[4]	79.56	8.93	-	-
文献[5]	84.40	2.28	85	88.76
文献[8]	80.09	-	-	83.19
文献[17]	73.7	5.48	73.7	78.7
本文结果	85.53	2.2528	85.54	88.01

网络	Dice/%			Hausdorff_95/mm			PPV/%			Sensitivity/%
网络	WT	TC	ET	WT	TC	ET	WT	TC	ET	WT	TC	ET
Unet	83.24	84.57	76.84	2.6167	1.6551	2.7735	85.57	85.34	78.52	85.94	90.58	80.55
Unet+Res	84.81	85.35	77.32	2.5977	1.6414	2.7408	86.06	85.95	78.64	86.47	90.89	81.23
Unet+Res+DCM	86.67	85.57	78.23	2.5677	1.6143	2.4234	87.07	86.50	78.79	86.55	91.24	81.92
Unet+Res+MATM	86.88	87.24	79.45	2.5681	1.5667	2.7588	87.70	86.30	79.75	87.13	92.01	82.16
Unet+Res+DCM+MATM （本文方法）	88.15	87.98	80.46	2.5637	1.5323	2.6623	87.75	88.98	79.89	88.22	92.16	83.66

模态	Dice/%			Hausdorff_95/mm			PPV/%			Sensitivity/%
模态	WT	TC	ET	WT	TC	ET	WT	TC	ET	WT	TC	ET
Flair	86.63	84.71	78.46	2.5831	1.5842	2.6957	86.07	86.84	79.76	86.54	89.75	81.75
T1	84.83	86.94	77.23	2.6011	1.5548	2.7634	84.51	88.07	77.61	85.24	90.84	78.21
T1ce	84.61	87.26	79.26	2.6109	1.5424	2.6847	84.47	88.51	80.47	85.19	91.57	82.47
T2	87.05	84.09	77.48	2.5785	1.6042	2.7498	86.64	85.89	77.89	86.67	88.43	78.64
Multi modal （本文方法）	88.15	87.98	80.46	2.5637	1.5323	2.6623	87.75	88.98	81.09	88.22	92.16	83.66