Chinese Journal of Magnetic Resonance ›› 2023, Vol. 40 ›› Issue (3): 280-292.doi: 10.11938/cjmr20233059
• Articles • Previous Articles Next Articles
ZHAO Xin1,*(),ZHANG Xin1,LI Xinjie1,WANG Hongkai2
Received:
2023-03-13
Published:
2023-09-05
Online:
2023-04-19
Contact:
*Tel: 18525639818, E-mail: CLC Number:
ZHAO Xin, ZHANG Xin, LI Xinjie, WANG Hongkai. Multimodal Glioma Segmentation with Fusion of Multiple Self-attention and Deformable Convolutions[J]. Chinese Journal of Magnetic Resonance, 2023, 40(3): 280-292.
Table 1
Indexes of segmentation results using different models on BraTs2019 dataset
网络 | Dice/% | Hausdorff_95 | PPV/% | Sensitivity/% | ||||||||||||
---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|
WT | TC | ET | MV | WT | TC | ET | MV | WT | TC | ET | MV | WT | TC | ET | MV | |
Unet | 83.24 | 84.57 | 76.84 | 81.55 | 2.6167 | 1.6551 | 2.7735 | 2.3488 | 85.57 | 85.34 | 78.52 | 83.14 | 85.94 | 90.58 | 80.55 | 85.69 |
DeepResUnet | 85.00 | 84.80 | 78.74 | 82.85 | 2.5781 | 1.6267 | 2.7588 | 2.3212 | 87.67 | 88.50 | 80.79 | 85.65 | 86.55 | 91.24 | 81.92 | 86.57 |
Unet++ | 85.76 | 86.44 | 77.88 | 83.36 | 2.5874 | 1.6902 | 2.7558 | 2.3445 | 86.70 | 86.30 | 79.75 | 84.25 | 87.13 | 92.51 | 82.16 | 87.27 |
TransUnet | 86.73 | 86.83 | 79.09 | 84.22 | 2.6562 | 1.5829 | 2.7395 | 2.3262 | 86.57 | 88.39 | 80.06 | 85.00 | 87.87 | 92.34 | 83.34 | 87.85 |
本文方法 | 88.15 | 87.98 | 80.46 | 85.33 | 2.5637 | 1.5323 | 2.6623 | 2.2528 | 87.75 | 88.98 | 79.89 | 85.54 | 88.22 | 92.16 | 83.66 | 88.01 |
Table 3
Indexes of segmentation results using Unet models adding with different modules on BraTs2019 dataset
网络 | Dice/% | Hausdorff_95/mm | PPV/% | Sensitivity/% | |||||||||||
---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|
WT | TC | ET | WT | TC | ET | WT | TC | ET | WT | TC | ET | ||||
Unet | 83.24 | 84.57 | 76.84 | 2.6167 | 1.6551 | 2.7735 | 85.57 | 85.34 | 78.52 | 85.94 | 90.58 | 80.55 | |||
Unet+Res | 84.81 | 85.35 | 77.32 | 2.5977 | 1.6414 | 2.7408 | 86.06 | 85.95 | 78.64 | 86.47 | 90.89 | 81.23 | |||
Unet+Res+DCM | 86.67 | 85.57 | 78.23 | 2.5677 | 1.6143 | 2.4234 | 87.07 | 86.50 | 78.79 | 86.55 | 91.24 | 81.92 | |||
Unet+Res+MATM | 86.88 | 87.24 | 79.45 | 2.5681 | 1.5667 | 2.7588 | 87.70 | 86.30 | 79.75 | 87.13 | 92.01 | 82.16 | |||
Unet+Res+DCM+MATM (本文方法) | 88.15 | 87.98 | 80.46 | 2.5637 | 1.5323 | 2.6623 | 87.75 | 88.98 | 79.89 | 88.22 | 92.16 | 83.66 |
Table 4
Ablation experiment of different modalities
模态 | Dice/% | Hausdorff_95/mm | PPV/% | Sensitivity/% | |||||||||||
---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|
WT | TC | ET | WT | TC | ET | WT | TC | ET | WT | TC | ET | ||||
Flair | 86.63 | 84.71 | 78.46 | 2.5831 | 1.5842 | 2.6957 | 86.07 | 86.84 | 79.76 | 86.54 | 89.75 | 81.75 | |||
T1 | 84.83 | 86.94 | 77.23 | 2.6011 | 1.5548 | 2.7634 | 84.51 | 88.07 | 77.61 | 85.24 | 90.84 | 78.21 | |||
T1ce | 84.61 | 87.26 | 79.26 | 2.6109 | 1.5424 | 2.6847 | 84.47 | 88.51 | 80.47 | 85.19 | 91.57 | 82.47 | |||
T2 | 87.05 | 84.09 | 77.48 | 2.5785 | 1.6042 | 2.7498 | 86.64 | 85.89 | 77.89 | 86.67 | 88.43 | 78.64 | |||
Multi modal (本文方法) | 88.15 | 87.98 | 80.46 | 2.5637 | 1.5323 | 2.6623 | 87.75 | 88.98 | 81.09 | 88.22 | 92.16 | 83.66 |
[1] | RONNEBERGER T, FISCHER P, BROX T. U-net: Convolutional networks for biomedical image segmentation[C]// International Conference on Medical image computing and computer-assisted intervention. Springer, 2015: 234-241. |
[2] |
REHMAN M U, CHO S B, KIM J, et al. Brainseg-net: Brain tumor mr image segmentation via enhanced encoder-decoder network[J]. Diagnostics, 2021, 11(2): 169.
doi: 10.3390/diagnostics11020169 |
[3] |
ZHAO L, MA J, SHAO Y, et al. MM-UNet: A multimodality brain tumor segmentation network in MRI images[J]. Front Oncol, 12: 950706. doi: 10.3389/fonc.2022.950706.
doi: 10.3389/fonc.2022.950706 |
[4] |
SHENG N, LIU D, ZHANG J, et al. Second-order ResU-Net for automatic MRI brain tumor segmentation[J]. Math Biosci Eng, 2021, 18(5): 4943-4960.
doi: 10.3934/mbe.2021251 pmid: 34517471 |
[5] | HAN Y, SONG J M, XUE A Y, et al. Triple attention segmentation network for brain tumor images[J]. Chin J Biomed Eng, 2022, 41(1): 57-63. |
韩阳, 宋金淼, 薛安懿, 等. 基于三重注意力的脑肿瘤图像分割网络[J]. 中国生物医学工程学报, 2022, 41(1): 57-63. | |
[6] | ZHOU Z, RAHMAN SIDDIQUEE M M, TAJBAKHSH N, et al. Unet++: A nested u-net architecture for medical image segmentation[C]// Deep Learning in Medical Image Analysis and Multimodal Learning for Clinical Decision Support:4th International Workshop, DLMIA 2018, and 8th International Workshop, ML-CDS 2018, Held in Conjunction with MICCAI 2018, Granada, Spain, Proceedings 4. Springer International Publishing, 2018: 3-11. |
[7] | HUANG H, LIN L, TONG R, et al. Unet 3+: A full-scale connected unet for medical image segmentation[C]// ICASSP 2020-2020 IEEE International Conference on Acoustics, Speech and Signal Processing (ICASSP). IEEE, 2020: 1055-1059. |
[8] |
QIN C, WU Y, LIAO W, et al. Improved U-Net3+ with stage residual for brain tumor segmentation[J]. BMC Medical Imaging, 2022, 22(1): 1-15.
doi: 10.1186/s12880-021-00730-0 |
[9] | CHILD R, GRAY S, RADFORD A, et al. Generating long sequences with sparse transformers[J]. arXiv preprint arXiv:1904.10509, 2019. |
[10] | YANG F, YANG H, FU J, et al. Learning texture transformer network for image super-resolution[C]// Proceedings of the IEEE/CVF conference on computer vision and pattern recognition. 2020: 5791-5800. |
[11] | DOSOVITSKIY A, BEYER L, KOLESNIKOV A, et al. An image is worth 16×16 words: Transformers for image recognition at scale[J]. arXiv preprint arXiv:2010.11929, 2020. |
[12] | TOUVRON H, CORD M, DOUZE M, et al. Training data-efficient image transformers & distillation through attention[C]// International conference on machine learning. PMLR, 2021: 10347-10357. |
[13] | LIU Z, LIN Y, CAO Y, et al. Swin transformer: Hierarchical vision transformer using shifted windows[C]// Proceedings of the IEEE/CVF international conference on computer vision. 2021: 10012-10022. |
[14] | GRAHAM B, EL-NOUBY A, TOUVRON H, et al. Levit: a vision transformer in convnet's clothing for faster inference[C]// Proceedings of the IEEE/CVF international conference on computer vision. 2021: 12259-12269. |
[15] | CHEN J, LU Y, YU Q, et al. Transunet: Transformers make strong encoders for medical image segmentation[J]. arXiv preprint arXiv:2102.04306, 2021. |
[16] | WANG W, CHEN C, DING M, et al. Transbts: Multimodal brain tumor segmentation using transformer[C]// Medical Image Computing and Computer Assisted Intervention-MICCAI 2021: 24th International Conference, Strasbourg, France, Proceedings, Part I 24. Springer International Publishing, 2021: 109-119. |
[17] | HUANG L, CHEN L, ZHANG B, et al. A transformer-based generative adversarial network for brain tumor segmentation[J]. arXiv preprint arXiv:2207.14134, 2022. |
[18] | HO J, KALCHBRENNER N, WEISSENBORN D, et al. Axial attention in multidimensional transformers[J]. arXiv preprint arXiv:1912.12180, 2019. |
[19] | GUO M H, LIU Z N, MU T J, et al. Beyond self-attention: External attention using two linear layers for visual tasks[J]. IEEE Transactions on Pattern Anal, 2023, 45(1): 5436-5447. |
[20] | DAI J, QI H, XIONG Y, et al. Deformable convolutional networks[C]// Proceedings of the IEEE international conference on computer vision. 2017: 764-773. |
[21] | HOU A, WU L, SUN H, et al. Brain Segmentation Based on UNet++ with Weighted Parameters and Convolutional Neural Network[C]// IEEE International Conference on Advances in Electrical Engineering and Computer Applications (AEECA). IEEE, 2021: 644-648. |
[22] | CHEN J, LU Y, YU Q, et al. Transunet: Transformers make strong encoders for medical image segmentation[J]. arXiv preprint arXiv:2102.04306, 2021. |
[23] |
SHAKER A, YAN W Y, LAROCQUE P E. Automatic land-water classification using multispectral airborne LiDAR data for near-shore and river environments[J]. ISPRS J Photogramm, 2019, 152: 94-108.
doi: 10.1016/j.isprsjprs.2019.04.005 |
[24] | ZHOU Z, SIDDIQUEE M M R, TAJBAKHSH N, et al. Unet++: Redesigning skip connections to exploit multiscale features in image segmentation[J]. IEEE T Med Maging, 2019, 39(6): 1856-1867. |
[25] | VASWANI A, SHAZEER N, PARMAR N, et al. Attention is all you need[C]// 31st Conference on Neural Information Processing Systems, Long Beach, CA, USA, 2017: 5998-6008. |
[1] | Li Yijie, YANG Xinyu, YANG Xiaomei. Magnetic Resonance Image Reconstruction of Multi-scale Residual Unet Fused with Attention Mechanism [J]. Chinese Journal of Magnetic Resonance, 2023, 40(3): 307-319. |
[2] | FANG Yi, WAN Qian, YUAN Jiawen, LIN Shaoqiang, LI Ye, LIU Xin, ZHENG Hairong, ZOU Chao. Comparison Study of the Metabolic Characteristics of Three Kinds of Deuterium-labeled Glucose in Rat Glioma Cells [J]. Chinese Journal of Magnetic Resonance, 2023, 40(3): 239-245. |
[3] | Ying-shan WANG, Ao-qi DENG, Jin-ling MAO, Zhong-qi ZHU, Jie SHI, Guang YANG, Wei-wei MA, Qing LU, Hong-zhi WANG. Automatic Segmentation of Knee Joint Synovial Magnetic Resonance Images Based on 3D VNetTrans [J]. Chinese Journal of Magnetic Resonance, 2022, 39(3): 303-315. |
[4] | Zhen-yu WANG, Ying-shan WANG, Jin-ling MAO, Wei-wei MA, Qing LU, Jie SHI, Hong-zhi WANG. Magnetic Resonance Images Segmentation of Synovium Based on Dense-UNet++ [J]. Chinese Journal of Magnetic Resonance, 2022, 39(2): 208-219. |
[5] | Yue QIU, Sheng-dong NIE, Long WEI. Segmentation of Breast Tumors Based on Fully Convolutional Network and Dynamic Contrast Enhanced Magnetic Resonance Image [J]. Chinese Journal of Magnetic Resonance, 2022, 39(2): 196-207. |
[6] | Han-wei WANG,Hao WU,Jing TIAN,Jun-feng ZHANG,Peng ZHONG,Li-zhao CHEN,Shu-nan WANG. The Diagnostic Value of Quantitative Parameters of T2/FLAIR Mismatch Sign in Evaluating the Molecular Typing of Lower-grade Gliomas [J]. Chinese Journal of Magnetic Resonance, 2022, 39(1): 56-63. |
[7] | Ying LIU,Yi-yun GUO,Jing-cong CHEN,Hao-wei ZHANG. Automatic Precise Segmentation of Cerebellopontine Angle Tumor Based on Faster-RCNN and Level-Set Method [J]. Chinese Journal of Magnetic Resonance, 2021, 38(3): 381-391. |
[8] | Qin-yi SHI,Fang YAN,Yang YANG,Yue-fu CHEN,Xiao-lang LIN,Yuan-jun WANG. Image Segmentation of Tooth and Alveolar Bone with the Level Set Model [J]. Chinese Journal of Magnetic Resonance, 2021, 38(2): 182-193. |
[9] | GONG Jin-chang, WANG Yu, WANG Yuan-jun. A Method for Segmentation of Glioma on Multimodal Magnetic Resonance Images Based on Wavelet Fusion and Deep Learning [J]. Chinese Journal of Magnetic Resonance, 2020, 37(2): 131-143. |
[10] | XIAO Liang, LOU Yu-kun, ZHOU Hang-yu. A U-Net Network-Based Rapid Construction of Knee Models for Specific Absorption Rate Estimation [J]. Chinese Journal of Magnetic Resonance, 2020, 37(2): 144-151. |
[11] | ZHANG Xue-ying, WANG Cheng-long, XIE Hai-bin, ZHANG Cheng-xiu, MA Chao, LU Jian-ping, YANG Guang. Parallel Segmentation and Tracking Algorithm for Magnetic Resonance Angiography Images Based on GPU [J]. Chinese Journal of Magnetic Resonance, 2016, 33(4): 570-580. |
[12] |
SHAO Wei1,GU Jin-ping1,HUANG Cai-hua2,HUANG Zi-cheng1,JI Tian-hai1*,LIN Dong-hai1*.
Metabolic Profiles of CHG5 and U87 Glioma Cell Lines Derived by 1H NMR
[J]. Chinese Journal of Magnetic Resonance, 2014, 31(1): 40-48.
|
Viewed | ||||||
Full text |
|
|||||
Abstract |
|
|||||