Fusing Attention Mechanism with Mask RCNN for Recognition of Acoustic Neuroma and Meningioma in Cerebellopontine Angle

doi:10.11938/cjmr20223045

Abstract

Abstract:

To investigate the performance of Mask RCNN (region-based convolutional neural network) using T1WI-enhanced images and fusing with attention mechanism in recognizing acoustic neuroma and meningioma in cerebellopontine angle area, this paper retrospectively collected 116 cases of meningioma and 427 cases of acoustic neuroma confirmed by pathology or clinical diagnosis. 872 images of meningioma and 2 467 images of acoustic neuroma were adopted after image screening. These images were divided into the training set, validation set, and test set in a ratio of approximately 7:1.5:1.5. After preprocessing the images, we employed five models, namely Mask RCNN model with Resnet50, Resnet101, VGG19 as the backbone network, and Mask RCNN models Resnet101-CBAM and VGG19-CBAM fusing with the convolution attention mechanism, for the detection and lesion segmentation of acoustic neuromas and meningioma in cerebellopontine angle area. The model performance was evaluated with mean average precision (mAP) and mean average recall (mAR). The results in the test set showed that the convolution attention mechanism could improve model performance. VGG19-CBAM model outperformed other models with mAP of 0.932 in classification and 0.930 in segmentation. This indicates that Mask RCNN fusing with attention mechanism has a better performance in recognizing acoustic neuroma and meningioma in cerebellopontine angle area, and hence can improve clinical efficiency by providing a reference for the diagnosis and target area segmentation.

Key words: Mask RCNN, CBAM, acoustic neuroma, meningioma, magnetic resonance image

CLC Number:

HU Xiaoyang, LIU Ying, CHEN Shu, DONG Binbin. Fusing Attention Mechanism with Mask RCNN for Recognition of Acoustic Neuroma and Meningioma in Cerebellopontine Angle[J]. Chinese Journal of Magnetic Resonance, 2023, 40(3): 293-306.

Figures/Tables 9

Fig. 1

Fig. 2

Fig. 3

Fig. 4

Fig. 5

Fig. 6

Fig. 7

Table 1

Fig. 8

References 38

[1]	YU H Q, YAO C Y, DENG Y S. Diagnosis and treatment of meningioma in cerebellopontine angle[J]. Journal of Clinical Medical, 2015, 2(27): 5650-5651.
	于洪泉, 姚长义, 邓玉生. 桥小脑角区脑膜瘤的诊断及治疗[J]. 临床医药文献电子杂志, 2015, 2(27): 5650-5651.
[2]	RADES D, FEHLAUER F, LAMSZUS K, et al. Well-differentiated neurocytoma: what is the best available treatment?[J]. Neuro-oncology, 2005, 7(1): 77-83. doi: 10.1215/S1152851704000584 pmid: 15701284
[3]	GEERT L, THIJS K, BABAK E B, et al. A survey on deep learning in medical image analysis[J]. Med Image Anal, 2017, 42: 60-88. doi: S1361-8415(17)30113-5 pmid: 28778026
[4]	ZHANG W L, LI R J, DENG H T, et al. Deep convolutional neural networks for multi-modality isointense infant brain image segmentation[J]. NeuroImage, 2015, 108: 214-224. doi: 10.1016/j.neuroimage.2014.12.061 pmid: 25562829
[5]	MOESKOPS P, VIERGEVER M A, MENDRIK A M, et al. Automatic segmentation of MR brain images with a convolutional neural network[J]. IEEE T Med Imaging, 2016, 35(5): 1252-1261. doi: 10.1109/TMI.2016.2548501 pmid: 27046893
[6]	HERENT P, SCHMAUCH B, JEHANNO P, et al. Detection and characterization of MRI breast lesions using deep learning[J]. Diagn Interv Imag, 2019, 100(4): 219-225. doi: S2211-5684(19)30056-7 pmid: 30926444
[7]	WEI Z H, YAN S J, HAN B S, et al. Diagnosis of alzheimer’s disease based on multi-output three-dimensional convolutional neural network[J]. Chinese J Magn Reson, 2021, 38(1): 92-100.
	魏志宏, 闫士举, 韩宝三, 等. 基于多输出的3D卷积神经网络诊断阿尔兹海默病[J]. 波谱学杂志, 2021, 38(1): 92-100.
[8]	TIAN J X, LIU G C, GU S S, et al. Deep learning in medical image analysis and its challenges[J]. Acta Automatica Sinica, 2018, 44(3): 401-424.
	田娟秀, 刘国才, 谷珊珊, 等. 医学图像分析深度学习方法研究与挑战[J]. 自动化学报, 2018, 44(3): 401-424.
[9]	JIANG Z K, LV X G, ZHANG J X, et al. Review of deep learning methods for MRI brain tumor image segmentation[J]. Journal of Image and Graphics, 2020, 25(2): 215-228.
	江宗康, 吕晓钢, 张建新, 等. MRI脑肿瘤图像分割的深度学习方法综述[J]. 中国图象图形学报, 2020, 25(2): 215-228.
[10]	HAVAEI M, DAVY A, WAROK-FARLEY D, et al. Brain tumor segmentation with deep neural networks[J]. Med Image Anal, 2017, 35: 18-31. doi: S1361-8415(16)30033-0 pmid: 27310171
[11]	WU Y C, LIN L, WU S C. Multimodal high-grade glioma semantic segmentation network with multi-scale and multi-attention fusion mechanism[J]. Journal of Biomedical Engineering, 2022, 39(3): 433-440.
	吴玉超, 林岚, 吴水才. 基于多尺度、多路注意力融合机制的多模态高等级脑胶质瘤语义分割网络[J]. 生物医学工程学杂志, 2022, 39(3): 433-440.
[12]	LOU Y Z, LIU Y, JIANG H, et al. A deep learning algorithm for classifying meningioma and auditory neuroma in the cerebellopontine angle from magnetic resonance images[J]. Chinese J Magn Reson, 2020, 37(3): 300-310.
	娄云重, 刘颖, 江华, 等. 基于MRI和深度学习的桥小脑角区脑膜瘤与听神经瘤分类算法研究[J]. 波谱学杂志, 2020, 37(3): 300-310.
[13]	LIU Y, CHEN J C. HU X Y, et al. Classification and localization of meningioma and acoustic neuroma in cerebellopontine angle based on Mask RCNN[J]. Chinese J Magn Reson, 2021, 38(1): 58-68.
	刘颖, 陈静聪, 胡小洋, 等. 基于Mask RCNN的桥小脑角区脑膜瘤与听神经瘤分类定位研究[J]. 波谱学杂志, 2021, 38(1): 58-68.
[14]	HE K, GKIOXARI G, DOLLAR P, et al. Mask R-CNN[C]// IEEE International Conference on Computer Vision. Washington D C: IEEE, 2017: 2961-2969.
[15]	YUAN L, QIU Z. Mask-RCNN with spatial attention for pedestrian segmentation in cyber-physical systems[J]. Comput Commun, 2021, 180: 109-114. doi: 10.1016/j.comcom.2021.09.002
[16]	AHMED B, GULLIVER T A, ALZAHIR S. Image splicing detection using mask-RCNN[J]. Signal Image Video P, 2020, 14(5): 1035-1042.
[17]	ZHANG H Y, XU X Y, MA X F, et al. Mask-RCNN recognition method of composite fold shape in ultrasound images[J]. Acta Physica Sinica, 2022, 71(1): 203-210.
	张海燕, 徐心语, 马雪芬, 等. 超声图像中复合材料褶皱形态的Mask-RCNN识别方法[J]. 物理学报, 2022, 71(7): 203-210.
[18]	HAO S Y, LEE D H, ZHAO D. Sequence to sequence learning with attention mechanism for short-term passenger flow prediction in large-scale metro system[J]. Transport Res C-Emer, 2019, 107(C): 287-300.
[19]	CHEN H H, WU G D, LI J X, et al. Research advances on deep learning recommendation based on attention mechanism[J]. Comput Eng Sci, 2021, 43(2): 370-380.
	陈海涵, 吴国栋, 李景霞, 等. 基于注意力机制的深度学习推荐研究进展[J]. 计算机工程与科学, 2021, 43(2): 370-380.
[20]	DAI Y, LIU W B, DONG X Y, et al. U-NET CSF cells segmentation based on attention mechanism[J]. Journal of Northeastern University(Natural Science), 2022, 43(7): 944-950. doi: 10.12068/j.issn.1005-3026.2022.07.005
	代茵, 刘维宾, 董昕阳, 等. 基于注意力机制的U-Net脑脊液细胞分割[J]. 东北大学学报(自然科学版), 2022, 43(7): 944-950. doi: 10.12068/j.issn.1005-3026.2022.07.005
[21]	KAREL Z. Contrast limited adaptive histogram equalization[C]// Graphics Gems IV. San Diego: Academic Press Professional, 1994: 474-485.
[22]	LIU X, LIU J B. Mammary image enhancement based on contrast limited adaptive histogram equalization[J]. Computer Engineering and Applications, 2008, 44(10): 173-175.
	刘轩, 刘佳宾. 基于对比度受限自适应直方图均衡的乳腺图像增强[J]. 计算机工程与应用, 2008, 44(10): 173-175.
[23]	LIN T Y, DOLLAR P, GIRSHICK R, et al. Feature pyramid networks for object detection[J]. IEEE Computer Society, 2017. DOI: 10.1109/CVPR.2017.106. doi: 10.1109/CVPR.2017.106
[24]	SIMONYAN K, ZISSERMAN A. Very deep convolutional networks for large-scale image recognition[C]// International Conference on Learning Representations (ICLR), 2015: 1-14.
[25]	HE K, ZHANG X, REN S, et al. Deep residual learning for image recognition[C]// 2016 IEEE Conference on Computer Vision and Pattern Recognition (CVPR), IEEE, 2016: 770-778.
[26]	LONG J, SHELHAMER E, DARRELL T. Fully convolutional networks for semantic segmentation[J]. IEEE Transactions on Pattern Analysis and Machine Intelligence, 2015, 39(4): 640-651. DOI:10.1109/CVPR.2015.7298965. doi: 10.1109/CVPR.2015.7298965
[27]	SHEN X X, HOU X W, YIN C H. State attention in deep reinforcement learning[J]. CAAI transactions on intelligent systems, 2020, 15(2): 317-322.
	申翔翔, 侯新文, 尹传环. 深度强化学习中状态注意力机制的研究[J]. 智能系统学报, 2020, 15(2): 317-322.
[28]	WOO S, PARK J, LEE J Y, et al. CBAM: convolutional block attention module[M]// Computer Vision (ECCV 2018). Cham: Springer International Publishing, 2018: 3-19.
[29]	黄泽桑. 基于深度学习的目标检测技术研究[D]. 北京: 北京邮电大学, 2019.
[30]	PADILLA R, PASSOS W L, DIAS T L B, et al. A comparative analysis of object detection metrics with a companion open-source toolkit[J]. Electronics. 2021, 10(3): 279. doi: 10.3390/electronics10030279
[31]	MNIH V, HEESS N, GRAVES A. Recurrent models of visual attention[C]// Proceedings of the 27th International Conference on Neural Information Processing Systems, Cambridge: MIT Press, 2014, 2: 2204-2212.
[32]	LI H Y, LI C G, AN J B, et al. Attention mechanism improves CNN remote sensing image object detection[J]. Journal of Image and Graphics, 2019, 24(8): 1400-1408.
	李红艳, 李春庚, 安居白, 等. 注意力机制改进卷积神经网络的遥感图像目标检测[J]. 中国图象图形学报, 2019, 24(8): 1400-1408.
[33]	XU W H, PEI Y J, GAO D L, et al. Mass classification of breast mammogram based on attention mechanism and transfer learning[J]. Laser Optoelectronics Progress, 2021, 58(4): 146-154.
	许文慧, 裴以建, 郜冬林, 等. 基于注意力机制与迁移学习的乳腺钼靶肿块分类[J]. 激光与光电子学进展, 2021, 58(4): 146-154.
[34]	ZHENG F, CHEN X Z. Status of artificial intelligence in meningioma image[J]. Chin J Magn Reson Imaging, 2020, 11(10): 934-936.
	郑飞, 陈绪珠. 脑膜瘤影像人工智能应用进展[J]. 磁共振成像, 2020, 11(10): 934-936.
[35]	DONG Y D, WEI W, MA X L. Progress in application of stereotactic radiosurgery in treatment of acoustic neuroma[J]. Journal of Otology, 2020, 18(1): 25-32.
	董耀东, 魏薇, 马秀岚. 立体放射治疗技术在听神经瘤治疗中的应用进展[J]. 中华耳科学杂志, 2020, 18(1): 25-32.
[36]	ARTHURS B J, LAMOREAUX W T, MACKAY A R, et al. Gamma knife radiosurgery for vestibular schwannomas: tumor control and functional preservation in 70 patients.[J]. Am J Clin Oncol, 2011, 34(3): 265-269. doi: 10.1097/COC.0b013e3181dbc2ab
[37]	FOOTE K D, FRIEDMAN W A, BUATTI J M, et al. Analysis of risk factors associated with radiosurgery for vestibular schwannoma[J]. J Neurosurg, 2001, 95(3): 440-449. doi: 10.3171/jns.2001.95.3.0440 pmid: 11565866
[38]	RYKACZEWSKI B, ZABEK M. A meta-analysis of treatment of vestibular schwannoma using gamma knife radiosurgery[J]. Contemporary Oncology, 2014, 18(1): 60-66.

模型	检测指标		分割指标
模型	mAR	mAP	mAR	mAP
Resnet50	0.693	0.852	0.772	0.857
Resnet101	0.741	0.890	0.774	0.898
VGG19	0.725	0.914	0.808	0.922
Resnet101-CBAM	0.743	0.918	0.795	0.923
VGG19-CBAM	0.749	0.932	0.799	0.930