融合注意力机制Mask RCNN的桥小脑角区听神经瘤和脑膜瘤的识别研究

doi:10.11938/cjmr20223045

摘要/Abstract

摘要：

为探讨采用T1WI增强图像，利用融合注意力机制的掩膜区域神经网络（Mask RCNN）模型实现对桥小脑角区听神经瘤和脑膜瘤的识别．本文回顾性收集经病理或临床诊断确诊的脑膜瘤116例和听神经瘤427例，经图像筛选后共采用脑膜瘤872张和听神经瘤2 467张．按近似7:1.5:1.5的比例分为训练集、验证集和测试集．对图像进行预处理后，采用以Resnet50、Resnet101和VGG19为主干网络的Mask RCNN模型，以及融合卷积注意力机制的Mask RCNN模型Resnet101-CBAM和VGG19-CBAM对桥小脑角区听神经瘤和脑膜瘤进行检测和病灶分割．并使用均值平均精度（mean average precision，mAP）和均值平均召回率（mean average recall，mAR）评价模型性能．测试集结果显示卷积注意力机制可以提升模型性能，VGG19-CBAM模型在5个模型中综合性能最高，在分类和病灶分割的mAP分别为0.932和0.930．这表明融合注意力机制的Mask RCNN模型对桥小脑角区听神经瘤和脑膜瘤的识别较为理想，可为诊断和靶区勾画提供参考，提高临床工作效率．

关键词: 掩膜区域神经网络, 卷积注意力模块, 听神经瘤, 脑膜瘤, 磁共振图像

Abstract:

To investigate the performance of Mask RCNN (region-based convolutional neural network) using T1WI-enhanced images and fusing with attention mechanism in recognizing acoustic neuroma and meningioma in cerebellopontine angle area, this paper retrospectively collected 116 cases of meningioma and 427 cases of acoustic neuroma confirmed by pathology or clinical diagnosis. 872 images of meningioma and 2 467 images of acoustic neuroma were adopted after image screening. These images were divided into the training set, validation set, and test set in a ratio of approximately 7:1.5:1.5. After preprocessing the images, we employed five models, namely Mask RCNN model with Resnet50, Resnet101, VGG19 as the backbone network, and Mask RCNN models Resnet101-CBAM and VGG19-CBAM fusing with the convolution attention mechanism, for the detection and lesion segmentation of acoustic neuromas and meningioma in cerebellopontine angle area. The model performance was evaluated with mean average precision (mAP) and mean average recall (mAR). The results in the test set showed that the convolution attention mechanism could improve model performance. VGG19-CBAM model outperformed other models with mAP of 0.932 in classification and 0.930 in segmentation. This indicates that Mask RCNN fusing with attention mechanism has a better performance in recognizing acoustic neuroma and meningioma in cerebellopontine angle area, and hence can improve clinical efficiency by providing a reference for the diagnosis and target area segmentation.

Key words: Mask RCNN, CBAM, acoustic neuroma, meningioma, magnetic resonance image

中图分类号:

胡小洋, 刘颖, 陈淑, 董彬彬. 融合注意力机制Mask RCNN的桥小脑角区听神经瘤和脑膜瘤的识别研究[J]. 波谱学杂志, 2023, 40(3): 293-306.

HU Xiaoyang, LIU Ying, CHEN Shu, DONG Binbin. Fusing Attention Mechanism with Mask RCNN for Recognition of Acoustic Neuroma and Meningioma in Cerebellopontine Angle[J]. Chinese Journal of Magnetic Resonance, 2023, 40(3): 293-306.

图/表 9

图1

图2

图3

图4

图5

图6

图7

表1

图8

参考文献 38

[1]	YU H Q, YAO C Y, DENG Y S. Diagnosis and treatment of meningioma in cerebellopontine angle[J]. Journal of Clinical Medical, 2015, 2(27): 5650-5651.
	于洪泉, 姚长义, 邓玉生. 桥小脑角区脑膜瘤的诊断及治疗[J]. 临床医药文献电子杂志, 2015, 2(27): 5650-5651.
[2]	RADES D, FEHLAUER F, LAMSZUS K, et al. Well-differentiated neurocytoma: what is the best available treatment?[J]. Neuro-oncology, 2005, 7(1): 77-83. doi: 10.1215/S1152851704000584 pmid: 15701284
[3]	GEERT L, THIJS K, BABAK E B, et al. A survey on deep learning in medical image analysis[J]. Med Image Anal, 2017, 42: 60-88. doi: S1361-8415(17)30113-5 pmid: 28778026
[4]	ZHANG W L, LI R J, DENG H T, et al. Deep convolutional neural networks for multi-modality isointense infant brain image segmentation[J]. NeuroImage, 2015, 108: 214-224. doi: 10.1016/j.neuroimage.2014.12.061 pmid: 25562829
[5]	MOESKOPS P, VIERGEVER M A, MENDRIK A M, et al. Automatic segmentation of MR brain images with a convolutional neural network[J]. IEEE T Med Imaging, 2016, 35(5): 1252-1261. doi: 10.1109/TMI.2016.2548501 pmid: 27046893
[6]	HERENT P, SCHMAUCH B, JEHANNO P, et al. Detection and characterization of MRI breast lesions using deep learning[J]. Diagn Interv Imag, 2019, 100(4): 219-225. doi: S2211-5684(19)30056-7 pmid: 30926444
[7]	WEI Z H, YAN S J, HAN B S, et al. Diagnosis of alzheimer’s disease based on multi-output three-dimensional convolutional neural network[J]. Chinese J Magn Reson, 2021, 38(1): 92-100.
	魏志宏, 闫士举, 韩宝三, 等. 基于多输出的3D卷积神经网络诊断阿尔兹海默病[J]. 波谱学杂志, 2021, 38(1): 92-100.
[8]	TIAN J X, LIU G C, GU S S, et al. Deep learning in medical image analysis and its challenges[J]. Acta Automatica Sinica, 2018, 44(3): 401-424.
	田娟秀, 刘国才, 谷珊珊, 等. 医学图像分析深度学习方法研究与挑战[J]. 自动化学报, 2018, 44(3): 401-424.
[9]	JIANG Z K, LV X G, ZHANG J X, et al. Review of deep learning methods for MRI brain tumor image segmentation[J]. Journal of Image and Graphics, 2020, 25(2): 215-228.
	江宗康, 吕晓钢, 张建新, 等. MRI脑肿瘤图像分割的深度学习方法综述[J]. 中国图象图形学报, 2020, 25(2): 215-228.
[10]	HAVAEI M, DAVY A, WAROK-FARLEY D, et al. Brain tumor segmentation with deep neural networks[J]. Med Image Anal, 2017, 35: 18-31. doi: S1361-8415(16)30033-0 pmid: 27310171
[11]	WU Y C, LIN L, WU S C. Multimodal high-grade glioma semantic segmentation network with multi-scale and multi-attention fusion mechanism[J]. Journal of Biomedical Engineering, 2022, 39(3): 433-440.
	吴玉超, 林岚, 吴水才. 基于多尺度、多路注意力融合机制的多模态高等级脑胶质瘤语义分割网络[J]. 生物医学工程学杂志, 2022, 39(3): 433-440.
[12]	LOU Y Z, LIU Y, JIANG H, et al. A deep learning algorithm for classifying meningioma and auditory neuroma in the cerebellopontine angle from magnetic resonance images[J]. Chinese J Magn Reson, 2020, 37(3): 300-310.
	娄云重, 刘颖, 江华, 等. 基于MRI和深度学习的桥小脑角区脑膜瘤与听神经瘤分类算法研究[J]. 波谱学杂志, 2020, 37(3): 300-310.
[13]	LIU Y, CHEN J C. HU X Y, et al. Classification and localization of meningioma and acoustic neuroma in cerebellopontine angle based on Mask RCNN[J]. Chinese J Magn Reson, 2021, 38(1): 58-68.
	刘颖, 陈静聪, 胡小洋, 等. 基于Mask RCNN的桥小脑角区脑膜瘤与听神经瘤分类定位研究[J]. 波谱学杂志, 2021, 38(1): 58-68.
[14]	HE K, GKIOXARI G, DOLLAR P, et al. Mask R-CNN[C]// IEEE International Conference on Computer Vision. Washington D C: IEEE, 2017: 2961-2969.
[15]	YUAN L, QIU Z. Mask-RCNN with spatial attention for pedestrian segmentation in cyber-physical systems[J]. Comput Commun, 2021, 180: 109-114. doi: 10.1016/j.comcom.2021.09.002
[16]	AHMED B, GULLIVER T A, ALZAHIR S. Image splicing detection using mask-RCNN[J]. Signal Image Video P, 2020, 14(5): 1035-1042.
[17]	ZHANG H Y, XU X Y, MA X F, et al. Mask-RCNN recognition method of composite fold shape in ultrasound images[J]. Acta Physica Sinica, 2022, 71(1): 203-210.
	张海燕, 徐心语, 马雪芬, 等. 超声图像中复合材料褶皱形态的Mask-RCNN识别方法[J]. 物理学报, 2022, 71(7): 203-210.
[18]	HAO S Y, LEE D H, ZHAO D. Sequence to sequence learning with attention mechanism for short-term passenger flow prediction in large-scale metro system[J]. Transport Res C-Emer, 2019, 107(C): 287-300.
[19]	CHEN H H, WU G D, LI J X, et al. Research advances on deep learning recommendation based on attention mechanism[J]. Comput Eng Sci, 2021, 43(2): 370-380.
	陈海涵, 吴国栋, 李景霞, 等. 基于注意力机制的深度学习推荐研究进展[J]. 计算机工程与科学, 2021, 43(2): 370-380.
[20]	DAI Y, LIU W B, DONG X Y, et al. U-NET CSF cells segmentation based on attention mechanism[J]. Journal of Northeastern University(Natural Science), 2022, 43(7): 944-950. doi: 10.12068/j.issn.1005-3026.2022.07.005
	代茵, 刘维宾, 董昕阳, 等. 基于注意力机制的U-Net脑脊液细胞分割[J]. 东北大学学报(自然科学版), 2022, 43(7): 944-950. doi: 10.12068/j.issn.1005-3026.2022.07.005
[21]	KAREL Z. Contrast limited adaptive histogram equalization[C]// Graphics Gems IV. San Diego: Academic Press Professional, 1994: 474-485.
[22]	LIU X, LIU J B. Mammary image enhancement based on contrast limited adaptive histogram equalization[J]. Computer Engineering and Applications, 2008, 44(10): 173-175.
	刘轩, 刘佳宾. 基于对比度受限自适应直方图均衡的乳腺图像增强[J]. 计算机工程与应用, 2008, 44(10): 173-175.
[23]	LIN T Y, DOLLAR P, GIRSHICK R, et al. Feature pyramid networks for object detection[J]. IEEE Computer Society, 2017. DOI: 10.1109/CVPR.2017.106. doi: 10.1109/CVPR.2017.106
[24]	SIMONYAN K, ZISSERMAN A. Very deep convolutional networks for large-scale image recognition[C]// International Conference on Learning Representations (ICLR), 2015: 1-14.
[25]	HE K, ZHANG X, REN S, et al. Deep residual learning for image recognition[C]// 2016 IEEE Conference on Computer Vision and Pattern Recognition (CVPR), IEEE, 2016: 770-778.
[26]	LONG J, SHELHAMER E, DARRELL T. Fully convolutional networks for semantic segmentation[J]. IEEE Transactions on Pattern Analysis and Machine Intelligence, 2015, 39(4): 640-651. DOI:10.1109/CVPR.2015.7298965. doi: 10.1109/CVPR.2015.7298965
[27]	SHEN X X, HOU X W, YIN C H. State attention in deep reinforcement learning[J]. CAAI transactions on intelligent systems, 2020, 15(2): 317-322.
	申翔翔, 侯新文, 尹传环. 深度强化学习中状态注意力机制的研究[J]. 智能系统学报, 2020, 15(2): 317-322.
[28]	WOO S, PARK J, LEE J Y, et al. CBAM: convolutional block attention module[M]// Computer Vision (ECCV 2018). Cham: Springer International Publishing, 2018: 3-19.
[29]	黄泽桑. 基于深度学习的目标检测技术研究[D]. 北京: 北京邮电大学, 2019.
[30]	PADILLA R, PASSOS W L, DIAS T L B, et al. A comparative analysis of object detection metrics with a companion open-source toolkit[J]. Electronics. 2021, 10(3): 279. doi: 10.3390/electronics10030279
[31]	MNIH V, HEESS N, GRAVES A. Recurrent models of visual attention[C]// Proceedings of the 27th International Conference on Neural Information Processing Systems, Cambridge: MIT Press, 2014, 2: 2204-2212.
[32]	LI H Y, LI C G, AN J B, et al. Attention mechanism improves CNN remote sensing image object detection[J]. Journal of Image and Graphics, 2019, 24(8): 1400-1408.
	李红艳, 李春庚, 安居白, 等. 注意力机制改进卷积神经网络的遥感图像目标检测[J]. 中国图象图形学报, 2019, 24(8): 1400-1408.
[33]	XU W H, PEI Y J, GAO D L, et al. Mass classification of breast mammogram based on attention mechanism and transfer learning[J]. Laser Optoelectronics Progress, 2021, 58(4): 146-154.
	许文慧, 裴以建, 郜冬林, 等. 基于注意力机制与迁移学习的乳腺钼靶肿块分类[J]. 激光与光电子学进展, 2021, 58(4): 146-154.
[34]	ZHENG F, CHEN X Z. Status of artificial intelligence in meningioma image[J]. Chin J Magn Reson Imaging, 2020, 11(10): 934-936.
	郑飞, 陈绪珠. 脑膜瘤影像人工智能应用进展[J]. 磁共振成像, 2020, 11(10): 934-936.
[35]	DONG Y D, WEI W, MA X L. Progress in application of stereotactic radiosurgery in treatment of acoustic neuroma[J]. Journal of Otology, 2020, 18(1): 25-32.
	董耀东, 魏薇, 马秀岚. 立体放射治疗技术在听神经瘤治疗中的应用进展[J]. 中华耳科学杂志, 2020, 18(1): 25-32.
[36]	ARTHURS B J, LAMOREAUX W T, MACKAY A R, et al. Gamma knife radiosurgery for vestibular schwannomas: tumor control and functional preservation in 70 patients.[J]. Am J Clin Oncol, 2011, 34(3): 265-269. doi: 10.1097/COC.0b013e3181dbc2ab
[37]	FOOTE K D, FRIEDMAN W A, BUATTI J M, et al. Analysis of risk factors associated with radiosurgery for vestibular schwannoma[J]. J Neurosurg, 2001, 95(3): 440-449. doi: 10.3171/jns.2001.95.3.0440 pmid: 11565866
[38]	RYKACZEWSKI B, ZABEK M. A meta-analysis of treatment of vestibular schwannoma using gamma knife radiosurgery[J]. Contemporary Oncology, 2014, 18(1): 60-66.

模型	检测指标		分割指标
模型	mAR	mAP	mAR	mAP
Resnet50	0.693	0.852	0.772	0.857
Resnet101	0.741	0.890	0.774	0.898
VGG19	0.725	0.914	0.808	0.922
Resnet101-CBAM	0.743	0.918	0.795	0.923
VGG19-CBAM	0.749	0.932	0.799	0.930