Multidimensional Information Fusion Method for Meniscal Tear Classification Based on CNN-SVM

doi:10.11938/cjmr20233076

Abstract

Abstract:

Aiming to address the problem of low classification accuracy caused by the different shapes of meniscus tears in the computer-aided diagnosis (CAD) system for meniscus, a multidimensional information fusion network (MDIFNet) model for menissus tear classification was proposed. Firstly, a convolutional neural network (CNN) architecture consisting of four sub-networks was used to obtain meniscus feature information from different perspectives and dimensions. Simultaneously, multi-scale attention mechanism was proposed to enrich fine-grained features. Finally, a multi kernel model based on support vector machines (SVM) was constructed as the final classifier. The experimental results on the MRNet dataset show that the proposed method has a meniscal tear classification accuracy of 0.782, which has promotion compared to the existing state-of-the-art meniscus tear classification methods based on deep learning.

Key words: meniscal tear, multi kernel learning, multi-view learning, magnetic resonance imaging, deep learning

CLC Number:

O482.53

LAI Jiawen, WANG Yuling, CAI Xiaoyu, ZHOU Lihua. Multidimensional Information Fusion Method for Meniscal Tear Classification Based on CNN-SVM[J]. Chinese Journal of Magnetic Resonance, 2023, 40(4): 423-434.

Figures/Tables 13

Fig. 1

Fig. 2

Fig. 3

Fig. 4

Fig. 5

Fig. 6

Table 1

Fig. 7

Table 2

Table 3

Table 4

Table 5

Table 6

References 28

[1]	GHOSH P, TAYLOR T K F. The Knee joint meniscus: a fibrocartilage of some distinction[J]. Clin Orthop Relat R, 1987, 224: 52-63.
[2]	MARKES A R, HODAX J D, MA C B. Meniscus form and function[J]. Clin Sport Med, 2020, 39(1): 1-12. doi: S0278-5919(19)30068-7 pmid: 31767101
[3]	HUTCHINSON I D, MORAN C J, POTTER H G, et al. Restoration of the meniscus: form and function[J]. Am J Sport Med, 2014, 42(4): 987-998. doi: 10.1177/0363546513498503
[4]	BEAUFILS P, PUJOL N. Management of traumatic meniscal tear and degenerative meniscal lesions. Save the meniscus[J]. Orthop Traumatol-Sur, 2017, 103(8): S237-S244.
[5]	OZEKI N, KOGA H, SEKIYA I. Degenerative meniscus in knee osteoarthritis: from pathology to treatment[J]. Life, 2022, 12(4): 603. doi: 10.3390/life12040603
[6]	OAKDEN-RAYNER L, CARNEIRO G, BESSEN T, et al. Precision radiology: predicting longevity using feature engineering and deep learning methods in a radiomics framework[J]. Sci Rep, 2017, 7(1): 1648. doi: 10.1038/s41598-017-01931-w
[7]	JIN W, LI X, FATEHI M, et al. Guidelines and evaluation of clinical explainable AI in medical image analysis[J]. Med Image Anal, 2023, 84: 102684. doi: 10.1016/j.media.2022.102684
[8]	FARUCH-BILFELD M, LAPÈGUE F, CHIAVASSA H, et al. Imaging of meniscus and ligament injuries of the knee[J]. Diagn Interv Imag, 2016, 97(7-8): 749-765.
[9]	NGUYEN J C, DE SMET A A, GRAF B K, et al. MR imaging-based diagnosis and classification of meniscal tears[J]. Radiographics, 2014, 34(4): 981-999. doi: 10.1148/rg.344125202 pmid: 25019436
[10]	BONIATIS I, PANAYIOTAKIS G, PANAGIOTOPOULOS E. A computer-based system for the discrimination between normal and degenerated menisci from magnetic resonance images[C]// 2008 IEEE International Workshop on Imaging Systems and Techniques, Chania, Greece: IEEE, 2008: 335-339.
[11]	KÖSE C, GENÇALIOĞLU O, ŞEVIK U. An automatic diagnosis method for the knee meniscus tears in MR images[J]. Expert Syst Appl, 2009, 36(2): 1208-1216. doi: 10.1016/j.eswa.2007.11.036
[12]	FU J C, LIN C C, WANG C N, et al. Computer-aided diagnosis for knee meniscus tears in magnetic resonance imaging[J]. J Ind Prod Eng, 2013, 30(2): 67-77.
[13]	MA Y, XING C J, XIAO L. Knee joint image segmentation and model construction based on cascaded network[J]. Chinese J Magn Reson, 2022, 39(2): 184-195.
	马岩, 邢藏菊, 肖亮. 基于级联网络的膝关节图像分割与模型构建[J]. 波谱学杂志, 2022, 39(02): 184-195.
[14]	LECUN Y, BENGIO Y, HINTON G. Deep learning[J]. Nature, 2015, 521(7553): 436-444. doi: 10.1038/nature14539
[15]	ZEILER M D, FERGUS R. Visualizing and understanding convolutional networks[C]// Computer Vision-ECCV 2014: 13th European Conference, Zurich, Switzerland: Springer International Publishing, 2014: 818-833.
[16]	HU Y W, SU X Y, KE X T, et al. The research progress of diagnosing meniscus injury in MRI based on deep learning[J]. Chin J Magn Reson Imaging, 2022, 13(05):167-170.
	胡伟艺, 苏娴彦, 柯晓婷, 等. 基于深度学习的MRI诊断半月板损伤的研究进展[J]. 磁共振成像, 2022, 13(05): 167-170.
[17]	BIEN N, RAJPURKAR P, BALL R L, et al. Deep-learning-assisted diagnosis for knee magnetic resonance imaging: development and retrospective validation of MRNet[J]. PLoS Med, 2018, 15(11): e1002699. doi: 10.1371/journal.pmed.1002699
[18]	TSAI C H, KIRYATI N, KONEN E, et al. Knee injury detection using MRI with efficiently-layered network (ELNet)[C]// Medical Imaging with Deep Learning. Montréel, Canada: PMLR, 2020: 784-794.
[19]	FRITZ B, MARBACH G, CIVARDI F, et al. Deep convolutional neural network-based detection of meniscus tears: comparison with radiologists and surgery as standard of reference[J]. Skeletal Radiol, 2020, 49: 1207-1217. doi: 10.1007/s00256-020-03410-2 pmid: 32170334
[20]	RIZK B, BRAT H, ZILLE P, et al. Meniscal lesion detection and characterization in adult knee MRI: a deep learning model approach with external validation[J]. Phys Medica, 2021, 83: 64-71. doi: 10.1016/j.ejmp.2021.02.010
[21]	LU L X, ZHOU J Z, GUO Y C, et al. Prediction of knee injury based on multimodal fusion[J]. Comput Eng Appl, 2021, 57(09): 225-232.
	陆莉霞, 邹俊忠, 郭玉成, 等. 多模态融合的膝关节损伤预测[J]. 计算机工程与应用, 2021, 57(09): 225-232.
[22]	MA Y, QIN Y, LIANG C, et al. Visual cascaded-progressive convolutional neural network (C-PCNN) for diagnosis of meniscus injury[J]. Diagnostics, 2023, 13(12): 2049. doi: 10.3390/diagnostics13122049
[23]	HUNG T N K, VY V P T, TRI N M, et al. Automatic detection of meniscus tears using backbone convolutional neural networks on knee MRI[J]. J Magn Reson Imaging, 2023, 57(3): 740-749. doi: 10.1002/jmri.v57.3
[24]	HE K, ZHANG X, REN S, et al. Deep residual learning for image recognition[C]// Proceedings of the IEEE conference on computer vision and pattern recognition, Las Vegas, NV, USA: IEEE, 2016: 770-778.
[25]	BACH F R, LANCKRIET G R G, JORDAN M I. Multiple kernel learning, conic duality, and the SMO algorithm[C]// Proceedings of the twenty-first international conference on Machine learning. Banff, Alberta, Canada: ICML, 2004: 6.
[26]	SUBRAHMANYA N, SHIN Y C. Sparse multiple kernel learning for signal processing applications[J]. IEEE T Pattern Anal, 2009, 32(5): 788-798. doi: 10.1109/TPAMI.2009.98
[27]	DUNNHOFER M, MARTINEL N, MICHELONI C. Improving MRI-based knee disorder diagnosis with pyramidal feature details[C]// Medical Imaging with Deep Learning. Lübeck, Germany: PMLR, 2021: 131-147.
[28]	SHIN H, CHOI G S, SHON O J, et al. Development of convolutional neural network model for diagnosing meniscus tear using magnetic resonance image[J]. BMC Musculoskel Dis, 2022, 23(1): 1-9. doi: 10.1186/s12891-021-04954-7

名称	表达式
Linear Kernel	$K(x,y)={{x}^{\text{T}}}y$
Polynomial Kernel	$K(x,y)={{({{x}^{\text{T}}}y+c)}^{d}}$
Gaussian Kernel	$K(x,y)=\exp (-\frac{\|\|x-y\|{{\|}^{2}}}{2{{\sigma }^{2}}})$
Sigmoid Kernel	$K(x,y)=\tanh (\beta {{x}^{\text{T}}}y+\theta )$

MKL-SVM	特征	ROC-AUC	Accuracy	Sensitivity	Specificity
单核	2D轴面	0.661	0.650	0.750	0.573
	2D矢状面	0.728	0.716	0.719	0.647
	2D冠状面	0.679	0.675	0.712	0.648
多核	2D轴面+2D矢状面	0.718	0.719	0.771	0.649
	2D轴面+2D冠状面	0.681	0.677	0.759	0.635
	2D冠状面+2D矢状面	0.729	0.725	0.772	0.659
	2D三视图	0.740	0.732	0.775	0.664

特征	ROC-AUC	Accuracy	Sensitivity	Specificity
3D轴面	0.704	0.700	0.731	0.676
3D矢状面	0.776	0.743	0.710	0.765
3D冠状面	0.733	0.720	0.678	0.723

特征	分类器	ROC-AUC	Accuracy	Sensitivity	Specificity
3D矢状面+2D三视图	Softmax	0.831	0.775	0.762	0.795
3D矢状面+2D三视图	MKL-SVM	0.844	0.782	0.769	0.808

特征	注意力机制	ROC-AUC	Accuracy	Sensitivity	Specificity
3D矢状面+2D三视图	无注意力机制	0.826	0.765	0.752	0.790
	SE	0.835	0.772	0.759	0.791
	ECA	0.831	0.773	0.756	0.807
	CBAM	0.833	0.779	0.765	0.798
	MCBAM	0.844	0.782	0.769	0.808