Differentiation of Benign and Malignant Breast Lesions Based on Multimodal MRI and Deep Learning

doi:10.11938/cjmr20222969

Abstract

Abstract:

To improve the accuracy of computer aided diagnosis (CAD) based on dynamic contrast enhanced magnetic resonance imaging (DCE-MRI) in the differentiation of benign and malignant breast lesions, this study proposed a convolutional neural network model (AC_Ulsam_CNN) that is based on multi-modal feature fusion and the combination of asymmetric convolution (AC) and ultra-lightweight subspace attention module (Ulsam). Firstly, the transfer learning method was used to pre-train the model to screen out the most effective DCE-MRI time phase scan for differentiating benign and malignant breast lesions. Then, a network model based on AC_Ulsam_CNN was constructed based on the optimal time phase scan images to enhance the feature expression ability and robustness of the classification model. Finally, multimodal information such as breast imaging reporting and data system (BI-RADS) classification, apparent diffusion coefficient (ADC) and time-signal intensity curve (TIC) type were incorporated for feature fusion, to further improve the distinguishing performance of benign and malignant breast lesions. The performance of the model was verified by 5-fold cross-validation method, and the accuracy (ACC) of the proposed method was 0.826 and the area under the curve (AUC) was 0.877. The experimental results show that the proposed algorithm performs well in the classification of benign and malignant breast lesions with small sample size, and the fusion model based on multimodal data further enriches the feature information, thus this study improves the detection accuracy of lesions, and provides a new method for automatic differential diagnosis of benign and malignant breast lesions.

Key words: dynamic contrast enhanced magnetic resonance imaging, convolutional neural network, multimodal feature fusion, breast lesions, differentiation of benign and malignant lesions

CLC Number:

TP394.1

Yi-feng YANG, Zhang-xuan QI, Sheng-dong NIE. Differentiation of Benign and Malignant Breast Lesions Based on Multimodal MRI and Deep Learning[J]. Chinese Journal of Magnetic Resonance, 2022, 39(4): 401-412.

Figures/Tables 14

Table 1

Fig.1

Fig.2

Table 2

Fig.3

Table 3

Fig.4

Fig.5

Fig.6

Fig.7

Fig.8

Table 4

Table 5

Table 6

References 24

1	CORTAZAR P, ZHANG L, UNTCH M, et al Pathological complete response and long-term clinical benefit in breast cancer: the CTNeoBC pooled analysis[J]. Lancet, 2014, 384 (9938): 164- 172. doi: 10.1016/S0140-6736(13)62422-8
2	刘璐, 郑新宇乳腺良性病变的组织学分型及其乳腺癌风险[J]. 中国实用外科杂志, 2016, 36 (7): 720- 724.
	LIU L, ZHENG X Y Histologic types of benign breast disease and the risk of breast cancer[J]. Chinese Journal of Practical Surgery, 2016, 36 (7): 720- 724.
3	GIGER M L, KARSSEMEIJER N, SCHNABEL J A Breast image analysis for risk assessment, detection, diagnosis, and treatment of cancer[J]. Annu Rev Biomed Eng, 2013, 15, 327- 357. doi: 10.1146/annurev-bioeng-071812-152416
4	MANI S, CHEN Y K, ARLINGHAUS L R, et al. Early prediction of the response of breast tumors to neoadjuvant chemotherapy using quantitative MRI and machine learning[C]// AMIA Annu Symp Proc, 2011: 868-877.
5	LO GULLO R, ESKREIS-WINKLER S, MORRIS E A, et al Machine learning with multiparametric magnetic resonance imaging of the breast for early prediction of response to neoadjuvant chemotherapy[J]. Breast, 2020, 49, 115- 122. doi: 10.1016/j.breast.2019.11.009
6	ZHOU Y J, XU J X, LIU Q G, et al A radiomics approach with CNN for shear-wave elastography breast tumor classification[J]. IEEE Trans Biomed Eng, 2018, 65 (9): 1935- 1942. doi: 10.1109/TBME.2018.2844188
7	TANAKA H, CHIU S W, WATANABE T, et al Computer-aided diagnosis system for breast ultrasound images using deep learning[J]. Phys Med Biol, 2019, 64 (23): 235013. doi: 10.1088/1361-6560/ab5093
8	SAMALA R K, CHAN H P, HADJIISKI L, et al Breast cancer diagnosis in digital breast tomosynthesis: Effects of training sample size on multi-stage transfer learning using deep neural nets[J]. IEEE Trans Biomed Eng, 2019, 38 (3): 686- 696.
9	DALMIS M U, GUBERN-MERIDA A, VREEMANN S, et al Artificial intelligence-based classification of breast lesions imaged with a multiparametric breast mri protocol with ultrafast DCE-MRI, T₂, and DWI[J]. Invest Radiol, 2019, 54 (6): 325- 332. doi: 10.1097/RLI.0000000000000544
10	SARITAS I Prediction of breast cancer using artificial neural networks[J]. J Med Syst, 2012, 36 (5): 2901- 2907. doi: 10.1007/s10916-011-9768-0
11	霍璐, 胡晓欣, 肖勤, 等基于nnU-Net的乳腺DCE-MR图像中乳房和腺体自动分割[J]. 波谱学杂志, 2021, 38 (3): 367- 380.
	HUO L, HU X X, XIAO Q, et al Automatic segmentation of breast and fibroglandular tissues in DCE-MR images based on nnU-Net[J]. Chinese J Magn Reson, 2021, 38 (3): 367- 380.
12	ANTROPOVA N, HUYNH B Q, GIGER M L A deep feature fusion methodology for breast cancer diagnosis demonstrated on three imaging modality datasets[J]. Med Phys, 2017, 44 (10): 5162- 5171. doi: 10.1002/mp.12453
13	刘可文, 刘紫龙, 汪香玉, 等基于级联卷积神经网络的前列腺磁共振图像分类[J]. 波谱学杂志, 2020, 37 (2): 152- 161.
	LIU K W, LIU Z L, WANG X Y, et al Prostate cancer diagnosis based on cascaded convolutional neural networks[J]. Chinese J Magn Reson, 2020, 37 (2): 152- 161.
14	严冬, 陈再彦 MR动态增强显像在乳腺肿瘤良恶性鉴别诊断中的价值分析[J]. 影像研究与医学应用, 2020, 4 (9): 72- 73.
	YAN D, CHEN Z Y Value analysis of MR dynamic enhanced imaging in differential diagnosis of benign and malignant breast tumors[J]. Journal of Imaging Research and Medical, 2020, 4 (9): 72- 73.
15	SILBERMAN H, SHETH P A, PARISKY Y R, et al Modified bi-rads scoring of breast imaging findings improves clinical judgment[J]. Breast J, 2015, 21 (6): 642- 650. doi: 10.1111/tbj.12492
16	PAN S J, YANG Q J I T O K, ENGINEERING D A survey on transfer learning[J]. IEEE T Knowl Data En, 2009, 22 (10): 1345- 1359.
17	SZEGEDY C, VANHOUCKE V, IOFFE S, et al. Rethinking the inception architecture for computer vision[C]//Proceedings of the IEEE conference on computer vision and pattern recognition. 2016: 2818-2826.
18	SINGH V K, ROMANI S, RASHWAN H A, et al. Conditional generative adversarial and convolutional networks for X-ray breast mass segmentation and shape classification[C]//International Conference on Medical Image Computing and Computer-Assisted Intervention. Springer, Cham, 2018: 833-840.
19	NUNES A P, SILVA A C, DE PAIVA A C, et al Detection of masses in mammographic images using geometry, simpson's diversity index and SVM[J]. Int J Signal Imaging, 2010, 3 (1): 40- 51.
20	HUYNH B Q, LI H, GIGER M L Digital mammographic tumor classification using transfer learning from deep convolutional neural networks[J]. J Med Imaging, 2016, 3 (3): 034501.
21	DING X H, GUO Y C, DING G G, et al. ACNet: Strengthening the kernel skeletons for powerful cnn via asymmetric convolution blocks[C]// Proceedings of the IEEE/CVF International Conference on Computer Vision, 2019, 1911-1920.
22	SAINI R, JHA N K, DAS B, et al. Ulsam: Ultra-lightweight subspace attention module for compact convolutional neural networks[C]// Proceedings of the IEEE/CVF Winter Conference on Applications of Computer Vision, 2020, 1627-1636.
23	张丽娜, 赵作伟, 宋清伟, 等应用计算机辅助诊断技术评价血流动力学特征在乳腺MRI中的价值[J]. 中华放射学杂志, 2012, 11, 998- 1001.
	ZHANG L N, ZHAO Z W, ZHU Q W, et al Values of kinetic features measured by computer-aided diagnosis for breast MRI[J]. Chinese Journal of Radiology, 2012, 11, 998- 1001.
24	GUO Y, CAI Y Q, CAI Z L, et al Differentiation of clinically benign and malignant breast lesions using diffusion-weighted imaging[J]. J Magn Reson Imaging, 2002, 16 (2): 172- 178.

良性病变		恶性病变
病变表现	病例数	病变表现	病例数
纤维腺瘤	48	粘液癌	9
乳腺病	43	导管原位癌	21
导管内乳头状瘤	16	浸润性导管癌	28
导管扩张	6	乳头状癌	9
导管上皮增生	6	浸润性癌	91
累计	119	累计	158

时序类别	时序相位名称
单一时序	DynS0、DynS1、DynS2、DynS3、DynS4、SdynS0、SdynS1、SdynS2、SdynS3
组合时序	DynS012、DynS013、DynS014、DynS023、DynS024、DynS034、DynS123、DynS124、 DynS134、DynS234、SdynS012、SdynS013、SdynS123

#	输入层	输入尺寸	卷积核尺寸	通道数	步长	输出尺寸
（1）	Conv1+BN+LeakyReLU	150 × 150 × 3	3 × 3	16	1	150 × 150 × 16
（2）	Conv2+BN+LeakyReLU	150 × 150 × 16	1 × 1	8	1	150 × 150 × 8
（3）	Conv3+BN+LeakyReLU	150 × 150 × 8	3 × 3	8	1	150 × 150 × 8
（4）	Conv4+BN+LeakyReLU	150 × 150 × 8	1 × 1	16	1	150 × 150 × 16
（5）	Conv5+BN+LeakyReLU	150 × 150 × 16	1 × 1	16	1	150 × 150 × 16
（6）	Add1	150 × 150 × 16	-	-	-	150 × 150 × 16
（7）	Pool1	150 × 150 × 16	3 × 3	-	3	50 × 50 × 16
（8）	Conv6	50 × 50 × 16	1 × 1	8	1	50 × 50 × 8
（9）	AC Block1	50 × 50 × 8	-	8	-	50 × 50 × 16
（10）	Conv7	50 × 50 × 16	1 × 1	32	1	50 × 50 × 32
（11）	Conv8	50 × 50 × 16	1 × 1	32	1	50 × 50 × 32
（12）	Add2	50 × 50 × 32	-	-	-	50 × 50 × 32
（13）	Pool2	50 × 50 × 32	3 × 3	-	3	16 × 16 × 32
（14）	Conv9	16 × 16 × 32	1 × 1	32	1	16 × 16 × 32
（15）	AC Block2	16 × 16 × 32	-	32	-	16 × 16 × 64
（16）	Conv10	16 × 16 × 64	1 × 1	128	1	16 × 16 × 128
（17）	Conv11	50 × 50 × 32	3 × 3	16	3	16 × 16 × 16
（18）	Conv12	16 × 16 × 32	1 × 1	128	1	16 × 16 × 128
（19）	Add3	16 × 16 × 128	-	-	-	16 × 16 × 128
（20）	Concatenate1	16 × 16 × 128 16 × 16 × 16	-	-	-	16 × 16 × 144
（21）	Ulsam Block1	16 × 16 × 144	-	-	-	16 × 16 × 144
（22）	Ulsam Block2	16 × 16 × 144	-	-	-	16 × 16 × 144
（23）	Concatenate2	16 × 16 × 144 16 × 16 × 144	-	-	-	16 × 16 × 288
（24）	Conv13+BN+LeakyReLU	16 × 16 × 288	1 × 1	128	1	16 × 16 × 128
-	GAP	16 × 16 × 128	-	-	-	4096
-	Dense	4096	-	-	-	16
-	Sigmoid	16	-	-	-	1

网络模型结构	时序	ACC	AUC	Sens	Spec
InceptionV3	DynS4	0.725	0.730	0.763	0.620
InceptionV3	DynS024	0.749	0.775	0.789	0.664
AC_Ulsam_CNN	DynS4	0.771	0.784	0.733	0.790
AC_Ulsam_CNN	DynS024	0.762	0.826	0.783	0.758

特征类型	ACC	AUC	Sens	Spec
DynS024	0.762	0.826	0.783	0.758
DynS024+BI-RADS	0.769	0.840	0.791	0.754
DynS024+ADC	0.807	0.863	0.822	0.792
DynS024+TIC	0.778	0.829	0.810	0.746
DynS024+BI-RADS+ADC+TIC	0.826	0.877	0.861	0.783