基于3D VNetTrans的膝关节滑膜磁共振图像自动分割

doi:10.11938/cjmr20222988

摘要/Abstract

摘要：

膝关节是类风湿性关节炎（Rheumatoid Arthritis，RA）常见累及关节，膝关节滑膜的精准分割对RA诊断和治疗有重要影响，本文提出了一种基于VNet网络的改进算法对膝关节滑膜磁共振图像进行自动分割．首先对39名滑膜炎患者的膝关节磁共振图像进行数据预处理，通过将Transformer编码器嵌入VNet网络底部的方式构建VNetTrans网络，使用MemSwish激活函数进行训练. 最终模型平均Dice系数为0.758 5，HD为24.6 mm；相较于VNet，Dice系数提升0.083 6，HD距离减少10 mm．实验结果表明，该算法可对膝关节磁共振图像中滑膜增生区域实现较好的3D分割，具有诊断和监测RA发展过程的应用价值．

关键词: 磁共振图像, 医学图像分割, 深度学习, 滑膜炎

Abstract:

Knee joint is commonly hurt by rheumatoid arthritis (RA). Accurate segmentation of synovium is essential for the diagnosis and treatment of RA. This paper proposes an algorithm based on improved VNet for automatically segmenting knee joint synovial magnetic resonance images. Firstly, the knee joint magnetic resonance images of 39 patients with synovitis were preprocessed. VNetTrans was constructed by embedding Transformer at the bottom of VNet. The MemSwish activation function was used for training. The average Dice score of the final model is 0.758 5 and the HD is 24.6 mm. Compared with VNet, the proposed model increased Dice score by 0.083 6 and decreased HD by 10 mm. Experimental results demonstrated that the proposed algorithm achieved satisfying 3D segmentation of the synovial hyperplasia area in the knee magnetic resonance images. It can be utilized to facilitate the diagnosis and monitoring of RA.

Key words: magnetic resonance image, medical image segmentation, deep learning, synovitis

中图分类号:

O482.53

王颖珊, 邓奥琦, 毛瑾玲, 朱中旗, 石洁, 杨光, 马伟伟, 路青, 汪红志. 基于3D VNetTrans的膝关节滑膜磁共振图像自动分割[J]. 波谱学杂志, 2022, 39(3): 303-315.

Ying-shan WANG, Ao-qi DENG, Jin-ling MAO, Zhong-qi ZHU, Jie SHI, Guang YANG, Wei-wei MA, Qing LU, Hong-zhi WANG. Automatic Segmentation of Knee Joint Synovial Magnetic Resonance Images Based on 3D VNetTrans[J]. Chinese Journal of Magnetic Resonance, 2022, 39(3): 303-315.

图/表 11

图1

图2

图3

图4

图5

表1

表2

表3

图6

图7

表4

参考文献 28

1	SMOLEN J S, ALETAHA D, BARTON A, et alRheumatoid arthritis[J]. Nat Rev Dis Primers, 2018,4 (1): 18001. doi: 10.1038/nrdp.2018.1
2	JIN S Y, LI M T, FANG Y F, et alChinese registry of rheumatoid arthritis (CREDIT): II. prevalence and risk factors of major comorbidities in Chinese patients with rheumatoid arthritis[J]. Arthritis ResTher, 2017,19 (1): 251.
3	中华医学会风湿病学分会2018中国类风湿关节炎诊疗指南[J]. 中华内科杂志, 2018,57 (4): 242- 251. doi: 10.3760/cma.j.issn.0578-1426.2018.04.004
	ASSOCIATION C R2018 Chinese guideline for the diagnosis and treatment of rheumatoid arthritis[J]. Chin J Intern Med, 2018,57 (4): 242- 251. doi: 10.3760/cma.j.issn.0578-1426.2018.04.004
4	SUGIMOTO H, TAKEDA A, KANO SAssessment of disease activity in rheumatoid arthritis using magnetic resonance imaging: quantification of pannus volume in the hands[J]. Bri J Rheumatol, 1998,37 (8): 854- 861. doi: 10.1093/rheumatology/37.8.854
5	OSTERGAARD MDifferent approaches to synovial membrane volume determination by magnetic resonance imaging: manual versus automated segmentation[J]. Rheumatology, 1997,36 (11): 1166- 1177. doi: 10.1093/rheumatology/36.11.1166
6	SAKASHITA T, KAMISHIMA T, KOBAYASHI Y, et alAccurate quantitative assessment of synovitis in rheumatoid arthritis using pixel-by-pixel, time-intensity curve shape analysis[J]. Br J Radiol, 2016,89 (1061): 20151000. doi: 10.1259/bjr.20151000
7	FOTINOS-HOYER A K, GUERMAZI A, JARA H, et alAssessment of synovitis in the osteoarthritic knee: comparison between manual segmentation, semiautomated segmentation, and semiquantitative assessment using contrast-enhanced fat-suppressed T1-weighted MRI[J]. Magn Reson Med, 2010,64 (2): 604- 609. doi: 10.1002/mrm.22401
8	PERRY T A, GAIT A, O’NEILL T W, et alMeasurement of synovial tissue volume in knee osteoarthritis using a semiautomated MRI-based quantitative approach[J]. Magn Reson Med, 2019,81 (5): 3056- 3064. doi: 10.1002/mrm.27633
9	WANG A, FRANKE A, WESARG S. Semi-automatic segmentation of JIA-induced inflammation in MRI images of ankle joints[C]// Medical Imaging 2019: Image Processing, SPIE, 2019, 10949: 875-881.
10	ANDERSEN J K H, PEDERSEN J S, LAURSEN M S, et alNeural networks for automatic scoring of arthritis disease activity on ultrasound images[J]. RMD open, 2019,5 (1): e000891. doi: 10.1136/rmdopen-2018-000891
11	CHRISTENSEN A B H, JUST S A, ANDERSEN J K H, et alApplying cascaded convolutional neural network design further enhances automatic scoring of arthritis disease activity on ultrasound images from rheumatoid arthritis patients[J]. Ann Rheum Dise, 2020,79 (9): 1189- 1193. doi: 10.1136/annrheumdis-2019-216636
12	IQBAL I, SHAHZAD G, RAFIQ N, et alDeep learning-based automated detection of human knee joint's synovial fluid from magnetic resonance images with transfer learning[J]. IET Image Processing, 2020,14 (10): 1990- 1998. doi: 10.1049/iet-ipr.2019.1646
13	WONG L M, SHI L, XIAO F, et alFully automated segmentation of wrist bones on T2-weighted fat-suppressed MR images in early rheumatoid arthritis[J]. Quant Imag Med Surg, 2019,9 (4): 579. doi: 10.21037/qims.2019.04.03
14	魏小娜, 邢嘉祺, 王振宇, 等基于改进U-Net的关节滑膜磁共振图像的分割[J]. 计算机应用, 2020,40 (11): 3340- 3345.
	WEI X N, XIN J Q, WANG Z Y, et alMagnetic resonance image segmentation of articular synovium based on improved U-Net[J]. Journal of Computer Applications, 2020,40 (11): 3340- 3345.
15	王振宇, 王颖珊, 毛瑾玲, 等基于Dense-UNet++的关节滑膜磁共振图像分割[J]. 波谱学杂志, 2022,39 (2): 208- 219.
	WANG Z Y, WANG Y S, MAO J L, et alMagnetic resonance images segmentation of synovium based on Dense-UNet++[J]. Chinese J Magn Reson, 2022,39 (2): 208- 219.
16	WANG W X, CHEN C, DING M, et al. TransBTS: Multimodal brain tumor segmentation using transformer[C]// International Conference on Medical Image Computing and Computer-Assisted Intervention, Springer, 2021: 109-119.
17	LONG J, SHELHAMER E, DARRELL T. Fully convolutional networks for semantic segmentation[C]// Proceedings of the IEEE conference on computer vision and pattern recognition, 2015: 3431-3440.
18	RONNEBERGER O, FISCHER P, BROX T. U-Net: Convolutional Networks for Biomedical Image Segmentation[C]// International Conference on Medical image computing and computer-assisted intervention, Cham: Springer, 2015: 234-241.
19	ÇIçEK Ö, ABDULKADIR A, LIENKAMP S S, et al. 3D U-Net: learning dense volumetric segmentation from sparse annotation[C]// International conference on medical image computing and computer-assisted intervention. Springer, 2016: 424-432.
20	MILLETARI F, NAVAB N, AHMADI S-A. V-net: Fully convolutional neural networks for volumetric medical image segmentation[C]// 2016 fourth international conference on 3D vision (3DV), IEEE, 2016: 565-571.
21	VASWANI A, SHAZEER N, PARMAR N, et al. Attention is all you need[C]// Proceedings of the Advances in Neural Information Processing Systems, 2017: 6000-6010.
22	DOSOVITSKIY A, BEYER L, KOLESNIKOV A, et al. An image is worth 16×16 words: Transformers for image recognition at scale[OL]. arXiv preprint arXiv: 2010.11929, 2020.
23	NAIR V, HINTON G E. Rectified linear units improve restricted boltzmann machines[C]// Proceedings of the 27th International Conference on International Conference on Machine Learning, Haifa, Israel: 2010: 807-814.
24	RAMACHANDRAN P, ZOPH B, LE Q V. Searching for activation functions[J]. arXiv preprint arXiv: 1710.05941, 2017.
25	TAN M, LE Q. Efficientnet: Rethinking model scaling for convolutional neural networks[C]// International conference on machine learning, PMLR, 2019: 6105-6114.
26	YUAN L, CHEN Y, WANG T, et al. Tokens-to-token vit: Training vision transformers from scratch on imagenet[C]// Proceedings of the IEEE/CVF International Conference on Computer Vision, 2021: 558-567.
27	YUSHKEVICH P A, GAO Y, GERIG G. ITK-SNAP: An interactive tool for semi-automatic segmentation of multi-modality biomedical images[C]// 2016 38th Annual International Conference of the IEEE Engineering in Medicine and Biology Society (EMBC), IEEE, 2016: 3342-3345.
28	HATAMIZADEH A, TANG Y, NATH V, et al. Unetr: Transformers for 3d medical image segmentation[C]// Proceedings of the IEEE/CVF Winter Conference on Applications of Computer Vision, 2022: 574-584.

模型	卷积核维度	Dice	HD/mm
UNet	2D	0.6489	89.8
VNet	2D	0.6640	52.5
UNet	3D	0.6528	72.2
VNet	3D	0.6749	34.6

模型	激活函数	Dice	HD/mm	训练耗时
UNet	ReLU	0.6528	72.2	23min
	Swish	0.6613	50.8	34min
	MemSwish	0.6670	37.4	30min
VNetTrans	ReLU	0.7140	33.6	5h58min
	Swish	0.7505	29.0	6h36min
	MemSwish	0.7585	24.6	6h21min

模型	Dice	Sensitivity	Specificity	RVD/%	HD/mm
UNet	0.6528	0.7828	0.9864	33.84	72.2
VNet	0.6749	0.7262	0.9908	9.21	34.6
TransBTS	0.6996	0.7186	0.9917	4.87	36.0
UNETR	0.6355	0.6560	0.9920	?3.31	68.8
VNetTrans	0.7585	0.7633	0.9943	?1.34	24.6

网络模型	激活函数	是否加入Transformer模块	Dice	Sensitivity	Specificity	RVD/%	HD/mm
VNet	ReLU	否	0.6749	0.7262	0.9908	9.21	34.6
VNet + MemSwish	MemSwish	否	0.6962	0.7580	0.9904	14.92	35.8
VNet + Transformer	ReLU	是	0.7140	0.7527	0.9913	2.52	33.6
This work	MemSwish	是	0.7585	0.7633	0.9943	?1.34	24.6

[1]	周勤, 王远军. 基于变分推断的磁共振图像群组配准[J]. 波谱学杂志, 2022, 39(3): 291-302.
[2]	常晓,蔡昕,杨光,聂生东. 生成对抗网络在医学图像转换领域的应用[J]. 波谱学杂志, 2022, 39(3): 366-380.
[3]	陈萌, 耿辰, 李郁欣, 耿道颖, 鲍奕仿, 戴亚康. 基于模糊标签和深度学习的TOF-MRA影像脑动脉瘤自动检测[J]. 波谱学杂志, 2022, 39(3): 267-277.
[4]	王振宇, 王颖珊, 毛瑾玲, 马伟伟, 路青, 石洁, 汪红志. 基于Dense-UNet++的关节滑膜磁共振图像分割[J]. 波谱学杂志, 2022, 39(2): 208-219.
[5]	林剑圣,王丽嘉. 结合生物力学模型重建左心室位移场[J]. 波谱学杂志, 2022, 39(1): 72-86.
[6]	霍璐,胡晓欣,肖勤,顾雅佳,褚旭,姜娈. 基于nnU-Net的乳腺DCE-MR图像中乳房和腺体自动分割[J]. 波谱学杂志, 2021, 38(3): 367-380.
[7]	刘鹏, 钟玉敏, 王丽嘉. 基于密集多尺度U-net网络的电影心脏磁共振图像右心室自动分割[J]. 波谱学杂志, 2020, 37(4): 456-468.
[8]	王婉婷, 苏适, 贾森, 梁栋, 王海峰. 基于虚拟线圈和卷积神经网络的多层同时激发图像重建[J]. 波谱学杂志, 2020, 37(4): 407-421.
[9]	赵尚义, 王远军. 基于磁共振图像和改进的UNet++模型区分阿尔茨海默症患者和健康人群[J]. 波谱学杂志, 2020, 37(3): 321-331.
[10]	宫进昌, 王宇, 王远军. 结合小波融合和深度学习的脑胶质瘤自动分割[J]. 波谱学杂志, 2020, 37(2): 131-143.
[11]	程慧涛, 王珊珊, 柯子文, 贾森, 程静, 丘志浪, 郑海荣, 梁栋. 基于深度递归级联卷积神经网络的并行磁共振成像方法[J]. 波谱学杂志, 2019, 36(4): 437-445.
[12]	苏新宇, 王丽嘉, 聂生东, 胡立伟, 钟玉敏. 基于心脏磁共振短轴电影图像的右心室分割新进展[J]. 波谱学杂志, 2019, 36(3): 377-391.
[13]	孙京文, 闫士举, 韩勇森, 宋成利. 基于脑部磁共振图像三维局部模式变换特征提取进行阿尔茨海默病病程预测分类[J]. 波谱学杂志, 2019, 36(3): 268-277.
[14]	邵丹丹, 王雪雪, 潘自来, 陈克敏, 张中帅, 袁莉莉, 许子悦, 陈磊, 王金红. 刀锋伪影校正技术在精神疾病患者海马MRI检查中的应用[J]. 波谱学杂志, 2019, 36(3): 261-267.
[15]	徐嘉文, 徐健, 周晓东, 张聪, 陈群. 基于Gadgetron平台的多GPU分布式磁共振图像重建[J]. 波谱学杂志, 2018, 35(3): 303-317.