Please wait a minute...
Journal of ZheJiang University (Engineering Science)  2021, Vol. 55 Issue (1): 203-212    DOI: 10.3785/j.issn.1008-973X.2021.01.024
    
SE-Mask-RCNN: segmentation method for prostate cancer on multi-parametric MRI
Yi-peng HUANG1,2(),Ji-su HU1,2,Xu-sheng QIAN1,2,Zhi-yong ZHOU2,Wen-lu ZHAO3,Qi MA3,Jun-kang SHEN3,Ya-kang DAI2,*()
1. School of Biomedical Engineering (Suzhou), University of Science and Technology of China, Suzhou 215163, China
2. Suzhou Institute of Biomedical Engineering and Technology, Chinese Academy of Sciences, Suzhou 215163, China
3. The Second Affiliated Hospital of Suzhou University, Suzhou 215000, China
Download: HTML     PDF(963KB) HTML
Export: BibTeX | EndNote (RIS)      

Abstract  

A new deep convolutional neural network model called SE-Mask-RCNN was proposed to automatically segment prostate cancer regions in multi-parametric magnetic resonance imaging (mp-MRI) images. The candidate regions were extracted from the feature maps, and the possible lesions were segmented within the candidate regions. The feature maps were obtained from apparent diffusion coefficient (ADC) maps and T2-weighted (T2W) images through two parallel convolution networks, which were fused to fully use complementary information. The squeeze-and-excitation block was employed to automatically boost effective features and suppress the useless features. The experiments were conducted on a dataset containing 140 patients. The experimental results showed that the proposed model achieved a Dice coefficient of 0.654, a sensitivity of 0.695, a specificity of 0.970, and a positive predictive value of 0.685. The proposed SE-Mask-RCNN can effectively improve the segmentation accuracy of prostate cancer lesions in mp-MRI images compared with U-net, V-net, Resnet50-U-net and Mask-RCNN.



Key wordsprostate cancer      deep learning      squeeze-and-excitation block (SE-block)      Mask-RCNN      multi-parametric magnetic resonance imaging (mp-MRI)     
Received: 02 January 2020      Published: 27 January 2021
CLC:  R 318  
Corresponding Authors: Ya-kang DAI     E-mail: huang96@mail.ustc.edu.cn;daiyk@sibet.ac.cn
Cite this article:

Yi-peng HUANG,Ji-su HU,Xu-sheng QIAN,Zhi-yong ZHOU,Wen-lu ZHAO,Qi MA,Jun-kang SHEN,Ya-kang DAI. SE-Mask-RCNN: segmentation method for prostate cancer on multi-parametric MRI. Journal of ZheJiang University (Engineering Science), 2021, 55(1): 203-212.

URL:

http://www.zjujournals.com/eng/10.3785/j.issn.1008-973X.2021.01.024     OR     http://www.zjujournals.com/eng/Y2021/V55/I1/203


SE-Mask-RCNN:多参数MRI前列腺癌分割方法

为了从多参数磁共振(mp-MRI)的前列腺区域中自动提取前列腺癌病灶区域,提出新的深度卷积神经网络模型SE-Mask-RCNN. 在特征图上搜索定位包含病灶的候选区域,基于候选区域实现病灶的精细分割.为了利用mp-MRI中的互补信息,通过2个并行卷积网络分别提取表观扩散系数(ADC)和T2加权(T2W)图像的特征图后进行融合,使用挤压与激励块自动提升融合特征图中的有效特征并抑制无效特征.在收集得到的140例数据上进行实验.结果表明,使用SE-Mask-RCNN得到前列腺癌病灶分割Dice系数为0.654,敏感度为0.695,特异度为0.970,阳性预测值为0.685.与U-net、V-net、Resnet50-U-net和Mask-RCNN等模型相比,SE-Mask-RCNN能够有效提升mp-MRI中前列腺癌病灶区域的分割精度.


关键词: 前列腺癌,  深度学习,  挤压与激励块(SE-block),  Mask-RCNN,  多参数磁共振成像(mp-MRI) 
Fig.1 Network structure of Mask-RCNN
Fig.2 Structure of SE-block
Fig.3 Model structure diagram of SE-Mask-RCNN
Fig.4 mp-MRI information fusion method improved by SE-block
Fig.5 Example of data preprocessing
网络模型 MRI类型 DSC 敏感度 特异度 PPV
U-net ADC 0.539 0.639 0.970 0.324
V-net ADC 0.525 0.707 0.945 0.472
Resnet50-U-net ADC 0.525 0.720 0.956 0.462
Mask-RCNN ADC 0.592 0.582 0.979 0.680
U-net T2W 0.370 0.651 0.886 0.283
V-net T2W 0.343 0.580 0.896 0.282
Resnet50-U-net T2W 0.349 0.624 0.902 0.276
Mask-RCNN T2W 0.403 0.395 0.962 0.473
U-net ADC+T2W 0.562 0.765 0.955 0.480
V-net ADC+T2W 0.553 0.763 0.949 0.476
Resnet50-U-net ADC+T2W 0.561 0.683 0.972 0.532
Mask-RCNN ADC+T2W 0.612 0.653 0.968 0.650
Tab.1 Quantitative comparison of different network model results of prostate cancer lesion segmentation
Fig.6 Examples of prostate cancer lesion segmentation results from different network models on mp-MRI
Fig.7 Four methods of mp-MRI information fusion
融合方式 卷积网络 DSC 敏感度 特异度 PPV
输入图像串联(Mask-RCNN) Resnet50 0.612 0.653 0.968 0.650
特征图相加 Resnet50 0.611 0.667 0.966 0.643
特征图串联 Resnet50 0.609 0.645 0.968 0.659
SE-特征图串联 Resnet50 0.630 0.676 0.970 0.663
输入图像串联 SE-Resnet 0.637 0.660 0.970 0.685
特征图相加 SE-Resnet 0.638 0.684 0.970 0.658
特征图串联 SE-Resnet 0.640 0.697 0.967 0.652
SE-特征图串联(SE-Mask-RCNN) SE-Resnet 0.654 0.695 0.970 0.685
Tab.2 Comparison of four methods of mp-MRI information fusion
Fig.8 Comparison of candidate regions and heat maps
融合方式 ADC SE-Resnet是否去掉SE-block T2W SE-Resnet是否去掉SE-block DSC 敏感度 特异度 PPV
SE-特征图串联 0.654 0.695 0.970 0.685
SE-特征图串联 0.633 0.722 0.960 0.634
SE-特征图串联 0.633 0.726 0.960 0.630
SE-特征图串联 0.621 0.641 0.975 0.670
Tab.3 Performance comparison of SE-block removal methods for convolutional networks
模型 运算时间/ms
U-net 78
V-net 84
Resnet50-U-net 85
Mask-RCNN 131
SE-Mask-RCNN 144
Tab.4 Average image processing time for different models
[1]   CHEN W, ZHENG R, BAADE P D, et al Cancer statistics in China, 2015[J]. CA: A Cancer Journal for Clinicians, 2016, 66 (2): 115- 132
doi: 10.3322/caac.21338
[2]   CHEN W, ZHENG R, ZENG H, et al Annual report on status of cancer in China, 2011[J]. Chinese Journal of Cancer Research, 2015, 27 (1): 2
[3]   陈金东 中国各类癌症的发病率和死亡率现状及发展趋势[J]. 遵义医学院学报, 2018, 41 (6): 653- 662
CHEN Jin-dong Trends of cancer incidence and mortality in China[J]. Journal of Zunyi Medical University, 2018, 41 (6): 653- 662
doi: 10.3969/j.issn.1000-2715.2018.06.001
[4]   王宁, 刘硕, 杨雷, 等 2018全球癌症统计报告解读[J]. 肿瘤综合治疗电子杂志, 2019, 5 (1): 87- 97
WANG Ning, LIU Shuo, YANG Lei, et al Interpretation on the report of global cancer statistics 2018[J]. Journal of Multidisciplinary Cancer Management (Electronic Version), 2019, 5 (1): 87- 97
[5]   SIEGEL R L, MILLER K D, JEMAL A Cancer statistics, 2019[J]. CA: A Cancer Journal for Clinicians, 2019, 69 (1): 7- 34
doi: 10.3322/caac.21551
[6]   PANEBIANCO V, BARCHETTI F, SCIARRA A, et al Multiparametric magnetic resonance imaging vs. standard care in men being evaluated for prostate cancer: a randomized study[J]. Urologic Oncology: Seminars and Original Investigations, 2015, 33 (1): 17.e1- 17.e7
doi: 10.1016/j.urolonc.2014.09.013
[7]   FüTTERER J J, BRIGANTI A, DE VISSCHERE P, et al Can clinically significant prostate cancer be detected with multiparametric magnetic resonance imaging? a systematic review of the literature[J]. European Urology, 2015, 68 (6): 1045- 1053
doi: 10.1016/j.eururo.2015.01.013
[8]   王建业, 陈鑫, 刘明, 等 MRI在前列腺癌诊断和治疗中的应用[J]. 磁共振成像, 2010, 1 (4): 253- 256
WANG Jian-ye, CHEN Xin, LIU Ming, et al MRI in clinical diagnosis and treatment of prostate cancer[J]. Chinese Journal of Magnetic Resonance Imaging, 2010, 1 (4): 253- 256
[9]   KOHL S, BONEKAMP D, SCHLEMMER H P, et al. Adversarial networks for the detection of aggressive prostate cancer [DB/OL]. [2017-02-26]. https://arxiv.org/abs/1702.08014.
[10]   蒋宏达, 叶西宁 一种改进的I-Unet网络的皮肤病图像分割算法[J]. 现代电子技术, 2019, 42 (12): 52- 56
JIANG Hong-da, YE Xi-ning An improved skin disease image segmentation algorithm based on I-Unet network[J]. Modern Electronics Technique, 2019, 42 (12): 52- 56
[11]   赖小波, 许茂盛, 徐小媚 多分类CNN的胶质母细胞瘤多模态MR图像分割[J]. 电子学报, 2019, 47 (8): 140- 149
LAI Xiao-bo, XU Mao-sheng, XU Xiao-mei Glioblastoma multiforme nulti-modal MRI images segmentation using multi-class CNN[J]. Acta Electronica Sinica, 2019, 47 (8): 140- 149
[12]   邢波涛, 李锵, 关欣 改进的全卷积神经网络的脑肿瘤图像分割[J]. 信号处理, 2018, 34 (8): 911- 922
XING Bo-tao, LI Qiang, GUAN Xin A brain tumor image segmentation method based on improved fully convolutional neural network[J]. Journal of Signal Processing, 2018, 34 (8): 911- 922
[13]   田萱, 王亮, 丁琪 基于深度学习的图像语义分割方法综述[J]. 软件学报, 2019, 30 (2): 250- 278
TIAN Xuan, WANG Liang, DING Qi Review of image semantic segmentation based on deep learning[J]. Journal of Software, 2019, 30 (2): 250- 278
[14]   RONNEBERGER O, FISCHER P, BROX T. U-net: convolutional networks for biomedical image segmentation [C] // International Conference on Medical Image Computing and Computer-Assisted Intervention. Munich: Springer, 2015: 234-241.
[15]   HE K, GKIOXARI G, DOLLáR P, et al. Mask R-CNN [C] // IEEE International Conference on Computer Vision. Washington D. C. : IEEE, 2017: 2961-2969.
[16]   TSEHAY Y K, LAY N S, ROTH H R, et al. Convolutional neural network based deep-learning architecture for prostate cancer detection on multi-parametric magnetic resonance images [C] // Medical Imaging 2017: Computer-Aided Diagnosis. Orlando: SPIE, 2017: 1013405.
[17]   YANG X, LIU C, WANG Z, et al Co-trained convolutional neural networks for automated detection of prostate cancer in multi-parametric MRI[J]. Medical Image Analysis, 2017, 42: 212- 227
doi: 10.1016/j.media.2017.08.006
[18]   YUAN Y, QIN W, BUYYOUNOUSKI M, et al Prostate cancer classification with multiparametric MRI transfer learning model[J]. Medical Physics, 2019, 46 (2): 756- 765
doi: 10.1002/mp.13367
[19]   HU J, SHEN L, SUN G. Squeeze-and-excitation networks [C] // Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition. Salt Lake City: IEEE, 2018: 7132-7141.
[20]   LIU P, WANG S, TURKBEY B, et al. A prostate cancer computer-aided diagnosis system using multimodal magnetic resonance imaging and targeted biopsy labels [C] // Medical Imaging 2013: Computer-Aided Diagnosis. Lake Buena Vista: SPIE, 2013: 86701G.
[21]   YU Q, LIU F, SONG Y Z, et al. Sketch me that shoe [C]// Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition. Las Vegas: IEEE, 2016: 799-807.
[22]   QIAN C, WANG L, GAO Y, et al In vivo MRI based prostate cancer localization with random forests and auto-context model[J]. Computerized Medical Imaging and Graphics, 2016, 52: 44- 57
doi: 10.1016/j.compmedimag.2016.02.001
[23]   MAZZETTI S, GIANNINI V, RUSSO F, et al Computer-aided diagnosis of prostate cancer using multi-parametric MRI: comparison between PUN and Tofts models[J]. Physics in Medicine and Biology, 2018, 63 (9): 095004
doi: 10.1088/1361-6560/aab956
[24]   MILLETARI F, NAVAB N, AHMADI S A. V-net: fully convolutional neural networks for volumetric medical image segmentation [C]// 2016 4th International Conference on 3D Vision. Stanford: IEEE, 2016: 565-571.
[25]   ISHIOKA J, MATSUOKA Y, UEHARA S, et al Computer‐aided diagnosis of prostate cancer on magnetic resonance imaging using a convolutional neural network algorithm[J]. BJU International, 2018, 122 (3): 411- 417
doi: 10.1111/bju.14397
[26]   陆洋, 柏根基 定量动态增强磁共振成像及其在前列腺癌诊断和治疗中的研究进展[J]. 磁共振成像, 2015, 6 (10): 796- 800
LU Yang, BO Gen-ji Progress of quantitative dynamic contrast-enhanced magnetic resonance imaging and in diagnosis and evaluation on the therapeutic response of prostate cancer[J]. Chinese Journal of Magnetic Resonance Imaging, 2015, 6 (10): 796- 800
doi: 10.3969/j.issn.1674-8034.2015.10.016
[1] Jia-hui XU,Jing-chang WANG,Ling CHEN,Yong WU. Surface water quality prediction model based on graph neural network[J]. Journal of ZheJiang University (Engineering Science), 2021, 55(4): 601-607.
[2] Hong-li WANG,Bin GUO,Si-cong LIU,Jia-qi LIU,Yun-gang WU,Zhi-wen YU. End context-adaptative deep sensing model with edge-end collaboration[J]. Journal of ZheJiang University (Engineering Science), 2021, 55(4): 626-638.
[3] Teng ZHANG,Xin-long JIANG,Yi-qiang CHEN,Qian CHEN,Tao-mian MI,Piu CHAN. Wrist attitude-based Parkinson's disease ON/OFF state assessment after medication[J]. Journal of ZheJiang University (Engineering Science), 2021, 55(4): 639-647.
[4] Li-feng XU,Hai-fan HUANG,Wei-long DING,Yu-lei FAN. Detection of small fruit target based on improved DenseNet[J]. Journal of ZheJiang University (Engineering Science), 2021, 55(2): 377-385.
[5] Hao-can XU,Ji-tuo LI,Guo-dong LU. Reconstruction of three-dimensional human bodies from single image by LeNet-5[J]. Journal of ZheJiang University (Engineering Science), 2021, 55(1): 153-161.
[6] Pu ZHENG,Hong-yang BAI,Wei LI,Hong-wei GUO. Small target detection algorithm in complex background[J]. Journal of ZheJiang University (Engineering Science), 2020, 54(9): 1777-1784.
[7] Qiao-hong CHEN,YI CHEN,Wen-shu Li,Yu-bo JIA. Clothing image classification based on multi-scale SE-Xception[J]. Journal of ZheJiang University (Engineering Science), 2020, 54(9): 1727-1735.
[8] Deng-wen ZHOU,Jin-yue TIAN,Lu-yao MA,Xiu-xiu SUN. Lightweight image semantic segmentation based on multi-level feature cascaded network[J]. Journal of ZheJiang University (Engineering Science), 2020, 54(8): 1516-1524.
[9] Tao MING,Dan WANG,Ji-chang GUO,Qiang LI. Breast cancer histopathological image classification using multi-scale channel squeeze-and-excitation model[J]. Journal of ZheJiang University (Engineering Science), 2020, 54(7): 1289-1297.
[10] Xu YAN,Xiao-liang FAN,Chuan-pan ZHENG,Yu ZANG,Cheng WANG,Ming CHENG,Long-biao CHEN. Urban traffic flow prediction algorithm based on graph convolutional neural networks[J]. Journal of ZheJiang University (Engineering Science), 2020, 54(6): 1147-1155.
[11] Zhou-fei WANG,Wei-na YUAN. Channel estimation and detection method for multicarrier system based on deep learning[J]. Journal of ZheJiang University (Engineering Science), 2020, 54(4): 732-738.
[12] Bing YANG,Wen-bo MO,Jin-liang YAO. 3D palmprint recognition by using local features and deep learning[J]. Journal of ZheJiang University (Engineering Science), 2020, 54(3): 540-545.
[13] Yan-jia HONG,Tie-bao MENG,Hao-jiang LI,Li-zhi LIU,Li LI,Shuo-yu XU,Sheng-wen GUO. Deep segmentation method of tumor boundaries from MR images of patients with nasopharyngeal carcinoma using multi-modality and multi-dimension fusion[J]. Journal of ZheJiang University (Engineering Science), 2020, 54(3): 566-573.
[14] Zi-yu JIA,You-fang LIN,Hong-jun ZHANG,Jing WANG. Sleep stage classification model based ondeep convolutional neural network[J]. Journal of ZheJiang University (Engineering Science), 2020, 54(10): 1899-1905.
[15] Wan-liang WANG,Xiao-han YANG,Yan-wei ZHAO,Nan GAO,Chuang LV,Zhao-juan ZHANG. Image enhancement algorithm with convolutional auto-encoder network[J]. Journal of ZheJiang University (Engineering Science), 2019, 53(9): 1728-1740.