Please wait a minute...
浙江大学学报(工学版)  2023, Vol. 57 Issue (2): 226-234    DOI: 10.3785/j.issn.1008-973X.2023.02.002
计算机技术     
基于多源信息融合的医学图像分割方法
杨长春1(),叶赞挺1,刘半藤1,2,王柯2,3,*(),崔海东4
1. 常州大学 计算机与人工智能学院,江苏 常州 213164
2. 浙江树人学院 信息科技学院,浙江 杭州 310015
3. 浙江大学工业控制技术国家重点实验室,浙江 杭州 310027
4. 浙江大学第一附属医院 乳腺外科,浙江 杭州 310009
Medical image segmentation method based on multi-source information fusion
Chang-chun YANG1(),Zan-ting YE1,Ban-teng LIU1,2,Ke WANG2,3,*(),Hai-dong CUI4
1. School of Computer Science and Artificial Intelligence, Changzhou University, Changzhou 213164, China
2. College of Information Science and Technology, Zhejiang Shuren University, Hangzhou 310015, China
3. State Key Laboratory of Industrial Control Technology, Zhejiang University, Hangzhou 310027, China
4. Breast Surgery, First Affiliated Hospital, Zhejiang University, Hangzhou 310009, China
 全文: PDF(1105 KB)   HTML
摘要:

医学图像中各成像方式存在自身缺陷,以单一源数据作为输入进行分割模型的构建与训练易导致病灶的分割准确率不足,因此提出基于多源信息融合的医学图像分割方法,并以乳腺癌微钙化簇病灶诊断中的FFDM与DBT这2类数据源为例,验证方法的有效性. 方法利用Yolov4区域候选网络对FFDM数据进行可疑区域筛选;根据同一病人FFDM可疑区域进行DBT图像预处理;以预处理后的DBT图像作为改进U-Net模型的输入实现病灶分割;通过基于序贯相似性判别的断层分割结果融合策略,综合DBT中多断层结果完成病灶最终分割. 方法在20例病人的FFDM与DBT数据上得到98.52%的真阳性率、10.45%的假阳性率与94.07%的精度. 结果表明,本研究提出的基于多源信息融合的医学图像分割方法,有效利用多源数据优势,最终实现病灶的快速精确分割,可以为医学图像诊疗智能化提供一种全新的解决方案.

关键词: 医学图像神经网络语义分割乳腺癌检测技术    
Abstract:

The segmentation model construction and training based on single source data may lead to insufficient segmentation accuracy due to the defects of various imaging methods in medical images. Aiming at this problem, a medical image segmentation method based on multi-source information fusion was proposed. The FFDM and DBT data sources in the breast tumour microcalcification cluster lesion were used as examples to verify the effectiveness of the proposed method. The Yolov4 region candidate network was used to screen the suspicious regions of the FFDM data. DBT image was preprocessed by using the suspicious region information. The preprocessed DBT image was used as the input of the improved U-Net model to achieve lesion segmentation. Finally, through the fusion strategy of fault segmentation results based on sequential similarity discrimination, the multi-slice results in DBT were combined to complete the final lesion segmentation. True positive rate of 98.52%, false positive rate of 10.45% and accuracy of 94.07% were obtained from the FFDM and DBT data of 20 patients by using this method. Results show that the medical image segmentation method based on multi-source information fusion can effectively utilize the advantages of multi-source data, and achieve the rapid and accurate segmentation of lesions. The method can provide a novel solution for intelligent medical image diagnosis and treatment.

Key words: medical image    neural network    segmentation    breast cancer    measurement technique
收稿日期: 2022-08-02 出版日期: 2023-02-28
CLC:  TP 391  
基金资助: 浙江省“领雁”研发攻关计划资助项目(2022C03122);浙江省公益技术应用研究资助项目(LGF22F020006,LGF21F010004);浙江大学工业控制技术国家重点实验室开放课题资助项目(ICT2022B34)
通讯作者: 王柯     E-mail: ycc@cczu.edu.cn;wangke1992@zju.edu.cn
作者简介: 杨长春(1963—),男,教授,博士,从事数据挖掘研究. orcid.org/0000-0001-9567-630X. E-mail: ycc@cczu.edu.cn
服务  
把本文推荐给朋友
加入引用管理器
E-mail Alert
作者相关文章  
杨长春
叶赞挺
刘半藤
王柯
崔海东

引用本文:

杨长春,叶赞挺,刘半藤,王柯,崔海东. 基于多源信息融合的医学图像分割方法[J]. 浙江大学学报(工学版), 2023, 57(2): 226-234.

Chang-chun YANG,Zan-ting YE,Ban-teng LIU,Ke WANG,Hai-dong CUI. Medical image segmentation method based on multi-source information fusion. Journal of ZheJiang University (Engineering Science), 2023, 57(2): 226-234.

链接本文:

https://www.zjujournals.com/eng/CN/10.3785/j.issn.1008-973X.2023.02.002        https://www.zjujournals.com/eng/CN/Y2023/V57/I2/226

图 1  基于多源信息融合的医学图像分割方法总框图
图 2  微钙化簇伽马变换前、后效果图
图 3  基于残差结构的改进U-Net网络模型结构图
阈值 TPR/% FPR/% ACC/% Pr/% DCS/%
0.1 99.53 48.85 68.14 51.15 67.33
0.2 83.33 14.39 84.80 75.95 79.47
0.3 60.27 10.69 78.92 75.86 67.18
0.4 43.84 6.87 75.49 78.05 56.14
0.5 26.39 6.11 69.95 70.37 38.38
表 1  基于Yolov4的微钙化簇病灶检测结果
数据集 DBT数量 TPR/% FPR/% ACC/%
数据集1 828 92.86 27.22 77.54
数据集2 644 92.71 21.68 82.61
数据集3 586 93.94 11.16 89.42
数据集4 780 78.72 7.18 91.14
平均值 ? 89.56 16.81 85.18
表 2  U-Net网络分割结果
图 4  网络改进前、后分割结果对比图
数据集 DBT数量 TPR/% FPR/% ACC/%
数据集1 828 96.08 0.64 98.55
数据集2 644 98.00 0.45 99.07
数据集3 586 97.30 2.30 97.61
数据集4 780 97.87 0.29 99.49
平均值 ? 97.31 0.92 98.68
表 3  基于残差结构的改进U-Net网络分割结果
方法 TPR/% FPR/% ACC/% Pr/% DCS/%
DDBNs 90.31 15.71 83.68 51.66 66.67
AB-DT 90.35 11.12 91.75 90.09 91.54
Yolov4 99.12 47.76 76.30 68.12 77.95
联合检测方法-L 98.52 16.25 89.63 83.75 91.89
联合检测方法 98.52 10.45 94.07 90.54 94.37
表 4  微钙化簇病灶检测任务中各方法检测对比结果
图 5  检测方法实例图
1 IBTEHAZ N, RAHMAN S MultiResUNet: rethinking the U-Net architecture for multimodal biomedical image segmentation[J]. Neural Networks, 2020, 121: 74- 87
doi: 10.1016/j.neunet.2019.08.025
2 ZHENG T, HUANG Y, LIU Y, et al. CLRNet: cross layer refinement network for lane detection [C]// Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition. New Orleans: CVPR, 2022.
3 BAI X, HU Z, ZHU X, et al. Transfusion: robust lidar-camera fusion for 3d object detection with transformers [C]// Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition. New Orleans: CVPR, 2022.
4 KRUSE E, DOLLINGER M, SCHUTZENBERGER A, et al GlottisNetV2: temporal glottal midline detection using deep convolutional neural networks[J]. IEEE Journal of Translational Engineering in Health and Medicine, 2023, 11: 137- 144
5 DIAKOGIANNIS F, WALDNER F, CACCETTA P, et al ResUNet-a: a deep learning framework for semantic segmentation of remotely sensed data[J]. ISPRS Journal of Photogrammetry and Remote Sensing, 2020, 162: 94- 114
doi: 10.1016/j.isprsjprs.2020.01.013
6 DOSOVITSKIY A, BEYER L, KOLESNIKOV A, et al. An image is worth 16x16 words: transformers for image recognition at scale [EB/OL]. [2021-06-03]. https://arxiv.org/pdf/2010.11929.pdf.
7 LIU Z, LIN Y, CAO Y, et al. Swin transformer: hierarchical vision transformer using shifted windows [C]// Proceedings of the IEEE/CVF International Conference on Computer Vision. Montreal: ICCV, 2021.
8 刘清清, 周志勇, 范国华, 等 基于3D scSE-UNet的肝脏CT图像半监督学习分割方法[J]. 浙江大学学报: 工学版, 2021, 55 (11): 2033- 2044
LIU Qing-qing, ZHOU Zhi-yong, FAN Guo-hua, et al Semi-supervised learning segmentation method of liver CT images based on 3D scSE-UNet[J]. Journal of Zhejiang University: Engineering Science, 2021, 55 (11): 2033- 2044
9 黄毅鹏, 胡冀苏, 钱旭升, 等. SE-Mask-RCNN: 多参数MRI前列腺癌分割方法. 浙江大学学报: 工学版, 2021, 55(1): 203-212.
HUANG Yi-peng, HU Ji-su, QIAN Xu-sheng, et al. SE-Mask-RCNN: segmentation method for prostate cancer on multi-parametric MRI[J]. Journal of Zhejiang University: Engineering Science, 2021, 55(1): 203-212.
10 曹霖, 陈后金, 李居朋, 等 对比双侧视图信息的致密型乳腺X线图像肿块检测[J]. 计算机辅助设计与图形学学报, 2018, 30 (10): 1917- 1924
CAO Lin, CHEN Hou-jin, LI Ju-peng, et al Bilateral analysis of mass detection for dense mammograms[J]. Journal of Computer-Aided Design and Computer Graphics, 2018, 30 (10): 1917- 1924
11 王磊, 朱淼良, 邓丽萍, 等 一种基于二维粒子的自动检测乳腺钼靶片上微钙化点簇的方法[J]. 计算机研究与发展, 2009, 46 (9): 1438- 1445
WANG Lei, ZHU Miao-liang, DENG Li-ping, et al Automatic detection on clustered microcalcifications on mammograms based on 2D particles[J]. Journal of Computer Research and Development, 2009, 46 (9): 1438- 1445
12 LOIZIDOU K, SKOUROUMOUNI G, NIKOLAOU C, et al An automated breast micro-calcification detection and classification technique using temporal subtraction of mammograms[J]. IEEE Access, 2020, 8: 52785- 52795
doi: 10.1109/ACCESS.2020.2980616
13 BERNARD S, MULLER S, ONATIVIA J. Computer-aided microcalcification detection on digital breast tomosynthesis data: a preliminary evaluation [C]// International Workshop on Digital Mammography. Berlin: Springer, 2008.
14 WEI J, CHAN H, HADJIISKI L, et al Multichannel response analysis on 2D projection views for detection of clustered microcalcifications in digital breast tomosynthesis[J]. Medical Physics, 2014, 41 (4): 041913
doi: 10.1118/1.4868694
15 JEONG J, CHAE S, CHAE E Y, et al. Simplified computer-aided detection scheme of microcalcification clusters in digital breast tomosynthesis images [C]// 2016 38th Annual International Conference of the IEEE Engineering in Medicine and Biology Society (EMBC). Orlando: IEEE, 2016.
16 宋立新, 魏雪芹, 王乾, 等 结合判别式深度置信网络的乳腺图像微钙化簇区域检测[J]. 生物医学工程学杂志, 2021, 38 (2): 268- 275
SONG Li-xing, WEI Xue-qin, WANG Qian, et al Detection of mi-crocalcification clusters regions in mammograms combining discriminative deep belief networks[J]. Journal of Biomedical Engineering, 2021, 38 (2): 268- 275
doi: 10.7507/1001-5515.202001034
17 BOCHKOVSKIY A, WANG C, LIAO H M. Yolov4: optimal speed and accuracy of object detection[EB/OL]. [2020-04-23]. https://arxiv.org/pdf/2004.10934.pdf.
18 RONNEBERGER O, FISCHER P, BROX T. U-net: convolutional networks for biomedical image segmentation [C]// International Conference on Medical Image Computing and Computer-assisted Intervention. Munich: MICCAI, 2015.
19 JHA D, SMEDSRUD P H, RIEGLER M A, et al. Resunet++: an advanced architecture for medical image segmentation [C]// 2019 IEEE International Symposium on Multimedia. San Diego: ISM, 2019.
20 HE K, ZHANG X, REN S, et al. Deep residual learning for image recognition [C]// Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition. Las Vegas: CVPR, 2016.
21 SPRINGENBERG J, DOSOVITSKIY A, BROX T, et al. Striving for simplicity: the all convolutional net [EB/OL]. [2015-04-13]. https://arxiv.org/pdf/1412.6806.pdf.
22 申楠, 邢素霞, 何湘萍, 等 基于Adaboost-决策树算法的乳腺微钙化区域真假阳性检测[J]. 中国医学物理学杂志, 2021, 38 (8): 940- 945
SHEN Nan, XING Su-xia, HE Xiang-ping, et al True- and false-positive detections of breast microcalcifications based on Adaboost-decision tree algorithm[J]. Chinese Journal of Medical Physics, 2021, 38 (8): 940- 945
doi: 10.3969/j.issn.1005-202X.2021.08.004
[1] 曾菊香,王平辉,丁益东,兰林,蔡林熹,管晓宏. 面向节点分类的图神经网络节点嵌入增强模型[J]. 浙江大学学报(工学版), 2023, 57(2): 219-225.
[2] 马骏驰,迪骁鑫,段宗涛,唐蕾. 程序表示学习综述[J]. 浙江大学学报(工学版), 2023, 57(1): 155-169.
[3] 黄华,赵秋舸,何再兴,李嘉然. 基于LSTM与牛顿迭代的两轴系统轮廓误差控制[J]. 浙江大学学报(工学版), 2023, 57(1): 10-20.
[4] 戴理朝,曹威,易善昌,王磊. 基于WPT-SVD和GA-BPNN的混凝土结构损伤识别[J]. 浙江大学学报(工学版), 2023, 57(1): 100-110.
[5] 杨正银,胡健,姚建勇,沙英哲,宋秋雨. 基于自适应神经网络滑模观测器的容错控制[J]. 浙江大学学报(工学版), 2022, 56(8): 1656-1665.
[6] 贺俊,张雅声,尹灿斌. 基于深度学习的星载SAR工作模式鉴别[J]. 浙江大学学报(工学版), 2022, 56(8): 1676-1684.
[7] 牛伟宏,黄晓峰,祁伟,殷海兵,颜成钢. H.266/VVC分步全零块判决快速算法[J]. 浙江大学学报(工学版), 2022, 56(7): 1285-1293, 1319.
[8] 莫仁鹏,司小胜,李天梅,朱旭. 基于多尺度特征与注意力机制的轴承寿命预测[J]. 浙江大学学报(工学版), 2022, 56(7): 1447-1456.
[9] 温竹鹏,陈捷,刘连华,焦玲玲. 基于小波变换和优化CNN的风电齿轮箱故障诊断[J]. 浙江大学学报(工学版), 2022, 56(6): 1212-1219.
[10] 于军琪,杨思远,赵安军,高之坤. 基于神经网络的建筑能耗混合预测模型[J]. 浙江大学学报(工学版), 2022, 56(6): 1220-1231.
[11] 朱雅光,刘春潮,张亮. 基于虚拟运动神经网络的六足机器人行为控制[J]. 浙江大学学报(工学版), 2022, 56(6): 1107-1118.
[12] 何立,庞善民. 结合年龄监督和人脸先验的语音-人脸图像重建[J]. 浙江大学学报(工学版), 2022, 56(5): 1006-1016.
[13] 张雪芹,李天任. 基于Cycle-GAN和改进DPN网络的乳腺癌病理图像分类[J]. 浙江大学学报(工学版), 2022, 56(4): 727-735.
[14] 王云灏,孙铭会,辛毅,张博宣. 基于压电薄膜传感器的机器人触觉识别系统[J]. 浙江大学学报(工学版), 2022, 56(4): 702-710.
[15] 吴泽康,赵姗,李宏伟,姜懿芮. 遥感图像语义分割空间全局上下文信息网络[J]. 浙江大学学报(工学版), 2022, 56(4): 795-802.