Please wait a minute...
Journal of ZheJiang University (Engineering Science)  2023, Vol. 57 Issue (2): 226-234    DOI: 10.3785/j.issn.1008-973X.2023.02.002
    
Medical image segmentation method based on multi-source information fusion
Chang-chun YANG1(),Zan-ting YE1,Ban-teng LIU1,2,Ke WANG2,3,*(),Hai-dong CUI4
1. School of Computer Science and Artificial Intelligence, Changzhou University, Changzhou 213164, China
2. College of Information Science and Technology, Zhejiang Shuren University, Hangzhou 310015, China
3. State Key Laboratory of Industrial Control Technology, Zhejiang University, Hangzhou 310027, China
4. Breast Surgery, First Affiliated Hospital, Zhejiang University, Hangzhou 310009, China
Download: HTML     PDF(1105KB) HTML
Export: BibTeX | EndNote (RIS)      

Abstract  

The segmentation model construction and training based on single source data may lead to insufficient segmentation accuracy due to the defects of various imaging methods in medical images. Aiming at this problem, a medical image segmentation method based on multi-source information fusion was proposed. The FFDM and DBT data sources in the breast tumour microcalcification cluster lesion were used as examples to verify the effectiveness of the proposed method. The Yolov4 region candidate network was used to screen the suspicious regions of the FFDM data. DBT image was preprocessed by using the suspicious region information. The preprocessed DBT image was used as the input of the improved U-Net model to achieve lesion segmentation. Finally, through the fusion strategy of fault segmentation results based on sequential similarity discrimination, the multi-slice results in DBT were combined to complete the final lesion segmentation. True positive rate of 98.52%, false positive rate of 10.45% and accuracy of 94.07% were obtained from the FFDM and DBT data of 20 patients by using this method. Results show that the medical image segmentation method based on multi-source information fusion can effectively utilize the advantages of multi-source data, and achieve the rapid and accurate segmentation of lesions. The method can provide a novel solution for intelligent medical image diagnosis and treatment.



Key wordsmedical image      neural network      segmentation      breast cancer      measurement technique     
Received: 02 August 2022      Published: 28 February 2023
CLC:  TP 391  
Fund:  浙江省“领雁”研发攻关计划资助项目(2022C03122);浙江省公益技术应用研究资助项目(LGF22F020006,LGF21F010004);浙江大学工业控制技术国家重点实验室开放课题资助项目(ICT2022B34)
Corresponding Authors: Ke WANG     E-mail: ycc@cczu.edu.cn;wangke1992@zju.edu.cn
Cite this article:

Chang-chun YANG,Zan-ting YE,Ban-teng LIU,Ke WANG,Hai-dong CUI. Medical image segmentation method based on multi-source information fusion. Journal of ZheJiang University (Engineering Science), 2023, 57(2): 226-234.

URL:

https://www.zjujournals.com/eng/10.3785/j.issn.1008-973X.2023.02.002     OR     https://www.zjujournals.com/eng/Y2023/V57/I2/226


基于多源信息融合的医学图像分割方法

医学图像中各成像方式存在自身缺陷,以单一源数据作为输入进行分割模型的构建与训练易导致病灶的分割准确率不足,因此提出基于多源信息融合的医学图像分割方法,并以乳腺癌微钙化簇病灶诊断中的FFDM与DBT这2类数据源为例,验证方法的有效性. 方法利用Yolov4区域候选网络对FFDM数据进行可疑区域筛选;根据同一病人FFDM可疑区域进行DBT图像预处理;以预处理后的DBT图像作为改进U-Net模型的输入实现病灶分割;通过基于序贯相似性判别的断层分割结果融合策略,综合DBT中多断层结果完成病灶最终分割. 方法在20例病人的FFDM与DBT数据上得到98.52%的真阳性率、10.45%的假阳性率与94.07%的精度. 结果表明,本研究提出的基于多源信息融合的医学图像分割方法,有效利用多源数据优势,最终实现病灶的快速精确分割,可以为医学图像诊疗智能化提供一种全新的解决方案.


关键词: 医学图像,  神经网络,  语义分割,  乳腺癌,  检测技术 
Fig.1 Diagram of medical image segmentation method based on multi-source information fusion
Fig.2 Diagram of microcalcification cluster before and after gamma transformation
Fig.3 Structure diagram of improved U-Net network based on residual structure
阈值 TPR/% FPR/% ACC/% Pr/% DCS/%
0.1 99.53 48.85 68.14 51.15 67.33
0.2 83.33 14.39 84.80 75.95 79.47
0.3 60.27 10.69 78.92 75.86 67.18
0.4 43.84 6.87 75.49 78.05 56.14
0.5 26.39 6.11 69.95 70.37 38.38
Tab.1 Results of microcalcification cluster lesions based on Yolov4
数据集 DBT数量 TPR/% FPR/% ACC/%
数据集1 828 92.86 27.22 77.54
数据集2 644 92.71 21.68 82.61
数据集3 586 93.94 11.16 89.42
数据集4 780 78.72 7.18 91.14
平均值 ? 89.56 16.81 85.18
Tab.2 Results of U-Net segmentation
Fig.4 Diagram of segmentation results before and after network improvement
数据集 DBT数量 TPR/% FPR/% ACC/%
数据集1 828 96.08 0.64 98.55
数据集2 644 98.00 0.45 99.07
数据集3 586 97.30 2.30 97.61
数据集4 780 97.87 0.29 99.49
平均值 ? 97.31 0.92 98.68
Tab.3 Results of improved U-Net based on residual structure segmentation
方法 TPR/% FPR/% ACC/% Pr/% DCS/%
DDBNs 90.31 15.71 83.68 51.66 66.67
AB-DT 90.35 11.12 91.75 90.09 91.54
Yolov4 99.12 47.76 76.30 68.12 77.95
联合检测方法-L 98.52 16.25 89.63 83.75 91.89
联合检测方法 98.52 10.45 94.07 90.54 94.37
Tab.4 Results of each method in detection task of microcalcified cluster lesions
Fig.5 Example diagram of detection method
[1]   IBTEHAZ N, RAHMAN S MultiResUNet: rethinking the U-Net architecture for multimodal biomedical image segmentation[J]. Neural Networks, 2020, 121: 74- 87
doi: 10.1016/j.neunet.2019.08.025
[2]   ZHENG T, HUANG Y, LIU Y, et al. CLRNet: cross layer refinement network for lane detection [C]// Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition. New Orleans: CVPR, 2022.
[3]   BAI X, HU Z, ZHU X, et al. Transfusion: robust lidar-camera fusion for 3d object detection with transformers [C]// Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition. New Orleans: CVPR, 2022.
[4]   KRUSE E, DOLLINGER M, SCHUTZENBERGER A, et al GlottisNetV2: temporal glottal midline detection using deep convolutional neural networks[J]. IEEE Journal of Translational Engineering in Health and Medicine, 2023, 11: 137- 144
[5]   DIAKOGIANNIS F, WALDNER F, CACCETTA P, et al ResUNet-a: a deep learning framework for semantic segmentation of remotely sensed data[J]. ISPRS Journal of Photogrammetry and Remote Sensing, 2020, 162: 94- 114
doi: 10.1016/j.isprsjprs.2020.01.013
[6]   DOSOVITSKIY A, BEYER L, KOLESNIKOV A, et al. An image is worth 16x16 words: transformers for image recognition at scale [EB/OL]. [2021-06-03]. https://arxiv.org/pdf/2010.11929.pdf.
[7]   LIU Z, LIN Y, CAO Y, et al. Swin transformer: hierarchical vision transformer using shifted windows [C]// Proceedings of the IEEE/CVF International Conference on Computer Vision. Montreal: ICCV, 2021.
[8]   刘清清, 周志勇, 范国华, 等 基于3D scSE-UNet的肝脏CT图像半监督学习分割方法[J]. 浙江大学学报: 工学版, 2021, 55 (11): 2033- 2044
LIU Qing-qing, ZHOU Zhi-yong, FAN Guo-hua, et al Semi-supervised learning segmentation method of liver CT images based on 3D scSE-UNet[J]. Journal of Zhejiang University: Engineering Science, 2021, 55 (11): 2033- 2044
[9]   黄毅鹏, 胡冀苏, 钱旭升, 等. SE-Mask-RCNN: 多参数MRI前列腺癌分割方法. 浙江大学学报: 工学版, 2021, 55(1): 203-212.
HUANG Yi-peng, HU Ji-su, QIAN Xu-sheng, et al. SE-Mask-RCNN: segmentation method for prostate cancer on multi-parametric MRI[J]. Journal of Zhejiang University: Engineering Science, 2021, 55(1): 203-212.
[10]   曹霖, 陈后金, 李居朋, 等 对比双侧视图信息的致密型乳腺X线图像肿块检测[J]. 计算机辅助设计与图形学学报, 2018, 30 (10): 1917- 1924
CAO Lin, CHEN Hou-jin, LI Ju-peng, et al Bilateral analysis of mass detection for dense mammograms[J]. Journal of Computer-Aided Design and Computer Graphics, 2018, 30 (10): 1917- 1924
[11]   王磊, 朱淼良, 邓丽萍, 等 一种基于二维粒子的自动检测乳腺钼靶片上微钙化点簇的方法[J]. 计算机研究与发展, 2009, 46 (9): 1438- 1445
WANG Lei, ZHU Miao-liang, DENG Li-ping, et al Automatic detection on clustered microcalcifications on mammograms based on 2D particles[J]. Journal of Computer Research and Development, 2009, 46 (9): 1438- 1445
[12]   LOIZIDOU K, SKOUROUMOUNI G, NIKOLAOU C, et al An automated breast micro-calcification detection and classification technique using temporal subtraction of mammograms[J]. IEEE Access, 2020, 8: 52785- 52795
doi: 10.1109/ACCESS.2020.2980616
[13]   BERNARD S, MULLER S, ONATIVIA J. Computer-aided microcalcification detection on digital breast tomosynthesis data: a preliminary evaluation [C]// International Workshop on Digital Mammography. Berlin: Springer, 2008.
[14]   WEI J, CHAN H, HADJIISKI L, et al Multichannel response analysis on 2D projection views for detection of clustered microcalcifications in digital breast tomosynthesis[J]. Medical Physics, 2014, 41 (4): 041913
doi: 10.1118/1.4868694
[15]   JEONG J, CHAE S, CHAE E Y, et al. Simplified computer-aided detection scheme of microcalcification clusters in digital breast tomosynthesis images [C]// 2016 38th Annual International Conference of the IEEE Engineering in Medicine and Biology Society (EMBC). Orlando: IEEE, 2016.
[16]   宋立新, 魏雪芹, 王乾, 等 结合判别式深度置信网络的乳腺图像微钙化簇区域检测[J]. 生物医学工程学杂志, 2021, 38 (2): 268- 275
SONG Li-xing, WEI Xue-qin, WANG Qian, et al Detection of mi-crocalcification clusters regions in mammograms combining discriminative deep belief networks[J]. Journal of Biomedical Engineering, 2021, 38 (2): 268- 275
doi: 10.7507/1001-5515.202001034
[17]   BOCHKOVSKIY A, WANG C, LIAO H M. Yolov4: optimal speed and accuracy of object detection[EB/OL]. [2020-04-23]. https://arxiv.org/pdf/2004.10934.pdf.
[18]   RONNEBERGER O, FISCHER P, BROX T. U-net: convolutional networks for biomedical image segmentation [C]// International Conference on Medical Image Computing and Computer-assisted Intervention. Munich: MICCAI, 2015.
[19]   JHA D, SMEDSRUD P H, RIEGLER M A, et al. Resunet++: an advanced architecture for medical image segmentation [C]// 2019 IEEE International Symposium on Multimedia. San Diego: ISM, 2019.
[20]   HE K, ZHANG X, REN S, et al. Deep residual learning for image recognition [C]// Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition. Las Vegas: CVPR, 2016.
[21]   SPRINGENBERG J, DOSOVITSKIY A, BROX T, et al. Striving for simplicity: the all convolutional net [EB/OL]. [2015-04-13]. https://arxiv.org/pdf/1412.6806.pdf.
[22]   申楠, 邢素霞, 何湘萍, 等 基于Adaboost-决策树算法的乳腺微钙化区域真假阳性检测[J]. 中国医学物理学杂志, 2021, 38 (8): 940- 945
SHEN Nan, XING Su-xia, HE Xiang-ping, et al True- and false-positive detections of breast microcalcifications based on Adaboost-decision tree algorithm[J]. Chinese Journal of Medical Physics, 2021, 38 (8): 940- 945
doi: 10.3969/j.issn.1005-202X.2021.08.004
[1] Ju-xiang ZENG,Ping-hui WANG,Yi-dong DING,Lin LAN,Lin-xi CAI,Xiao-hong GUAN. Graph neural network based node embedding enhancement model for node classification[J]. Journal of ZheJiang University (Engineering Science), 2023, 57(2): 219-225.
[2] Jun-chi MA,Xiao-xin DI,Zong-tao DUAN,Lei TANG. Survey on program representation learning[J]. Journal of ZheJiang University (Engineering Science), 2023, 57(1): 155-169.
[3] Hua HUANG,Qiu-ge ZHAO,Zai-xing HE,Jia-ran LI. Contour error control of two-axis system based on LSTM and Newton iteration[J]. Journal of ZheJiang University (Engineering Science), 2023, 57(1): 10-20.
[4] Li-zhao DAI,Wei CAO,Shan-chang YI,Lei WANG. Damage identification of concrete structure based on WPT-SVD and GA-BPNN[J]. Journal of ZheJiang University (Engineering Science), 2023, 57(1): 100-110.
[5] Wan-liang WANG,Tie-jun WANG,Jia-cheng CHEN,Wen-bo YOU. Medical image segmentation method combining multi-scale and multi-head attention[J]. Journal of ZheJiang University (Engineering Science), 2022, 56(9): 1796-1805.
[6] Xiao-liang YIN,Cheng QIAN. Global approximation of complex model based on adaptive sampling[J]. Journal of ZheJiang University (Engineering Science), 2022, 56(9): 1815-1823.
[7] Zheng-yin YANG,Jian HU,Jian-yong YAO,Ying-zhe SHA,Qiu-yu SONG. Fault-tolerant control based on adaptive neural network sliding mode observer[J]. Journal of ZheJiang University (Engineering Science), 2022, 56(8): 1656-1665.
[8] Jun HE,Ya-sheng ZHANG,Can-bin YIN. Operating modes identification of spaceborne SAR based on deep learning[J]. Journal of ZheJiang University (Engineering Science), 2022, 56(8): 1676-1684.
[9] Wei-hong NIU,Xiao-feng HUANG,Wei QI,Hai-bing YIN,Cheng-gang YAN. Fast stepwise all zero block detection algorithm for H.266/VVC[J]. Journal of ZheJiang University (Engineering Science), 2022, 56(7): 1285-1293, 1319.
[10] Ren-peng MO,Xiao-sheng SI,Tian-mei LI,Xu ZHU. Bearing life prediction based on multi-scale features and attention mechanism[J]. Journal of ZheJiang University (Engineering Science), 2022, 56(7): 1447-1456.
[11] Zhu-peng WEN,Jie CHEN,Lian-hua LIU,Ling-ling JIAO. Fault diagnosis of wind power gearbox based on wavelet transform and improved CNN[J]. Journal of ZheJiang University (Engineering Science), 2022, 56(6): 1212-1219.
[12] Jun-qi YU,Si-yuan YANG,An-jun ZHAO,Zhi-kun GAO. Hybrid prediction model of building energy consumption based on neural network[J]. Journal of ZheJiang University (Engineering Science), 2022, 56(6): 1220-1231.
[13] Li HE,Shan-min PANG. Face reconstruction from voice based on age-supervised learning and face prior information[J]. Journal of ZheJiang University (Engineering Science), 2022, 56(5): 1006-1016.
[14] Xue-qin ZHANG,Tian-ren LI. Breast cancer pathological image classification based on Cycle-GAN and improved DPN network[J]. Journal of ZheJiang University (Engineering Science), 2022, 56(4): 727-735.
[15] Yun-hao WANG,Ming-hui SUN,Yi XIN,Bo-xuan ZHANG. Robot tactile recognition system based on piezoelectric film sensor[J]. Journal of ZheJiang University (Engineering Science), 2022, 56(4): 702-710.