|
|
Semi-supervised learning segmentation method of liver CT images based on 3D scSE-UNet |
Qing-qing LIU1,2( ),Zhi-yong ZHOU2,Guo-hua FAN3,Xu-sheng QIAN1,2,Ji-su HU1,2,Guang-qiang CHEN3,Ya-kang DAI2,4,*( ) |
1. School of Biomedical Engineering (Suzhou), Division of Life Sciences and Medicine, University of Science and Technology of China, Suzhou 215163, China 2. Suzhou Institute of Biomedical Engineering and Technology, Chinese Academy of Science, Suzhou 215163, China 3. The Second Affiliated Hospital of Suzhou University, Suzhou 215000, China 4. Jinan Guoke Medical Engineering Technology Development Limited Company, Jinan 250000, China |
|
|
Abstract A semi-supervised learning segmentation method based on 3D scSE-UNet was proposed aiming at the problem that segmentation network requires a large number of high-quality labels and it is difficult to obtain. A self-training semi-supervised learning framework is used and 3D scSE-UNet containing the improved concurrent spatial and channel squeeze and excitation module (scSE-block+) in 3D UNet is utilized as the segmentation network. The scSE-block+ can automatically learn effective features of an image from two aspects, image space and feature channel, and suppress redundant features, which helps to preserve more edge information. During the self-training process, dense conditional random field (CRF) is used to refine the segmentation results generated by 3D scSE-UNet, so as to improve the accuracy of the pseudo labels. The effectiveness of the proposed method was verified on LiTS17 Challenge and SLIVER07 dataset. When the labeled images accounted for 30% of the total images in the training set, the dice score of the proposed method was 0.941. Results show that the proposed semi-supervised learning segmentation method can achieve comparable segmentation results with the fully-supervised 3D UNet segmentation method, which effectively reduces the dependence on expert labeled data in liver CT images segmentation.
|
Received: 11 December 2020
Published: 05 November 2021
|
|
Fund: 国家重点研发计划资助项目(2018YFA0703101);中国科学院青年创新促进会资助项目(2021324);苏州市科技计划资助项目(SS201854);丽水市重点研发计划资助项目(2019ZDYF17);泉城5150人才计划资助项目;济南创新团队资助项目(2018GXRC017);江苏省医疗器械联合资金资助项目(SYC2020002) |
Corresponding Authors:
Ya-kang DAI
E-mail: 17865198623@163.com;daiyk@sibet.ac.cn
|
基于3D scSE-UNet的肝脏CT图像半监督学习分割方法
针对分割神经网络需要大量的高质量标签但较难获取的问题,提出基于3D scSE-UNet的半监督学习分割方法. 该方法使用自训练的半监督学习框架,将包含改进的并行空间/特征通道压缩和激励模块(scSE-block+)的3D scSE-UNet作为分割网络. scSE-block+可以从图像空间和特征通道2个方面自动学习图像的有效特征,抑制无用冗余特征,更好地保留图像边缘信息. 在自训练过程中加入全连接条件随机场,对分割网络产生的伪标签进行边缘细化,提升伪标签的精确度. 在LiTS17 Challenge和SLIVER07数据集上验证所提出方法的有效性. 当有标签图像占训练集总图像的30%时,所提方法的Dice相似系数(dice score)为0.941. 结果表明,所提出的半监督学习分割方法可以在仅使用少量标注数据的情况下,取得与全监督分割方法相当的分割效果,有效减轻肝脏CT图像分割对专家标注数据的依赖.
关键词:
半监督学习,
自训练,
3D UNet,
注意力模块,
全连接条件随机场
|
|
[1] |
LEO J Computer-aided surgery meets predictive, preventive and personalized medicine[J]. EPMA Journal, 2017, 8 (1): 1- 4
doi: 10.1007/s13167-017-0084-8
|
|
|
[2] |
ZHOU S, CHENG Y, TAMURA S Automated lung seg- mentation and smoothing techniques for inclusion of juxtapleural nodules and pulmonary vessels on chest CT images[J]. Biomedical Signal Processing and Control, 2014, 13: 62- 70
doi: 10.1016/j.bspc.2014.03.010
|
|
|
[3] |
SHI C, CHENG Y, WANG J, et al Low-rank and sparse decomposition based shape model and probabilistic atlas for automatic pathological organ segmentation[J]. Medical Image Analysis, 2017, 38: 30- 49
doi: 10.1016/j.media.2017.02.008
|
|
|
[4] |
SHI C, CHENG Y, LIU F, et al A hierarchical local region-based sparse shape composition for liver segmentation in CT scans[J]. Pattern Recognition, 2016, 50: 88- 106
doi: 10.1016/j.patcog.2015.09.001
|
|
|
[5] |
LI G, CHEN X, SHI F, et al Automatic liver segmentation based on shape constraints and deformable graph cut in CT images[J]. IEEE Transactions on Image Processing A Publication of the IEEE Signal Processing Society, 2015, 24 (12): 5315
doi: 10.1109/TIP.2015.2481326
|
|
|
[6] |
LIU Z, SONG Y, SHENG V, et al Liver CT sequence segmentation based with improved U-Net and graph cut[J]. Expert Systems with Applications, 2019, 126: 54- 63
doi: 10.1016/j.eswa.2019.01.055
|
|
|
[7] |
LU X, WU J, REN X, et al The study and application of the improved region growing algorithm for liver segmentation[J]. Optik-International Journal for Light and Electron Optics, 2014, 125 (9): 2142- 2147
doi: 10.1016/j.ijleo.2013.10.049
|
|
|
[8] |
RAFIEI S, KARIMI N, MIRMAHBOUB B, et al. Liver segmentation in abdominal CT images using probabilistic atlas and adaptive 3D region growing [C]// 2019 41st Annual International Conference of the IEEE Engineering in Medicine and Biology Society (EMBC). Berlin: IEEE, 2019.
|
|
|
[9] |
郑洲, 张学昌, 郑四鸣, 等 基于区域增长与统一化水平集的CT肝脏图像分割[J]. 浙江大学学报: 工学版, 2018, 52 (12): 145- 159 ZHENG Zhou, ZHANG Xue-chang, ZHENG Si-ming, et al Liver segmentation in CT images based on region-growing and unified level set method[J]. Journal of Zhejiang University: Engineering Science, 2018, 52 (12): 145- 159
|
|
|
[10] |
WANG J, CHENG Y, GUO C, et al Shape-intensity prior level set: combining probabilistic atlas and probability map constrains for automatic liver segmentation from abdominal CT images[J]. International Journal of Computer Assisted Radiology and Surgery, 2016, 11 (5): 817- 826
doi: 10.1007/s11548-015-1332-9
|
|
|
[11] |
LI X, CHEN H, QI X, et al H-DenseUNet: hybrid densely connected UNet for liver and tumor segmentation from CT volumes[J]. IEEE Transactions on Medical Imaging, 2018, 37 (12): 2663- 2674
doi: 10.1109/TMI.2018.2845918
|
|
|
[12] |
LONG J, SHELHAMER E, DARRELL T Fully convolutional networks for semantic segmentation[J]. IEEE Transactions on Pattern Analysis and Machine Intelligence, 2015, 39 (4): 640- 651
|
|
|
[13] |
孙明建, 徐军, 马伟, 等 基于新型深度全卷积网络的肝脏CT影像三维区域自动分割[J]. 中国生物医学工程学报, 2018, 37 (4): 385- 393 SUN Ming-jian, XU Jun, MA Wei, et al A new fully convolutional network for 3D liver region segmentation on CT images[J]. Chinese Journal of Biomedical Engineering, 2018, 37 (4): 385- 393
doi: 10.3969/j.issn.0258-8021.2018.04.001
|
|
|
[14] |
RONNEBERGER O, FISCHER P, BROX T. U-Net: convolutional networks for biomedical image segmentation [C]// International Conference on Medical Image Computing and Computer Assisted Intervention. Munich: Springer, 2015: 234-241.
|
|
|
[15] |
ÇIÇEK Ö, ABDULKADIR A, LIENKAMP S, et al. 3D U-Net: learning dense volumetric segmentation from sparse annotation [C]// International Conference on Medical Image Computing and Computer-Assisted Intervention. Athens: Springer, 2016: 424-432.
|
|
|
[16] |
CHRIST P, ELSHAER M, ETTLINGER F, et al. Automatic liver and lesion segmentation in CT using cascaded fully convolutional neural networks and 3D conditional random fields [C]// International Conference on Medical Image Computing and Computer-Assisted Intervention. Athens: Springer, 2016: 415-423.
|
|
|
[17] |
刘哲, 张晓林, 宋余庆, 等 结合改进的U-Net和Morphsnakes的肝脏分割[J]. 中国图象图形学报, 2018, 23 (8): 1254- 1262 LIU Zhe, ZHANG Xiao-lin, SONG Yu-qing, et al Liver segmentation with improved U-Net and Morphsnakes algorithm[J]. Journal of Image and Graphics, 2018, 23 (8): 1254- 1262
|
|
|
[18] |
HE F, ZHANG G, YANG H, et al Multi-scale attention module U-Net liver tumour segmentation method[J]. Journal of Physics: Conference Series, 2020, 1678: 012107
doi: 10.1088/1742-6596/1678/1/012107
|
|
|
[19] |
OKTAY O, SCHLEMPER J, FOLGOC L, et al. Attention U-Net: learning where to look for the pancreas [EB/OL]. [2020-11-20]. https://arxiv.org/pdf/1804.03999.
|
|
|
[20] |
HU J, SHEN L, SUN G, et al. Squeeze-and-excitation networks [C]// Proceedings of the IEEE conference on Computer Vision and Pattern Recognition. Salt Lake City: IEEE, 2018: 7132-7141.
|
|
|
[21] |
ROY A, NAVAB N, WACHINGER C. Concurrent spatial and channel ‘squeeze & excitation’ in fully convolutional networks [C]// International Conference on Medical Image Computing and Computer Assisted Intervention. Granada: Springer, 2018, 421-429.
|
|
|
[22] |
潘崇煜, 黄健, 郝建国, 等 融合零样本学习和小样本学习的弱监督学习方法综述[J]. 系统工程与电子技术, 2020, 42 (10): 2246- 2256 PAN Chong-yu, HUANG Jian, HAO Jian-guo, et al Survey of weakly supervised learning integrating zero-shot and few-shot learing[J]. Systems Engineering and Electionics, 2020, 42 (10): 2246- 2256
doi: 10.3969/j.issn.1001-506X.2020.10.13
|
|
|
[23] |
NAKAYAMA Y, LI Q, KATSURAGAWA S, et al Automated hepatic volumetry for living related liver transplantation at multisection CT[J]. Radiology, 2006, 240 (3): 743- 748
doi: 10.1148/radiol.2403050850
|
|
|
[24] |
CHEPLYGINA V, DE BRUIJNE M, PLUIM J Not-so- supervised: a survey of semi-supervised, multi-instance, and transfer learning in medical image analysis[J]. Medical Image Analysis, 2019, 54: 280- 296
doi: 10.1016/j.media.2019.03.009
|
|
|
[25] |
XIE Q, LUONG M, HOVY E, et al. Self-training with noisy student improves imagenet classification [C]// Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition. [s.l.]: IEEE, 2020, 10687-10698.
|
|
|
[26] |
ZHAO F, CHEN Y, CHEN F, et al Semi-supervised cerebrovascular segmentation by hierarchical convolutional neural network[J]. IEEE Access, 2018, 6: 67841- 67852
doi: 10.1109/ACCESS.2018.2879521
|
|
|
[27] |
XIA, Y, LIU F, YANG D, et al. 3D Semi-supervised learning with uncertainty-aware multi-view co-training [C]// 2020 Workshop on Applications of Computer Vision. Snowmass Village: IEEE, 2020: 3646-3655.
|
|
|
[28] |
YANG Z, COHEN W, SALAKHUTDINOV R. Revisiting semi-supervised learning with graph embeddings [C]// International Conference on Machine Learning. New York City: JMLR, 2016: 40-48.
|
|
|
[29] |
ZHOU Y, WANG Y, TANG P, et al. Semi-supervised 3D abdominal multi-organ segmentation via deep multi-planar co-training [C]// 2019 IEEE Winter Conference on Applications of Computer Vision (WACV). Hawaii: IEEE, 2019: 121-140.
|
|
|
[30] |
JIANG B, ZHANG Z, LIN D, et al. Semi-supervised learning with graph learning-convolutional networks [C]// Proceedings of the IEEE conference on Computer Vision and Pattern Recognition. Long Beach: IEEE,, 2019: 11313-11320.
|
|
|
[31] |
姜威威, 刘祥强, 韩金仓 基于深度协同训练的肝脏CT图像自动分割方法[J]. 电子设计工程, 2020, 28 (14): 175- 179 JIANG Wei-wei, LIU Xiang-qiang, HAN Jin-cang Automatic liver segmentation from CT images based on deep co-training[J]. Electronic Design Engineering, 2020, 28 (14): 175- 179
|
|
|
[32] |
LEE D. Pseudo-label: the simple and efficient semi-supervised learning method for deep neural networks [C]// In Workshop on Challenges in Representation Learning, ICML. Atlanta: JMLR, 2013, 3: 896.
|
|
|
[33] |
BAI W, OKTAY O, SINCLAIR M, et al. Semi-supervised learning for network-based cardiac MR image segmentation [C]// International Conference on Medical Image Computing and Computer Assisted Intervention. Quebec: Springer, 2017: 253-260.
|
|
|
[34] |
KRAHENBUHL P, KOLTUN V Efficient inference in fully connected CRFs with Gaussian edge potentials[J]. Advances in Neural Information Processing Systems, 2011, 24: 109- 117
|
|
|
|
Viewed |
|
|
|
Full text
|
|
|
|
|
Abstract
|
|
|
|
|
Cited |
|
|
|
|
|
Shared |
|
|
|
|
|
Discussed |
|
|
|
|