Please wait a minute...
浙江大学学报(工学版)  2022, Vol. 56 Issue (10): 1958-1966    DOI: 10.3785/j.issn.1008-973X.2022.10.007
自动化技术、信息工程     
基于多层级特征自适应融合的图像分割算法
袁小平(),何祥,王小倩,胡杨明
中国矿业大学 信息与控制工程学院,江苏 徐州 221116
Image segmentation algorithm based on multi-level feature adaptive fusion
Xiao-ping YUAN(),Xiang HE,Xiao-qian WANG,Yang-ming HU
School of Information and Control Engineering , China University of Mining and Technology, Xuzhou 221116, China
 全文: PDF(1738 KB)   HTML
摘要:

为了解决传统算法对医学图像分割时精度较低的问题,提出基于多层级特征自适应融合的新型FR-Unet图像分割算法. 在编码器阶段,设计采样加权模块替代传统卷积层,对图像空间信息进行逐层提取和特征融合,获得相邻像素之间的相关性和不同层次的语义信息. 在解码器阶段,设计多层级自适应融合模块,通过非线性跳跃连接逐层提取图像通道信息,自适应地融合邻近连接层的上下文信息,使各层专注不同特征信息的提取. FR-Unet在模型参数量上大幅度减少,让网络在场景部署上得到更好的支持. 实验结果表明,该网络在动物细胞分割、肝脏器官分割、皮肤病变分割等众多任务中均表现突出.

关键词: 图像分割FR-Unet逐层提取采样加权模块多层级自适应融合模块    
Abstract:

A novel FR-Unet image segmentation algorithm based on multi-layer feature adaptive fusion was proposed in order to solve the problem of low accuracy when segmenting medical images by traditional algorithms. The sampling weighting module was designed to replace the traditional convolutional layer in the encoder stage. The image spatial information was extracted and fused by layer, and the correlation between neighboring pixels and the semantic information at different levels was obtained. A multi-layer adaptive fusion module was designed in the decoder stage in order to extract image channel information layer by layer through nonlinear jump connections and adaptively fuse the contextual information of neighboring connected layers. Then each layer focuses on the extraction of different feature information. FR-Unet was substantially reduced in the number of model parameters, allowing the network to be better supported in scene deployment. The experimental results show that the network excels in numerous tasks such as animal cell segmentation, liver organ segmentation and skin lesion segmentation.

Key words: image segmentation    FR-Unet    layer-by-layer extraction    sampling weighting module    multi-level adaptive fusion module
收稿日期: 2021-11-08 出版日期: 2022-10-25
CLC:  TP 391  
基金资助: 国家科技支撑计划资助项目(2013BAK06B08);江苏省研究生科研与实践创新计划资助项目(SJCX20_0812)
作者简介: 袁小平(1966—),男,教授,从事图像处理的研究. orcid.org/0000-0002-7936-0070. E-mail: 1941@cumt.edu.cn
服务  
把本文推荐给朋友
加入引用管理器
E-mail Alert
作者相关文章  
袁小平
何祥
王小倩
胡杨明

引用本文:

袁小平,何祥,王小倩,胡杨明. 基于多层级特征自适应融合的图像分割算法[J]. 浙江大学学报(工学版), 2022, 56(10): 1958-1966.

Xiao-ping YUAN,Xiang HE,Xiao-qian WANG,Yang-ming HU. Image segmentation algorithm based on multi-level feature adaptive fusion. Journal of ZheJiang University (Engineering Science), 2022, 56(10): 1958-1966.

链接本文:

https://www.zjujournals.com/eng/CN/10.3785/j.issn.1008-973X.2022.10.007        https://www.zjujournals.com/eng/CN/Y2022/V56/I10/1958

图 1  提出算法模型的总体结构图
图 2  非对称特征提取的采样加权模块结构图
图 3  FR-conv(1,1)卷积层的结构图
图 4  多层级自适应融合模块的结构图
图 5  数据增强示意图
图 6  混淆矩阵示意图
图 7  网络参数对比图
方法 IOU Dice L P tr/ms
FCN 0.7421 0.8320 0.0661 0.8751 67
Unet 0.8705 0.9207 0.0637 0.9187 74
UnetPlus 0.9858 0.9879 0.0020 0.9891 141
DenseASPP 0.9862 0.9898 0.0003 0.9999 588
DeepLabV3 0.9874 0.9898 0.0002 0.9999 626
Base(Unet) 0.8705 0.9207 0.0637 0.9187 74
Base+采样加权模块 0.9584 0.9786 0.0334 0.9467 64
Base+多层级自适应融合模块 0.9650 0.9647 0.0398 0.9374 509
本文方法 0.9898 0.9882 0.0001 0.9992 288
表 1  动物细胞分割任务对比试验和消融研究结果
图 8  数据集1在不同算法上的分割对比图
方法 IOU Dice L P tr/ms
FCN 0.7834 0.8473 0.0611 0.8679 65
Unet 0.8388 0.8667 0.0634 0.8786 76
UnetPlus 0.9041 0.9163 0.0502 0.8994 149
DenseASPP 0.9072 0.9474 0.0405 0.9267 592
DeepLabV3 0.9541 0.9513 0.0378 0.9451 631
Base(Unet) 0.8388 0.8667 0.0634 0.8786 76
Base+采样加权模块 0.9127 0.9479 0.0527 0.9114 60
Base+多层级自适应融合模块 0.9433 0.9487 0.0401 0.9421 510
本文方法 0.9437 0.9623 0.0279 0.9536 289
表 2  肝脏分割任务对比试验和消融研究结果
图 9  数据集2在不同算法上的分割对比图
图 10  不同分割算法的客观评价指标图
图 11  数据集3在不同算法上的分割对比图
1 扈晓君, 杨宝栋. 基于深度学习的医学图像分析综述[J]电子技术与软件工程, 2021(18): 137-138.
HU Xiao-jun, YANG Bao-dong. A review of medical image analysis based on deep learning [J]. Electronic Technology and Software Engineering, 2021 (18): 137-138.
2 LONG J, SHELHAMER E, DARRELL T. Fully convolutional networks for semantic segmentation [C]// Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition. Boston: IEEE, 2015: 3431-3440.
3 RONNEBERGER O, FISCHER P, BROX T. U-net: convolutional networks for biomedical image segmentation [C]// International Conference on Medical Image Computing and Computer-Assisted Intervention. Cham: Springer, 2015: 234-241.
4 张娴, 刘玉林, 高智勇 深度学习在医学影像中的应用研究进展[J]. 临床放射学杂志, 2021, 40 (10): 2041- 2044
ZHANG Xian, LIU Yu-lin, GAO Zhi-yong Research advances in the application of deep learning in medical imaging[J]. Journal of Clinical Radiology, 2021, 40 (10): 2041- 2044
5 CHEN C, QIN C, QIU H, et al Deep learning for cardiac image segmentation: a review[J]. Frontiers in Cardiovascular Medicine, 2020, 7: 25
doi: 10.3389/fcvm.2020.00025
6 李继凡, 陈硕, 章强 基于U-Net神经网络的多模态MR颈动脉血管成像的分割方法研究[J]. 中华放射学杂志, 2019, (12): 1091- 1095
LI Ji-fan, CHEN Shuo, ZHANG Qiang Study on segmentation method of multimodal MR carotid angiography based on U-Net neural network[J]. Chinese Journal of Radiology, 2019, (12): 1091- 1095
7 DUAN L, XIONG X, LIU Q, et al Field rice panicle segmentation based on deep full convolutional neural network[J]. Transactions of the Chinese Society of Agricultural Engineering, 2018, 34 (12): 202- 209
8 ZHOU Z, SIDDIQUEE M, TAJBAKHSH N, et al. Unet++: a nested Unet architecture for medical image segmentation [M]// Deep learning in medical image analysis and multimodal learning for clinical decision support. Cham: Springer, 2018: 3-11.
9 TAN L, MA W, XIA J, et al Multimodal magnetic resonance image brain tumor segmentation based on ACU-net network[J]. IEEE Access, 2021, 9: 14608- 14618
doi: 10.1109/ACCESS.2021.3052514
10 TREBING K, STANCZYK T, MEHRKANOON S SmaAt-U-net: precipitation nowcasting using a small attention-U-net architecture[J]. Pattern Recognition Letters, 2021, (145): 275- 289
11 CAO K, ZHANG X An improved Res-UNet model for tree species classification using airborne high-resolution images[J]. Remote Sensing, 2020, 12 (7): 1128- 1135
doi: 10.3390/rs12071128
12 HE N, FANG L, PLAZA A Hybrid first and second order attention U-net for building segmentation in remote sensing images[J]. Science China Information Sciences, 2020, 63 (4): 346- 362
13 赵小虎, 李晓, 叶圣, 等 基于改进U-Net网络的多尺度番茄病害分割算法[J]. 计算机工程与应用, 2022, 58 (10): 216- 223
ZHAO Xiao-hu, LI Xiao, YE Sheng, et al Multiscale tomato disease segmentation algorithm based on improved U-Net network[J]. Computer Engineering and Application, 2022, 58 (10): 216- 223
doi: 10.3778/j.issn.1002-8331.2105-0201
14 周涛, 董雅丽, 霍兵强, 等 U-Net网络医学图像分割应用综述[J]. 中国图象图形学报, 2021, 26 (9): 2058- 2077
ZHOU Tao, DONG Ya-li, HUO Bing-qiang, et al A review of U-Net network medical image segmentation applications[J]. Journal of Image and Graphics, 2021, 26 (9): 2058- 2077
15 MINAEE S, BOYKOV Y Y, PORIKLI F, et al Image segmentation using deep learning: a survey[J]. IEEE Transactions on Pattern Analysis and Machine Intelligence, 2021, 40 (7): 3523- 3542
16 冯婷婷, 葛华勇, 孙家慧 一种基于SSD与FRN相结合的密集连接行人检测算法[J]. 信息技术与网络安全, 2020, 39 (12): 56- 60
FENG Ting-ting, GE Hua-yong, SUN Jia-hui A densely connected pedestrian detection algorithm based on the combination of SSD and FRN[J]. Information Technology and Network Security, 2020, 39 (12): 56- 60
17 ARORA R, SAINI I, SOOD N. Modified UNet++ model: a deep model for automatic segmentation of lungs from chest X-ray images [C]// 2021 2nd International Conference on Secure Cyber Computing and Communications. Shanghai: IEEE, 2021: 166-169.
[1] 王万良,王铁军,陈嘉诚,尤文波. 融合多尺度和多头注意力的医疗图像分割方法[J]. 浙江大学学报(工学版), 2022, 56(9): 1796-1805.
[2] 郑洲, 张学昌, 郑四鸣, 施岳定. 基于区域增长与统一化水平集的CT肝脏图像分割[J]. 浙江大学学报(工学版), 2018, 52(12): 2382-2396.
[3] 廖苗, 赵于前, 曾业战, 黄忠朝, 张丙奎, 邹北骥. 基于支持向量机和椭圆拟合的细胞图像自动分割[J]. 浙江大学学报(工学版), 2017, 51(4): 722-728.
[4] 张建廷,张立民. 新型自适应稳健双边滤波图像分割[J]. 浙江大学学报(工学版), 2016, 50(9): 1703-1710.
[5] 胡祝华, 赵瑶池, 程杰仁, 彭金莲. 基于改进DRLSE的运动目标分割方法[J]. 浙江大学学报(工学版), 2014, 48(8): 1488-1495.
[6] 刘中, 陈伟海, 吴星明, 邹宇华, 王建华. 基于双目视觉的显著性区域检测[J]. J4, 2014, 48(2): 354-359.
[7] 李光廷, 禹卫东. 马尔可夫随机场SAR图像分割的快速实现技术[J]. J4, 2012, 46(10): 1810-1815.
[8] 吴一全,张晓杰,吴诗婳,张生伟. 基于混沌PSO或分解的二维最小误差阈值分割[J]. J4, 2011, 45(7): 1198-1205.
[9] 谢强军, 侯迪波, 黄平捷, 张光新, 周泽魁. 基于半隐差分的单参数水平集快速分割[J]. J4, 2010, 44(8): 1496-1501.
[10] 蔡晋辉, 张光新, 才辉. 基于连通掩模的重构开算子及应用[J]. J4, 2010, 44(4): 675-680.
[11] 孔丁科, 汪国昭. 用于图像分割的边界保持局部拟合模型[J]. J4, 2010, 44(12): 2236-2240.
[12] 张赞超, 夏顺仁. 全自动尿液图像识别技术[J]. J4, 2009, 43(5): 832-838.
[13] 杨彬蔚 陆系群. 抗纹理噪声干扰的纺织印染图像分割技术[J]. , 2009, 43(4): 668-672+737.
[14] 许晓燕, 夏顺仁, LIU Tian-ming, 等. 基于Hough变换的斑马鱼胚胎图像分析技术[J]. J4, 2009, 43(11): 2034-2037.
[15] 吴洪森 冯志林 周佳男 刘小明 董金祥. 基于相位场模型的提花织物图像分割算法[J]. J4, 2008, 42(3): 444-449.