Please wait a minute...
浙江大学学报(工学版)
浙江大学学报(工学版)  2022, Vol. 56 Issue (4): 727-735    DOI: 10.3785/j.issn.1008-973X.2022.04.012
计算机技术、信息工程     
基于Cycle-GAN和改进DPN网络的乳腺癌病理图像分类
张雪芹(),李天任
华东理工大学 信息科学与工程学院，上海 200000
Breast cancer pathological image classification based on Cycle-GAN and improved DPN network
Xue-qin ZHANG(),Tian-ren LI
School of Information Science and Engineering, East China University of Science and Technology, Shanghai 200000, China
 全文: PDF(1177 KB)   HTML
摘要：

针对病理图像染色不均匀及良恶性难以鉴别的问题，提出基于Cycle-GAN和改进的双路径网络（DPN）的算法框架. 利用Cycle-GAN进行颜色归一化处理，解决因病理图像染色不均匀导致的检测模型精度偏低问题，通过对图像进行重叠切片，基于DPN网络采用增加小卷积、反卷积和注意力机制，增强模型对病理图像纹理特征的分类能力. 在BreaKHis数据集上的实验结果表明，所提算法有效提高了乳腺癌病理图像良恶性分类的准确性.
关键词： 乳腺癌病理图像分类;  深度学习;  Cycle-GAN网络;  双路径网络（DPN）;  注意力机制    
Abstract:

An algorithm framework based on cycle-consistent adversarial network (Cycle-GAN) and improved dual path network (DPN) was proposed aiming at the uneven staining of pathological images and the difficulty in distinguishing between benign and malignant. Cycle-GAN was used for color normalization in order to solve the problem of low accuracy of the detection model caused by uneven staining of pathological images. The mechanism of adding small convolution, deconvolution and attention was adopted based on the DPN network by overlapping and slicing the image. The model’s ability to classify pathological image texture features was enhanced. The experimental results on the BreaKHis dataset show that the proposed algorithm effectively improves the accuracy of the classification of benign and malignant breast cancer pathological images.
Key words: breast cancer pathological image classification    deep learning    Cycle-GAN network    dual path network (DPN)    attention mechanism
收稿日期: 2021-05-20 出版日期: 2022-04-24
CLC:  TP 391  
作者简介: 张雪芹（1972—），女，教授，从事计算机视觉与信息安全的研究. orcid.org/0000-0001-7020-1033. E-mail： zxq@ecust.edu.com
服务　　
把本文推荐给朋友
加入引用管理器
E-mail Alert
作者相关文章　　
张雪芹
李天任

引用本文:

张雪芹,李天任. 基于Cycle-GAN和改进DPN网络的乳腺癌病理图像分类[J]. 浙江大学学报(工学版), 2022, 56(4): 727-735.

Xue-qin ZHANG,Tian-ren LI. Breast cancer pathological image classification based on Cycle-GAN and improved DPN network. Journal of ZheJiang University (Engineering Science), 2022, 56(4): 727-735.

链接本文:

https://www.zjujournals.com/eng/CN/10.3785/j.issn.1008-973X.2022.04.012    或    https://www.zjujournals.com/eng/CN/Y2022/V56/I4/727

图 1  Cycle-GAN模型
图 2  DPN68网络结构
图 3  DPN网络block结构
图 4  注意力层结构
图 5  基于Cycle-GAN和DPN网络的乳腺癌病理图像分类模型
图 6  Cycle-GAN病理图像颜色归一化模型
图 7  DPN68-A网络结构
A Nib Nim Ni
40 625 1370 1995
100 644 1437 2081
200 623 1390 2013
400 588 1232 1820
表 1  不同放大倍数的良、恶性肿瘤图像的数量
图 8  病理图像重叠切割
数据集 Nb Nm N
Data1 23856 22632 46488
Data2 16656 22248 38904
Data3 17856 20880 38736
表 2  40倍下扩充后三折数据集中良、恶性切片的具体分布
%
方法 FPR FNR Recall Precision F1-score I
无归一化 24.4 8.9 90.0 78.64 83.94 83.33
Vahadane法归一化 13.3 4.4 95.5 87.76 91.47 91.11
Cycle-GAN归一化 10.0 3.3 96.7 90.63 93.56 93.33
表 3  不同归一化方法下良恶性分类准确率对比
%
模型 FPR FNR I R
VGG16 36.48 9.78 81.85 82.68
AlexNet 31.84 11.53 82.11 84.68
GoogLeNet 30.72 10.00 83.51 85.49
ResNet34 19.50 9.60 87.27 90.89
ResNet101 22.88 8.90 86.67 89.18
表 4  不同CNN模型图像级别和患者级别准确率对比结果
网络 FPR FNR Recall Precision F1-score I R AUC
DPN68 12.30 6.7 93.28 94.04 93.66 91.33 92.76 93.36
DPN68+小卷积 10.40 6.5 93.50 95.17 94.33 92.28 93.72 93.50
DPN68-A 7.04 6.2 93.80 96.69 95.22 93.53 94.68 94.72
表 5  DPN68网络改进分类准确率的对比结果
图 9  ROC曲线图
图 10  DPN68-A与其他分类算法的患者级别准确率对比结果
A FPR/% FNR/% Recall/% Precision/% F1-score/% I/% R/%
40 7.04 6.20 93.80 96.69 95.22 93.53 94.68
100 7.05 4.28 95.72 97.60 96.65 94.70 94.51
200 8.02 4.70 95.30 96.73 96.01 93.34 94.35
400 7.33 5.06 94.94 97.15 96.03 94.31 94.72
表 6  DPN68-A在所有放大倍数下测试结果
1 CHEN W, ZHENG R, BAADE P, et al Cancer Statistics in China, 2015[J]. CA: A Cancer Journal for Clinicians, 2016, 66 (2): 115- 132
doi: 10.3322/caac.21338
2 KOWAL M, FILIPCZUK P, OBUCHOWICZ A, et al Computer-aided diagnosis of breast cancer based on fine needle biopsy microscopic images[J]. Computers in Biology and Medicine, 2013, 43 (10): 1563- 1572
doi: 10.1016/j.compbiomed.2013.08.003
3 ISABEL G, POLONIA A, SARMIENTO A, et al Automatic classification of tissue malignancy for breast carcinoma diagnosis[J]. Computers in Biology and Medicine, 2018, 96: 41- 51
doi: 10.1016/j.compbiomed.2018.03.003
4 ABDULLAH-AL N, YINAN K Histopathological breast-image classification using concatenated R-G-B histogram information[J]. Annals of Data Science, 2019, 6 (3): 513- 529
doi: 10.1007/s40745-018-0162-3
5 WANG P, HU X, LI Y, et al Automatic cell nuclei segmentation and classification of breast cancer histopathology images[J]. Signal Processing, 2016, 122: 1- 13
doi: 10.1016/j.sigpro.2015.11.011
6 SPANHOL F, OLIVEIRA L, PETITJEAN C, et al A dataset for breast cancer histopathological image classification[J]. IEEE Transactions on Biomedical Engineering, 2016, 63 (7): 1455- 1462
doi: 10.1109/TBME.2015.2496264
7 VETA M, PLUIM J, VAN D, et al Breast cancer histopathology image analysis: a rcview[J]. IEEE Transactions on Biomedical Engineering, 2014, 61 (5): 1400- 1411
doi: 10.1109/TBME.2014.2303852
8 SHEN D, WU G, SUK H Deep learning in medical image analysis[J]. Annual Review of Biomedical Engineering, 2017, 19 (1): 221- 248
doi: 10.1146/annurev-bioeng-071516-044442
9 SPANHOL F, OLIVEIRA L, PETITJEAN C, et al. Breast cancer histopathological image classification using convolutional neural networks [C]// Proceedings of 2016 International Joint Conference on Neural Networks. Vancouver: IEEE, 2016: 2560-2567.
10 BAYRAMOGLU N, KANNALA J, JANNE H. Deep learning for magnification independent breast cancer histopathology image classification [C]// Proceedings of International Conference on Pattern Recognition. Cancun: IEEE, 2016: 2441-2446.
11 何雪英, 韩忠义, 魏本征 基于深度学习的乳腺癌病理图像自动分类[J]. 计算机工程与应用, 2018, 54 (12): 121- 125
HE Xue-ying, HAN Zhong-yi, WEI Ben-zheng Breast cancer histopathological image auto-classification using deep learning[J]. Computer Engineering and Applications, 2018, 54 (12): 121- 125
doi: 10.3778/j.issn.1002-8331.1701-0392
12 明涛, 王丹, 郭继昌, 等 基于多尺度通道重校准的乳腺癌病理图像分类[J]. 浙江大学学报:工学版, 2020, 54 (7): 1289- 1297
MING Tao, WANG Dan, GUO Ji-chang, et al Breast cancer histopathological image classification using multi-scale channel squeeze-and-excitation model[J]. Journal of Zhejiang University: Engineering Science, 2020, 54 (7): 1289- 1297
13 邹文凯, 陆慧娟, 叶敏超, 等 基于卷积神经网络的乳腺癌组织病理图像分类[J]. 计算机工程与设计, 2020, 41 (6): 1749- 1754
ZOU Wen-kai, LU Hui-juan, YE Min-chao, et al Breast cancer histopathological image classification using convolutional neural network[J]. Computer Engineering and Design, 2020, 41 (6): 1749- 1754
14 NAHID A, MEHRABI M, KONG Y. Frequency-domain information along with LSTM and GRU methods for histopathological breast-image classification [C]//International Symposium on Signal Processing and Information Technology. Bilbao: IEEE, 2017: 410-415.
15 GOODFELLOW I, POUGET-ABADIE J, MIRZA M, et al. Generative adversarial networks [C]// Advances in Neural Information Processing Systems. Montreal: MIT Press, 2014: 2672-2680.
16 ZHU J, PARK T, ISOLA P, et al. Unpaired image-to-image translation using cycle-consistent adversarial networks [C]// International Conference On Computer Vision. Venice: IEEE, 2017: 2242-2251.
17 HUANG G, LIU Z, LAURENS V, et al. Densely connected convolutional networks [C]// Conference on Computer Vision and Pattern Recognition. Honolulu: IEEE, 2017: 2261-2269.
18 CHEN Y, LI J, XIAO H, et al. Dual path networks [C]// Conference on Neural Information Processing Systems. Long Beach: IEEE, 2017: 1-9.
19 JIE H, LI S, GANG S, et al Squeeze-and-excitation networks[J]. IEEE Transactions on Pattern Analysis and Machine Intelligence, 2017, 42 (99): 2011- 2023
20 SHORTEN C, KHOSHGOFTAAR T A survey on image data augmentation for deep learning[J]. Journal of Big Data, 2019, 6 (1): 1- 48
doi: 10.1186/s40537-018-0162-3
[1] 陈巧红,裴皓磊,孙麒. 基于视觉关系推理与上下文门控机制的图像描述[J]. 浙江大学学报(工学版), 2022, 56(3): 542-549.
[2] 褚晶辉,史李栋,井佩光,吕卫. 适用于目标检测的上下文感知知识蒸馏网络[J]. 浙江大学学报(工学版), 2022, 56(3): 503-509.
[3] 程若然,赵晓丽,周浩军,叶翰辰. 基于深度学习的中文字体风格转换研究综述[J]. 浙江大学学报(工学版), 2022, 56(3): 510-519, 530.
[4] 农元君,王俊杰,陈红,孙文涵,耿慧,李书悦. 基于注意力机制和编码-解码架构的施工场景图像描述方法[J]. 浙江大学学报(工学版), 2022, 56(2): 236-244.
[5] 刘英莉,吴瑞刚,么长慧,沈韬. 铝硅合金实体关系抽取数据集的构建方法[J]. 浙江大学学报(工学版), 2022, 56(2): 245-253.
[6] 董红召,方浩杰,张楠. 旋转框定位的多尺度再生物品目标检测算法[J]. 浙江大学学报(工学版), 2022, 56(1): 16-25.
[7] 王鑫,陈巧红,孙麒,贾宇波. 基于关系推理与门控机制的视觉问答方法[J]. 浙江大学学报(工学版), 2022, 56(1): 36-46.
[8] 陈彤,郭剑锋,韩心中,谢学立,席建祥. 基于生成对抗模型的可见光-红外图像匹配方法[J]. 浙江大学学报(工学版), 2022, 56(1): 63-74.
[9] 任松,朱倩雯,涂歆玥,邓超,王小书. 基于深度学习的公路隧道衬砌病害识别方法[J]. 浙江大学学报(工学版), 2022, 56(1): 92-99.
[10] 刘兴,余建波. 注意力卷积GRU自编码器及其在工业过程监控的应用[J]. 浙江大学学报(工学版), 2021, 55(9): 1643-1651.
[11] 陈雪云,黄小巧,谢丽. 基于多尺度条件生成对抗网络血细胞图像分类检测方法[J]. 浙江大学学报(工学版), 2021, 55(9): 1772-1781.
[12] 陈智超,焦海宁,杨杰,曾华福. 基于改进MobileNet v2的垃圾图像分类算法[J]. 浙江大学学报(工学版), 2021, 55(8): 1490-1499.
[13] 刘嘉诚,冀俊忠. 基于宽度学习系统的fMRI数据分类方法[J]. 浙江大学学报(工学版), 2021, 55(7): 1270-1278.
[14] 金立生,华强,郭柏苍,谢宪毅,闫福刚,武波涛. 基于优化DeepSort的前方车辆多目标跟踪[J]. 浙江大学学报(工学版), 2021, 55(6): 1056-1064.
[15] 许佳辉,王敬昌,陈岭,吴勇. 基于图神经网络的地表水水质预测模型[J]. 浙江大学学报(工学版), 2021, 55(4): 601-607.