Please wait a minute...
浙江大学学报(工学版)
Journal of ZheJiang University (Engineering Science)  2022, Vol. 56 Issue (4): 727-735    DOI: 10.3785/j.issn.1008-973X.2022.04.012
    
Breast cancer pathological image classification based on Cycle-GAN and improved DPN network
Xue-qin ZHANG(),Tian-ren LI
School of Information Science and Engineering, East China University of Science and Technology, Shanghai 200000, China
Download: HTML     PDF(1177KB) HTML
Export: BibTeX | EndNote (RIS)      

Abstract  

An algorithm framework based on cycle-consistent adversarial network (Cycle-GAN) and improved dual path network (DPN) was proposed aiming at the uneven staining of pathological images and the difficulty in distinguishing between benign and malignant. Cycle-GAN was used for color normalization in order to solve the problem of low accuracy of the detection model caused by uneven staining of pathological images. The mechanism of adding small convolution, deconvolution and attention was adopted based on the DPN network by overlapping and slicing the image. The model’s ability to classify pathological image texture features was enhanced. The experimental results on the BreaKHis dataset show that the proposed algorithm effectively improves the accuracy of the classification of benign and malignant breast cancer pathological images.

Key words： breast cancer pathological image classification      deep learning      Cycle-GAN network      dual path network (DPN)      attention mechanism     
Received: 20 May 2021      Published: 24 April 2022
CLC:  TP 391  
Service
E-mail this article
Add to my bookshelf
Add to citation manager
E-mail Alert
RSS
Articles by authors
Xue-qin ZHANG
Tian-ren LI
Cite this article:

Xue-qin ZHANG,Tian-ren LI. Breast cancer pathological image classification based on Cycle-GAN and improved DPN network. Journal of ZheJiang University (Engineering Science), 2022, 56(4): 727-735.

URL:

https://www.zjujournals.com/eng/10.3785/j.issn.1008-973X.2022.04.012     OR     https://www.zjujournals.com/eng/Y2022/V56/I4/727


基于Cycle-GAN和改进DPN网络的乳腺癌病理图像分类

针对病理图像染色不均匀及良恶性难以鉴别的问题，提出基于Cycle-GAN和改进的双路径网络（DPN）的算法框架. 利用Cycle-GAN进行颜色归一化处理，解决因病理图像染色不均匀导致的检测模型精度偏低问题，通过对图像进行重叠切片，基于DPN网络采用增加小卷积、反卷积和注意力机制，增强模型对病理图像纹理特征的分类能力. 在BreaKHis数据集上的实验结果表明，所提算法有效提高了乳腺癌病理图像良恶性分类的准确性.


关键词： 乳腺癌病理图像分类,  深度学习,  Cycle-GAN网络,  双路径网络（DPN）,  注意力机制 
Fig.1 Cycle-GAN networks structure
Fig.2 DPN68 network structure
Fig.3 Block structure in DPN network
Fig.4 Attention mechanism structure
Fig.5 Breast cancer pathological image classification model based on Cycle-GAN and DPN network
Fig.6 Cycle-GAN pathological image color normalization model
Fig.7 DPN68-A network structure
A Nib Nim Ni
40 625 1370 1995
100 644 1437 2081
200 623 1390 2013
400 588 1232 1820
Tab.1 Number of benign and malignant tumor images with different magnification
Fig.8 Overlapping cutting of pathological images
数据集 Nb Nm N
Data1 23856 22632 46488
Data2 16656 22248 38904
Data3 17856 20880 38736
Tab.2 Specific distribution of benign and malignant sections in expanded trifold dataset at 40×
%
方法 FPR FNR Recall Precision F1-score I
无归一化 24.4 8.9 90.0 78.64 83.94 83.33
Vahadane法归一化 13.3 4.4 95.5 87.76 91.47 91.11
Cycle-GAN归一化 10.0 3.3 96.7 90.63 93.56 93.33
Tab.3 Comparison of accuracy of benign and malignant classification under different normalization methods
%
模型 FPR FNR I R
VGG16 36.48 9.78 81.85 82.68
AlexNet 31.84 11.53 82.11 84.68
GoogLeNet 30.72 10.00 83.51 85.49
ResNet34 19.50 9.60 87.27 90.89
ResNet101 22.88 8.90 86.67 89.18
Tab.4 Comparison of accuracy of image level and patient level of different CNN models
网络 FPR FNR Recall Precision F1-score I R AUC
DPN68 12.30 6.7 93.28 94.04 93.66 91.33 92.76 93.36
DPN68+小卷积 10.40 6.5 93.50 95.17 94.33 92.28 93.72 93.50
DPN68-A 7.04 6.2 93.80 96.69 95.22 93.53 94.68 94.72
Tab.5 Comparison results of improved classification accuracy of DPN68 network %
Fig.9 ROC curves of networks
Fig.10 Comparison results of patient level accuracy between DPN68-A and other classification algorithms
A FPR/% FNR/% Recall/% Precision/% F1-score/% I/% R/%
40 7.04 6.20 93.80 96.69 95.22 93.53 94.68
100 7.05 4.28 95.72 97.60 96.65 94.70 94.51
200 8.02 4.70 95.30 96.73 96.01 93.34 94.35
400 7.33 5.06 94.94 97.15 96.03 94.31 94.72
Tab.6 DPN68-A results at all magnification
[1]   CHEN W, ZHENG R, BAADE P, et al Cancer Statistics in China, 2015[J]. CA: A Cancer Journal for Clinicians, 2016, 66 (2): 115- 132
doi: 10.3322/caac.21338
[2]   KOWAL M, FILIPCZUK P, OBUCHOWICZ A, et al Computer-aided diagnosis of breast cancer based on fine needle biopsy microscopic images[J]. Computers in Biology and Medicine, 2013, 43 (10): 1563- 1572
doi: 10.1016/j.compbiomed.2013.08.003
[3]   ISABEL G, POLONIA A, SARMIENTO A, et al Automatic classification of tissue malignancy for breast carcinoma diagnosis[J]. Computers in Biology and Medicine, 2018, 96: 41- 51
doi: 10.1016/j.compbiomed.2018.03.003
[4]   ABDULLAH-AL N, YINAN K Histopathological breast-image classification using concatenated R-G-B histogram information[J]. Annals of Data Science, 2019, 6 (3): 513- 529
doi: 10.1007/s40745-018-0162-3
[5]   WANG P, HU X, LI Y, et al Automatic cell nuclei segmentation and classification of breast cancer histopathology images[J]. Signal Processing, 2016, 122: 1- 13
doi: 10.1016/j.sigpro.2015.11.011
[6]   SPANHOL F, OLIVEIRA L, PETITJEAN C, et al A dataset for breast cancer histopathological image classification[J]. IEEE Transactions on Biomedical Engineering, 2016, 63 (7): 1455- 1462
doi: 10.1109/TBME.2015.2496264
[7]   VETA M, PLUIM J, VAN D, et al Breast cancer histopathology image analysis: a rcview[J]. IEEE Transactions on Biomedical Engineering, 2014, 61 (5): 1400- 1411
doi: 10.1109/TBME.2014.2303852
[8]   SHEN D, WU G, SUK H Deep learning in medical image analysis[J]. Annual Review of Biomedical Engineering, 2017, 19 (1): 221- 248
doi: 10.1146/annurev-bioeng-071516-044442
[9]   SPANHOL F, OLIVEIRA L, PETITJEAN C, et al. Breast cancer histopathological image classification using convolutional neural networks [C]// Proceedings of 2016 International Joint Conference on Neural Networks. Vancouver: IEEE, 2016: 2560-2567.
[10]   BAYRAMOGLU N, KANNALA J, JANNE H. Deep learning for magnification independent breast cancer histopathology image classification [C]// Proceedings of International Conference on Pattern Recognition. Cancun: IEEE, 2016: 2441-2446.
[11]   何雪英, 韩忠义, 魏本征 基于深度学习的乳腺癌病理图像自动分类[J]. 计算机工程与应用, 2018, 54 (12): 121- 125
HE Xue-ying, HAN Zhong-yi, WEI Ben-zheng Breast cancer histopathological image auto-classification using deep learning[J]. Computer Engineering and Applications, 2018, 54 (12): 121- 125
doi: 10.3778/j.issn.1002-8331.1701-0392
[12]   明涛, 王丹, 郭继昌, 等 基于多尺度通道重校准的乳腺癌病理图像分类[J]. 浙江大学学报:工学版, 2020, 54 (7): 1289- 1297
MING Tao, WANG Dan, GUO Ji-chang, et al Breast cancer histopathological image classification using multi-scale channel squeeze-and-excitation model[J]. Journal of Zhejiang University: Engineering Science, 2020, 54 (7): 1289- 1297
[13]   邹文凯, 陆慧娟, 叶敏超, 等 基于卷积神经网络的乳腺癌组织病理图像分类[J]. 计算机工程与设计, 2020, 41 (6): 1749- 1754
ZOU Wen-kai, LU Hui-juan, YE Min-chao, et al Breast cancer histopathological image classification using convolutional neural network[J]. Computer Engineering and Design, 2020, 41 (6): 1749- 1754
[14]   NAHID A, MEHRABI M, KONG Y. Frequency-domain information along with LSTM and GRU methods for histopathological breast-image classification [C]//International Symposium on Signal Processing and Information Technology. Bilbao: IEEE, 2017: 410-415.
[15]   GOODFELLOW I, POUGET-ABADIE J, MIRZA M, et al. Generative adversarial networks [C]// Advances in Neural Information Processing Systems. Montreal: MIT Press, 2014: 2672-2680.
[16]   ZHU J, PARK T, ISOLA P, et al. Unpaired image-to-image translation using cycle-consistent adversarial networks [C]// International Conference On Computer Vision. Venice: IEEE, 2017: 2242-2251.
[17]   HUANG G, LIU Z, LAURENS V, et al. Densely connected convolutional networks [C]// Conference on Computer Vision and Pattern Recognition. Honolulu: IEEE, 2017: 2261-2269.
[18]   CHEN Y, LI J, XIAO H, et al. Dual path networks [C]// Conference on Neural Information Processing Systems. Long Beach: IEEE, 2017: 1-9.
[19]   JIE H, LI S, GANG S, et al Squeeze-and-excitation networks[J]. IEEE Transactions on Pattern Analysis and Machine Intelligence, 2017, 42 (99): 2011- 2023
[20]   SHORTEN C, KHOSHGOFTAAR T A survey on image data augmentation for deep learning[J]. Journal of Big Data, 2019, 6 (1): 1- 48
doi: 10.1186/s40537-018-0162-3
[1] Qiao-hong CHEN,Hao-lei PEI,Qi SUN. Image caption based on relational reasoning and context gate mechanism[J]. Journal of ZheJiang University (Engineering Science), 2022, 56(3): 542-549.
[2] Jing-hui CHU,Li-dong SHI,Pei-guang JING,Wei LV. Context-aware knowledge distillation network for object detection[J]. Journal of ZheJiang University (Engineering Science), 2022, 56(3): 503-509.
[3] Ruo-ran CHENG,Xiao-li ZHAO,Hao-jun ZHOU,Han-chen YE. Review of Chinese font style transfer research based on deep learning[J]. Journal of ZheJiang University (Engineering Science), 2022, 56(3): 510-519, 530.
[4] Yuan-jun NONG,Jun-jie WANG,Hong CHEN,Wen-han SUN,Hui GENG,Shu-yue LI. A image caption method of construction scene based on attention mechanism and encoding-decoding architecture[J]. Journal of ZheJiang University (Engineering Science), 2022, 56(2): 236-244.
[5] Ying-li LIU,Rui-gang WU,Chang-hui YAO,Tao SHEN. Construction method of extraction dataset of Al-Si alloy entity relationship[J]. Journal of ZheJiang University (Engineering Science), 2022, 56(2): 245-253.
[6] Xin WANG,Qiao-hong CHEN,Qi SUN,Yu-bo JIA. Visual question answering method based on relational reasoning and gating mechanism[J]. Journal of ZheJiang University (Engineering Science), 2022, 56(1): 36-46.
[7] Tong CHEN,Jian-feng GUO,Xin-zhong HAN,Xue-li XIE,Jian-xiang XI. Visible and infrared image matching method based on generative adversarial model[J]. Journal of ZheJiang University (Engineering Science), 2022, 56(1): 63-74.
[8] Song REN,Qian-wen ZHU,Xin-yue TU,Chao DENG,Xiao-shu WANG. Lining disease identification of highway tunnel based on deep learning[J]. Journal of ZheJiang University (Engineering Science), 2022, 56(1): 92-99.
[9] Xing LIU,Jian-bo YU. Attention convolutional GRU-based autoencoder and its application in industrial process monitoring[J]. Journal of ZheJiang University (Engineering Science), 2021, 55(9): 1643-1651.
[10] Xue-yun CHEN,Xiao-qiao HUANG,Li XIE. Classification and detection method of blood cells images based on multi-scale conditional generative adversarial network[J]. Journal of ZheJiang University (Engineering Science), 2021, 55(9): 1772-1781.
[11] Zhi-chao CHEN,Hai-ning JIAO,Jie YANG,Hua-fu ZENG. Garbage image classification algorithm based on improved MobileNet v2[J]. Journal of ZheJiang University (Engineering Science), 2021, 55(8): 1490-1499.
[12] Jia-cheng LIU,Jun-zhong JI. Classification method of fMRI data based on broad learning system[J]. Journal of ZheJiang University (Engineering Science), 2021, 55(7): 1270-1278.
[13] Li-sheng JIN,Qiang HUA,Bai-cang GUO,Xian-yi XIE,Fu-gang YAN,Bo-tao WU. Multi-target tracking of vehicles based on optimized DeepSort[J]. Journal of ZheJiang University (Engineering Science), 2021, 55(6): 1056-1064.
[14] Jia-hui XU,Jing-chang WANG,Ling CHEN,Yong WU. Surface water quality prediction model based on graph neural network[J]. Journal of ZheJiang University (Engineering Science), 2021, 55(4): 601-607.
[15] Hong-li WANG,Bin GUO,Si-cong LIU,Jia-qi LIU,Yun-gang WU,Zhi-wen YU. End context-adaptative deep sensing model with edge-end collaboration[J]. Journal of ZheJiang University (Engineering Science), 2021, 55(4): 626-638.