Please wait a minute...
Journal of ZheJiang University (Engineering Science)  2021, Vol. 55 Issue (9): 1772-1781    DOI: 10.3785/j.issn.1008-973X.2021.09.019
    
Classification and detection method of blood cells images based on multi-scale conditional generative adversarial network
Xue-yun CHEN1(),Xiao-qiao HUANG1,Li XIE2
1. School of Electrical Engineering, Guangxi University, Nanning 530004, China
2. Department of Clinical Laboratory, The Second Affiliated Hospital of Guangxi Medical University, Nanning 530007, China
Download: HTML     PDF(1155KB) HTML
Export: BibTeX | EndNote (RIS)      

Abstract  

A conditional generative adversarial network structure was proposed based on multi-scale discriminator to generate a large number of realistic leukocyte images, aiming at the problem of low detection accuracy due to insufficient leukocyte samples and unclear details of the generated cell images. The leukocyte images were added to the training set of the classification detection network to realize the generation and classification of blood cell images. A multi-scale convolution kernel and a multi-scale pooling domain were introduced into the authenticity discriminator of the generated confrontation network, and the channel connection which also improved the discriminator's ability to distinguish micro-detail texture features and macro-geometric features. The gradient similarity loss function was introduced to improve the brightness and edge clarity of the generated cell image to enhance the authenticity of the image. Results show that the quality of cell image is improved by the addition of multi-scale discriminator and gradient similarity loss function during the image generation stage. Compared with the case of real data training, the average accuracy of cell classification and detection is improved from 90.4% to 94.7% through increasing the diversity of cell samples during the image classification and detection stage.



Key wordsdeep learning      classification and detection of blood cell images      conditional generative adversarial network      gradient similarity      multi-scale discriminator     
Received: 18 August 2020      Published: 20 October 2021
CLC:  TP 391.4  
Fund:  国家自然科学基金资助项目(62061002)
Cite this article:

Xue-yun CHEN,Xiao-qiao HUANG,Li XIE. Classification and detection method of blood cells images based on multi-scale conditional generative adversarial network. Journal of ZheJiang University (Engineering Science), 2021, 55(9): 1772-1781.

URL:

https://www.zjujournals.com/eng/10.3785/j.issn.1008-973X.2021.09.019     OR     https://www.zjujournals.com/eng/Y2021/V55/I9/1772


基于多尺度条件生成对抗网络血细胞图像分类检测方法

针对血细胞图像中白细胞样本较少和生成细胞图像细节不清晰,导致检测精度较低的问题,提出基于多尺度鉴别器的条件生成对抗网络. 该网络通过生成并添加大量逼真的白细胞图像到分类检测网络训练集的方式,实现对血细胞图像的生成和分类检测. 在现有条件生成对抗网络真假鉴别器中,引入多尺度卷积核、池化域并在通道上拼接,提升鉴别器对微观细节纹理特征和宏观几何特征的鉴别能力;引入梯度相似性损失函数,以提高生成细胞图像的亮度及边缘清晰度,提升图像的真实感. 实验证明,在图像生成阶段,增加多尺度鉴别器和梯度相似性损失函数提高了生成细胞图像的质量;在图像分类检测阶段,对比仅有真实数据训练的情况,增加细胞样本多样性使细胞分类检测的平均精度由90.4%提升至94.7%.


关键词: 深度学习,  血细胞图像分类检测,  条件生成对抗网络,  梯度相似性,  多尺度鉴别器 
Fig.1 Diagram of MCGAN network structure
Fig.2 Diagram of MCGAN generator structure
Fig.3 Diagram of multi-scale operation structure
${\lambda _2}$ I P/%
1 100 000 92.4
5 70 000 93.7
10 50 000 94.7
15 40 000 94.1
Tab.1 Comparison of different values of ${\lambda _{\rm{2}}}$
Fig.4 Comparison of cell image generated experiments.
Fig.5 Graph of loss function of each network experiment.
模型 R/dB S F
CGAN 13.63 0.56 84.13
pix2pix 18.41 0.69 54.37
pix2pix+MD 22.83 0.83 37.34
MCGAN 23.68 0.87 34.45
Tab.2 Comparison of PNSR, SSIM and FID values for different models
Fig.6 Comparison of cell classification and detection experimental results.
模型 P/% T M e
SSD 82.7 84 202 11 883 1 254
YOLOV3 86.5 84 202 10 367 824
FCRN 89.3 84 202 8 910 314
Res-Net 89.7 84 202 8 573 291
U-Net 90.4 84 202 7 903 236
U-Net+pix2pix 92.7 84 202 6 138 172
U-Net+pix2pix+MD 94.1 84 202 5 115 121
MCGAN 94.7 84 202 4 624 76
Tab.3 Comparison of detection results at different models
训练集 P1 P2 P3 P4 P5 P6
RD 90.5 91.3 88.6 87.6 90.9 88.1
RD+CGAN 91.0 91.7 90.7 88.7 91.9 89.1
RD+pix2pix 92.2 93.1 91.7 89.9 93.5 91.0
RD+pix2pix+MD 94.1 96.9 96.9 95.5 96.7 96.0
RD+MCGAN 94.7 99.1 98.9 98.9 98.9 99.0
Tab.4 Comparison of all kinds of cells detection results in U-Net under different database training           %
[1]   王亚品, 曹益平, 付光凯, 等 基于深度卷积神经网络的人体外周血白细胞显微图像分类[J]. 光电子·激光, 2019, 30 (5): 546- 555
WANG Ya-pin, CAO Yi-ping, FU Guang-kai, et al Human peripheral blood leukocyte microscopic image classification based on deep convolutional neural network[J]. Journal of Optoelectronics·Laser, 2019, 30 (5): 546- 555
[2]   REN S, HE K, GIRSHICK R, et al Faster R-CNN: towards real-time object detection with region proposal networks[J]. IEEE Transactions on Pattern Analysis and Machine Intelligence, 2017, 39 (6): 1137- 1149
doi: 10.1109/TPAMI.2016.2577031
[3]   LIU W, ANGUELOY D, ERHAN D, et al. SSD: single shot multibox detector[C]// European Conference on Computer Vision. Amsterdam: Springer, 2016: 21-37.
[4]   REDMON J, DIVVALA S, GIRSHICK R, et al. You only look once: unified, real-time object detection[C]// Proceedings of the 2016 IEEE Conference on Computer Vision and Pattern Recognition. Las Vegas: IEEE, 2016: 779-788.
[5]   REDMON J, FARHADI A. YOLO9000: better, faster, stronger[C]// Proceedings of the 2017 IEEE Conference on Computer Vision and Pattern Recognition. Washington: IEEE, 2017: 6517-6525.
[6]   REDMON J, FARHADI A. YOLOV3: an incremental improvement [EB/OL]. (2018-4-10) [2020-8-18]. https://arxiv.org/pdf/1804.02767.pdf.
[7]   徐晓涛, 孙亚东, 章军 基于YOLO框架的血细胞自动计数研究[J]. 计算机工程与应用, 2020, 56 (14): 98- 103
XU Xiao-tao, SUN Ya-dong, ZHANG Jun Automated counting of blood cells based on YOLO framework[J]. Computer Engineering and Applications, 2020, 56 (14): 98- 103
doi: 10.3778/j.issn.1002-8331.1904-0268
[8]   刘树杰. 基于卷积神经网络的红细胞检测和计数方法[D]. 广州: 华南理工大学, 2017: 40-57.
LIU Shu-jie. Red blood cell detection and counting based on convolutional neural network[D]. Guangzhou: South China University of Technology, 2017: 40-57.
[9]   HILAL T, KIM G S, KIL T C, et al Vehicle detection and counting in high-resolution aerial images using convolutional regression neural network[J]. IEEE Access, 2017, 11 (6): 2220- 2230
[10]   CHEN X, LIN J, XIANG S, et al Detecting maneuvering target accurately based on a two-phase approach from remote sensing imagery[J]. IEEE Geoscience and Remote Sensing Letters, 2020, 17 (5): 849- 853
doi: 10.1109/LGRS.2019.2935230
[11]   GOODFELLOW I J, POUGET A J, MIRZA M, et al Generative adversarial networks[J]. Advances in Neural Information Processing Systems, 2014, 3: 2672- 2680
[12]   CUI Y R, LIU Q, GAO C Y, et al FashionGAN: display your fashion design using conditional generative adversarial nets[J]. Computer Graphics Forum, 2018, 37 (7): 345- 359
[13]   ISOLA P, ZHU J Y, ZHOU T, et al. Image-to-image translation with conditional adversarial networks[C]// Proceedings of the IEEE conference on computer vision and pattern recognition. Hawaii: IEEE, 2017: 1125-1134.
[14]   RONNEBRGER O, FISCHER P, BROX T. U-Net: convolutional networks for biomedical image segmentation[C]// International Conference on Medical Image Computing and Computer-Assisted Intervention. Munich: Springer, 2015: 234-241.
[15]   PRATT H, WILLIAMS B, COENEN F, et al. FCNN: Fourier convolutional neural networks[C]// Joint European Conference on Machine Learning and Knowledge Discovery in Databases. Berlin: Springer, 2017: 786-798.
[16]   赖小波, 许茂盛, 徐小媚 多分类CNN的胶质母细胞瘤多模态MR图像分割[J]. 电子学报, 2019, 47 (8): 1738- 1747
LAI Xiao-bo, XU Mao-sheng, XU Xiao-mei Glioblastoma multiforme multi-modal MR image segmentation using multi-class CNN[J]. Acta Electronica Sinica, 2019, 47 (8): 1738- 1747
doi: 10.3969/j.issn.0372-2112.2019.08.018
[17]   段然, 周登文, 赵丽娟, 等 基于多尺度特征映射网络的图像超分辨率重建[J]. 浙江大学学报: 工学版, 2019, 53 (7): 1331- 1339
DUAN Ran, ZHOU Deng-wen, ZHAO Li-juan, et al Image super-resolution reconstruction based on multi-scale feature mapping network[J]. Journal of Zhejiang University: Engineering Science, 2019, 53 (7): 1331- 1339
[18]   王成凯, 杨晓敏, 严斌宇 基于随机森林的红外图像超分辨力算法[J]. 太赫兹科学与电子信息学报, 2020, 18 (4): 665- 671
WANG Cheng-kai, YANG Xiao-min, YAN Bin-yu Infrared image super-resolution algorithm based on random forest[J]. Journal of Terahertz Science and Electronic Information Technology, 2020, 18 (4): 665- 671
doi: 10.11805/TKYDA2019139
[19]   胡麟苗, 张湧 基于生成对抗网络的短波红外−可见光人脸图像翻译[J]. 光学学报, 2020, 40 (5): 75- 84
HU Lin-miao, ZHANG Yong Facial image translation in short-wavelength infrared and visible light based on generative adversarial network[J]. Acta Optica Sinica, 2020, 40 (5): 75- 84
[1] Wen-chao BAI,Xi-xian HAN,Jin-bao WANG. Efficient approximate query processing framework based on conditional generative model[J]. Journal of ZheJiang University (Engineering Science), 2022, 56(5): 995-1005.
[2] Li HE,Shan-min PANG. Face reconstruction from voice based on age-supervised learning and face prior information[J]. Journal of ZheJiang University (Engineering Science), 2022, 56(5): 1006-1016.
[3] Xue-qin ZHANG,Tian-ren LI. Breast cancer pathological image classification based on Cycle-GAN and improved DPN network[J]. Journal of ZheJiang University (Engineering Science), 2022, 56(4): 727-735.
[4] Jing-hui CHU,Li-dong SHI,Pei-guang JING,Wei LV. Context-aware knowledge distillation network for object detection[J]. Journal of ZheJiang University (Engineering Science), 2022, 56(3): 503-509.
[5] Ruo-ran CHENG,Xiao-li ZHAO,Hao-jun ZHOU,Han-chen YE. Review of Chinese font style transfer research based on deep learning[J]. Journal of ZheJiang University (Engineering Science), 2022, 56(3): 510-519, 530.
[6] Tong CHEN,Jian-feng GUO,Xin-zhong HAN,Xue-li XIE,Jian-xiang XI. Visible and infrared image matching method based on generative adversarial model[J]. Journal of ZheJiang University (Engineering Science), 2022, 56(1): 63-74.
[7] Song REN,Qian-wen ZHU,Xin-yue TU,Chao DENG,Xiao-shu WANG. Lining disease identification of highway tunnel based on deep learning[J]. Journal of ZheJiang University (Engineering Science), 2022, 56(1): 92-99.
[8] Xing LIU,Jian-bo YU. Attention convolutional GRU-based autoencoder and its application in industrial process monitoring[J]. Journal of ZheJiang University (Engineering Science), 2021, 55(9): 1643-1651.
[9] Jia-cheng LIU,Jun-zhong JI. Classification method of fMRI data based on broad learning system[J]. Journal of ZheJiang University (Engineering Science), 2021, 55(7): 1270-1278.
[10] Li-sheng JIN,Qiang HUA,Bai-cang GUO,Xian-yi XIE,Fu-gang YAN,Bo-tao WU. Multi-target tracking of vehicles based on optimized DeepSort[J]. Journal of ZheJiang University (Engineering Science), 2021, 55(6): 1056-1064.
[11] Jia-hui XU,Jing-chang WANG,Ling CHEN,Yong WU. Surface water quality prediction model based on graph neural network[J]. Journal of ZheJiang University (Engineering Science), 2021, 55(4): 601-607.
[12] Hong-li WANG,Bin GUO,Si-cong LIU,Jia-qi LIU,Yun-gang WU,Zhi-wen YU. End context-adaptative deep sensing model with edge-end collaboration[J]. Journal of ZheJiang University (Engineering Science), 2021, 55(4): 626-638.
[13] Teng ZHANG,Xin-long JIANG,Yi-qiang CHEN,Qian CHEN,Tao-mian MI,Piu CHAN. Wrist attitude-based Parkinson's disease ON/OFF state assessment after medication[J]. Journal of ZheJiang University (Engineering Science), 2021, 55(4): 639-647.
[14] Li-feng XU,Hai-fan HUANG,Wei-long DING,Yu-lei FAN. Detection of small fruit target based on improved DenseNet[J]. Journal of ZheJiang University (Engineering Science), 2021, 55(2): 377-385.
[15] Han-juan CHEN,Fei-peng DA,Shao-yan GAI. Deep 3D point cloud classification network based on competitive attention fusion[J]. Journal of ZheJiang University (Engineering Science), 2021, 55(12): 2342-2351.