Classification and detection method of blood cells images based on multi-scale conditional generative adversarial network
Xue-yun CHEN1(),Xiao-qiao HUANG1,Li XIE2
1. School of Electrical Engineering, Guangxi University, Nanning 530004, China 2. Department of Clinical Laboratory, The Second Affiliated Hospital of Guangxi Medical University, Nanning 530007, China
A conditional generative adversarial network structure was proposed based on multi-scale discriminator to generate a large number of realistic leukocyte images, aiming at the problem of low detection accuracy due to insufficient leukocyte samples and unclear details of the generated cell images. The leukocyte images were added to the training set of the classification detection network to realize the generation and classification of blood cell images. A multi-scale convolution kernel and a multi-scale pooling domain were introduced into the authenticity discriminator of the generated confrontation network, and the channel connection which also improved the discriminator's ability to distinguish micro-detail texture features and macro-geometric features. The gradient similarity loss function was introduced to improve the brightness and edge clarity of the generated cell image to enhance the authenticity of the image. Results show that the quality of cell image is improved by the addition of multi-scale discriminator and gradient similarity loss function during the image generation stage. Compared with the case of real data training, the average accuracy of cell classification and detection is improved from 90.4% to 94.7% through increasing the diversity of cell samples during the image classification and detection stage.
Xue-yun CHEN,Xiao-qiao HUANG,Li XIE. Classification and detection method of blood cells images based on multi-scale conditional generative adversarial network. Journal of ZheJiang University (Engineering Science), 2021, 55(9): 1772-1781.
Tab.1Comparison of different values of ${\lambda _{\rm{2}}}$
Fig.4Comparison of cell image generated experiments.
Fig.5Graph of loss function of each network experiment.
模型
R/dB
S
F
CGAN
13.63
0.56
84.13
pix2pix
18.41
0.69
54.37
pix2pix+MD
22.83
0.83
37.34
MCGAN
23.68
0.87
34.45
Tab.2Comparison of PNSR, SSIM and FID values for different models
Fig.6Comparison of cell classification and detection experimental results.
模型
P/%
T
M
e
SSD
82.7
84 202
11 883
1 254
YOLOV3
86.5
84 202
10 367
824
FCRN
89.3
84 202
8 910
314
Res-Net
89.7
84 202
8 573
291
U-Net
90.4
84 202
7 903
236
U-Net+pix2pix
92.7
84 202
6 138
172
U-Net+pix2pix+MD
94.1
84 202
5 115
121
MCGAN
94.7
84 202
4 624
76
Tab.3Comparison of detection results at different models
训练集
P1
P2
P3
P4
P5
P6
RD
90.5
91.3
88.6
87.6
90.9
88.1
RD+CGAN
91.0
91.7
90.7
88.7
91.9
89.1
RD+pix2pix
92.2
93.1
91.7
89.9
93.5
91.0
RD+pix2pix+MD
94.1
96.9
96.9
95.5
96.7
96.0
RD+MCGAN
94.7
99.1
98.9
98.9
98.9
99.0
Tab.4Comparison of all kinds of cells detection results in U-Net under different database training %
[1]
王亚品, 曹益平, 付光凯, 等 基于深度卷积神经网络的人体外周血白细胞显微图像分类[J]. 光电子·激光, 2019, 30 (5): 546- 555 WANG Ya-pin, CAO Yi-ping, FU Guang-kai, et al Human peripheral blood leukocyte microscopic image classification based on deep convolutional neural network[J]. Journal of Optoelectronics·Laser, 2019, 30 (5): 546- 555
[2]
REN S, HE K, GIRSHICK R, et al Faster R-CNN: towards real-time object detection with region proposal networks[J]. IEEE Transactions on Pattern Analysis and Machine Intelligence, 2017, 39 (6): 1137- 1149
doi: 10.1109/TPAMI.2016.2577031
[3]
LIU W, ANGUELOY D, ERHAN D, et al. SSD: single shot multibox detector[C]// European Conference on Computer Vision. Amsterdam: Springer, 2016: 21-37.
[4]
REDMON J, DIVVALA S, GIRSHICK R, et al. You only look once: unified, real-time object detection[C]// Proceedings of the 2016 IEEE Conference on Computer Vision and Pattern Recognition. Las Vegas: IEEE, 2016: 779-788.
[5]
REDMON J, FARHADI A. YOLO9000: better, faster, stronger[C]// Proceedings of the 2017 IEEE Conference on Computer Vision and Pattern Recognition. Washington: IEEE, 2017: 6517-6525.
[6]
REDMON J, FARHADI A. YOLOV3: an incremental improvement [EB/OL]. (2018-4-10) [2020-8-18]. https://arxiv.org/pdf/1804.02767.pdf.
[7]
徐晓涛, 孙亚东, 章军 基于YOLO框架的血细胞自动计数研究[J]. 计算机工程与应用, 2020, 56 (14): 98- 103 XU Xiao-tao, SUN Ya-dong, ZHANG Jun Automated counting of blood cells based on YOLO framework[J]. Computer Engineering and Applications, 2020, 56 (14): 98- 103
doi: 10.3778/j.issn.1002-8331.1904-0268
[8]
刘树杰. 基于卷积神经网络的红细胞检测和计数方法[D]. 广州: 华南理工大学, 2017: 40-57. LIU Shu-jie. Red blood cell detection and counting based on convolutional neural network[D]. Guangzhou: South China University of Technology, 2017: 40-57.
[9]
HILAL T, KIM G S, KIL T C, et al Vehicle detection and counting in high-resolution aerial images using convolutional regression neural network[J]. IEEE Access, 2017, 11 (6): 2220- 2230
[10]
CHEN X, LIN J, XIANG S, et al Detecting maneuvering target accurately based on a two-phase approach from remote sensing imagery[J]. IEEE Geoscience and Remote Sensing Letters, 2020, 17 (5): 849- 853
doi: 10.1109/LGRS.2019.2935230
[11]
GOODFELLOW I J, POUGET A J, MIRZA M, et al Generative adversarial networks[J]. Advances in Neural Information Processing Systems, 2014, 3: 2672- 2680
[12]
CUI Y R, LIU Q, GAO C Y, et al FashionGAN: display your fashion design using conditional generative adversarial nets[J]. Computer Graphics Forum, 2018, 37 (7): 345- 359
[13]
ISOLA P, ZHU J Y, ZHOU T, et al. Image-to-image translation with conditional adversarial networks[C]// Proceedings of the IEEE conference on computer vision and pattern recognition. Hawaii: IEEE, 2017: 1125-1134.
[14]
RONNEBRGER O, FISCHER P, BROX T. U-Net: convolutional networks for biomedical image segmentation[C]// International Conference on Medical Image Computing and Computer-Assisted Intervention. Munich: Springer, 2015: 234-241.
[15]
PRATT H, WILLIAMS B, COENEN F, et al. FCNN: Fourier convolutional neural networks[C]// Joint European Conference on Machine Learning and KnowledgeDiscovery in Databases. Berlin: Springer, 2017: 786-798.
段然, 周登文, 赵丽娟, 等 基于多尺度特征映射网络的图像超分辨率重建[J]. 浙江大学学报: 工学版, 2019, 53 (7): 1331- 1339 DUAN Ran, ZHOU Deng-wen, ZHAO Li-juan, et al Image super-resolution reconstruction based on multi-scale feature mapping network[J]. Journal of Zhejiang University: Engineering Science, 2019, 53 (7): 1331- 1339
[18]
王成凯, 杨晓敏, 严斌宇 基于随机森林的红外图像超分辨力算法[J]. 太赫兹科学与电子信息学报, 2020, 18 (4): 665- 671 WANG Cheng-kai, YANG Xiao-min, YAN Bin-yu Infrared image super-resolution algorithm based on random forest[J]. Journal of Terahertz Science and Electronic Information Technology, 2020, 18 (4): 665- 671
doi: 10.11805/TKYDA2019139
[19]
胡麟苗, 张湧 基于生成对抗网络的短波红外−可见光人脸图像翻译[J]. 光学学报, 2020, 40 (5): 75- 84 HU Lin-miao, ZHANG Yong Facial image translation in short-wavelength infrared and visible light based on generative adversarial network[J]. Acta Optica Sinica, 2020, 40 (5): 75- 84
