Please wait a minute...
浙江大学学报(工学版)  2021, Vol. 55 Issue (9): 1772-1781    DOI: 10.3785/j.issn.1008-973X.2021.09.019
计算机与信息工程     
基于多尺度条件生成对抗网络血细胞图像分类检测方法
陈雪云1(),黄小巧1,谢丽2
1. 广西大学 电气工程学院,广西 南宁 530004
2. 广西医科大学第二附属医院 医学检验科,广西 南宁 530007
Classification and detection method of blood cells images based on multi-scale conditional generative adversarial network
Xue-yun CHEN1(),Xiao-qiao HUANG1,Li XIE2
1. School of Electrical Engineering, Guangxi University, Nanning 530004, China
2. Department of Clinical Laboratory, The Second Affiliated Hospital of Guangxi Medical University, Nanning 530007, China
 全文: PDF(1155 KB)   HTML
摘要:

针对血细胞图像中白细胞样本较少和生成细胞图像细节不清晰,导致检测精度较低的问题,提出基于多尺度鉴别器的条件生成对抗网络. 该网络通过生成并添加大量逼真的白细胞图像到分类检测网络训练集的方式,实现对血细胞图像的生成和分类检测. 在现有条件生成对抗网络真假鉴别器中,引入多尺度卷积核、池化域并在通道上拼接,提升鉴别器对微观细节纹理特征和宏观几何特征的鉴别能力;引入梯度相似性损失函数,以提高生成细胞图像的亮度及边缘清晰度,提升图像的真实感. 实验证明,在图像生成阶段,增加多尺度鉴别器和梯度相似性损失函数提高了生成细胞图像的质量;在图像分类检测阶段,对比仅有真实数据训练的情况,增加细胞样本多样性使细胞分类检测的平均精度由90.4%提升至94.7%.

关键词: 深度学习血细胞图像分类检测条件生成对抗网络梯度相似性多尺度鉴别器    
Abstract:

A conditional generative adversarial network structure was proposed based on multi-scale discriminator to generate a large number of realistic leukocyte images, aiming at the problem of low detection accuracy due to insufficient leukocyte samples and unclear details of the generated cell images. The leukocyte images were added to the training set of the classification detection network to realize the generation and classification of blood cell images. A multi-scale convolution kernel and a multi-scale pooling domain were introduced into the authenticity discriminator of the generated confrontation network, and the channel connection which also improved the discriminator's ability to distinguish micro-detail texture features and macro-geometric features. The gradient similarity loss function was introduced to improve the brightness and edge clarity of the generated cell image to enhance the authenticity of the image. Results show that the quality of cell image is improved by the addition of multi-scale discriminator and gradient similarity loss function during the image generation stage. Compared with the case of real data training, the average accuracy of cell classification and detection is improved from 90.4% to 94.7% through increasing the diversity of cell samples during the image classification and detection stage.

Key words: deep learning    classification and detection of blood cell images    conditional generative adversarial network    gradient similarity    multi-scale discriminator
收稿日期: 2020-08-18 出版日期: 2021-10-20
CLC:  TP 391.4  
基金资助: 国家自然科学基金资助项目(62061002)
作者简介: 陈雪云(1969—),男,副教授,博士,从事机器学习与模式识别方面的研究. orcid.org/0000-0001-5276-1707.E-mail: cxy17777@163.com
服务  
把本文推荐给朋友
加入引用管理器
E-mail Alert
作者相关文章  
陈雪云
黄小巧
谢丽

引用本文:

陈雪云,黄小巧,谢丽. 基于多尺度条件生成对抗网络血细胞图像分类检测方法[J]. 浙江大学学报(工学版), 2021, 55(9): 1772-1781.

Xue-yun CHEN,Xiao-qiao HUANG,Li XIE. Classification and detection method of blood cells images based on multi-scale conditional generative adversarial network. Journal of ZheJiang University (Engineering Science), 2021, 55(9): 1772-1781.

链接本文:

https://www.zjujournals.com/eng/CN/10.3785/j.issn.1008-973X.2021.09.019        https://www.zjujournals.com/eng/CN/Y2021/V55/I9/1772

图 1  MCGAN网络结构图
图 2  MCGAN生成器结构图
图 3  多尺度操作结构图
${\lambda _2}$ I P/%
1 100 000 92.4
5 70 000 93.7
10 50 000 94.7
15 40 000 94.1
表 1  不同 ${\lambda _{\rm{2}}}$实验结果对比
图 4  细胞图像生成实验对比图
图 5  网络实验损失函数曲线图
模型 R/dB S F
CGAN 13.63 0.56 84.13
pix2pix 18.41 0.69 54.37
pix2pix+MD 22.83 0.83 37.34
MCGAN 23.68 0.87 34.45
表 2  不同模型的PNSR、SSIM及FID值对比
图 6  细胞分类检测实验结果对比图
模型 P/% T M e
SSD 82.7 84 202 11 883 1 254
YOLOV3 86.5 84 202 10 367 824
FCRN 89.3 84 202 8 910 314
Res-Net 89.7 84 202 8 573 291
U-Net 90.4 84 202 7 903 236
U-Net+pix2pix 92.7 84 202 6 138 172
U-Net+pix2pix+MD 94.1 84 202 5 115 121
MCGAN 94.7 84 202 4 624 76
表 3  不同模型的检测结果对比
训练集 P1 P2 P3 P4 P5 P6
RD 90.5 91.3 88.6 87.6 90.9 88.1
RD+CGAN 91.0 91.7 90.7 88.7 91.9 89.1
RD+pix2pix 92.2 93.1 91.7 89.9 93.5 91.0
RD+pix2pix+MD 94.1 96.9 96.9 95.5 96.7 96.0
RD+MCGAN 94.7 99.1 98.9 98.9 98.9 99.0
表 4  U-Net网络中各类细胞在不同数据库训练下测试结果对比
1 王亚品, 曹益平, 付光凯, 等 基于深度卷积神经网络的人体外周血白细胞显微图像分类[J]. 光电子·激光, 2019, 30 (5): 546- 555
WANG Ya-pin, CAO Yi-ping, FU Guang-kai, et al Human peripheral blood leukocyte microscopic image classification based on deep convolutional neural network[J]. Journal of Optoelectronics·Laser, 2019, 30 (5): 546- 555
2 REN S, HE K, GIRSHICK R, et al Faster R-CNN: towards real-time object detection with region proposal networks[J]. IEEE Transactions on Pattern Analysis and Machine Intelligence, 2017, 39 (6): 1137- 1149
doi: 10.1109/TPAMI.2016.2577031
3 LIU W, ANGUELOY D, ERHAN D, et al. SSD: single shot multibox detector[C]// European Conference on Computer Vision. Amsterdam: Springer, 2016: 21-37.
4 REDMON J, DIVVALA S, GIRSHICK R, et al. You only look once: unified, real-time object detection[C]// Proceedings of the 2016 IEEE Conference on Computer Vision and Pattern Recognition. Las Vegas: IEEE, 2016: 779-788.
5 REDMON J, FARHADI A. YOLO9000: better, faster, stronger[C]// Proceedings of the 2017 IEEE Conference on Computer Vision and Pattern Recognition. Washington: IEEE, 2017: 6517-6525.
6 REDMON J, FARHADI A. YOLOV3: an incremental improvement [EB/OL]. (2018-4-10) [2020-8-18]. https://arxiv.org/pdf/1804.02767.pdf.
7 徐晓涛, 孙亚东, 章军 基于YOLO框架的血细胞自动计数研究[J]. 计算机工程与应用, 2020, 56 (14): 98- 103
XU Xiao-tao, SUN Ya-dong, ZHANG Jun Automated counting of blood cells based on YOLO framework[J]. Computer Engineering and Applications, 2020, 56 (14): 98- 103
doi: 10.3778/j.issn.1002-8331.1904-0268
8 刘树杰. 基于卷积神经网络的红细胞检测和计数方法[D]. 广州: 华南理工大学, 2017: 40-57.
LIU Shu-jie. Red blood cell detection and counting based on convolutional neural network[D]. Guangzhou: South China University of Technology, 2017: 40-57.
9 HILAL T, KIM G S, KIL T C, et al Vehicle detection and counting in high-resolution aerial images using convolutional regression neural network[J]. IEEE Access, 2017, 11 (6): 2220- 2230
10 CHEN X, LIN J, XIANG S, et al Detecting maneuvering target accurately based on a two-phase approach from remote sensing imagery[J]. IEEE Geoscience and Remote Sensing Letters, 2020, 17 (5): 849- 853
doi: 10.1109/LGRS.2019.2935230
11 GOODFELLOW I J, POUGET A J, MIRZA M, et al Generative adversarial networks[J]. Advances in Neural Information Processing Systems, 2014, 3: 2672- 2680
12 CUI Y R, LIU Q, GAO C Y, et al FashionGAN: display your fashion design using conditional generative adversarial nets[J]. Computer Graphics Forum, 2018, 37 (7): 345- 359
13 ISOLA P, ZHU J Y, ZHOU T, et al. Image-to-image translation with conditional adversarial networks[C]// Proceedings of the IEEE conference on computer vision and pattern recognition. Hawaii: IEEE, 2017: 1125-1134.
14 RONNEBRGER O, FISCHER P, BROX T. U-Net: convolutional networks for biomedical image segmentation[C]// International Conference on Medical Image Computing and Computer-Assisted Intervention. Munich: Springer, 2015: 234-241.
15 PRATT H, WILLIAMS B, COENEN F, et al. FCNN: Fourier convolutional neural networks[C]// Joint European Conference on Machine Learning and Knowledge Discovery in Databases. Berlin: Springer, 2017: 786-798.
16 赖小波, 许茂盛, 徐小媚 多分类CNN的胶质母细胞瘤多模态MR图像分割[J]. 电子学报, 2019, 47 (8): 1738- 1747
LAI Xiao-bo, XU Mao-sheng, XU Xiao-mei Glioblastoma multiforme multi-modal MR image segmentation using multi-class CNN[J]. Acta Electronica Sinica, 2019, 47 (8): 1738- 1747
doi: 10.3969/j.issn.0372-2112.2019.08.018
17 段然, 周登文, 赵丽娟, 等 基于多尺度特征映射网络的图像超分辨率重建[J]. 浙江大学学报: 工学版, 2019, 53 (7): 1331- 1339
DUAN Ran, ZHOU Deng-wen, ZHAO Li-juan, et al Image super-resolution reconstruction based on multi-scale feature mapping network[J]. Journal of Zhejiang University: Engineering Science, 2019, 53 (7): 1331- 1339
18 王成凯, 杨晓敏, 严斌宇 基于随机森林的红外图像超分辨力算法[J]. 太赫兹科学与电子信息学报, 2020, 18 (4): 665- 671
WANG Cheng-kai, YANG Xiao-min, YAN Bin-yu Infrared image super-resolution algorithm based on random forest[J]. Journal of Terahertz Science and Electronic Information Technology, 2020, 18 (4): 665- 671
doi: 10.11805/TKYDA2019139
19 胡麟苗, 张湧 基于生成对抗网络的短波红外−可见光人脸图像翻译[J]. 光学学报, 2020, 40 (5): 75- 84
HU Lin-miao, ZHANG Yong Facial image translation in short-wavelength infrared and visible light based on generative adversarial network[J]. Acta Optica Sinica, 2020, 40 (5): 75- 84
[1] 白文超,韩希先,王金宝. 基于条件生成模型的高效近似查询处理框架[J]. 浙江大学学报(工学版), 2022, 56(5): 995-1005.
[2] 何立,庞善民. 结合年龄监督和人脸先验的语音-人脸图像重建[J]. 浙江大学学报(工学版), 2022, 56(5): 1006-1016.
[3] 张雪芹,李天任. 基于Cycle-GAN和改进DPN网络的乳腺癌病理图像分类[J]. 浙江大学学报(工学版), 2022, 56(4): 727-735.
[4] 褚晶辉,史李栋,井佩光,吕卫. 适用于目标检测的上下文感知知识蒸馏网络[J]. 浙江大学学报(工学版), 2022, 56(3): 503-509.
[5] 程若然,赵晓丽,周浩军,叶翰辰. 基于深度学习的中文字体风格转换研究综述[J]. 浙江大学学报(工学版), 2022, 56(3): 510-519, 530.
[6] 陈彤,郭剑锋,韩心中,谢学立,席建祥. 基于生成对抗模型的可见光-红外图像匹配方法[J]. 浙江大学学报(工学版), 2022, 56(1): 63-74.
[7] 任松,朱倩雯,涂歆玥,邓超,王小书. 基于深度学习的公路隧道衬砌病害识别方法[J]. 浙江大学学报(工学版), 2022, 56(1): 92-99.
[8] 刘兴,余建波. 注意力卷积GRU自编码器及其在工业过程监控的应用[J]. 浙江大学学报(工学版), 2021, 55(9): 1643-1651.
[9] 刘嘉诚,冀俊忠. 基于宽度学习系统的fMRI数据分类方法[J]. 浙江大学学报(工学版), 2021, 55(7): 1270-1278.
[10] 金立生,华强,郭柏苍,谢宪毅,闫福刚,武波涛. 基于优化DeepSort的前方车辆多目标跟踪[J]. 浙江大学学报(工学版), 2021, 55(6): 1056-1064.
[11] 许佳辉,王敬昌,陈岭,吴勇. 基于图神经网络的地表水水质预测模型[J]. 浙江大学学报(工学版), 2021, 55(4): 601-607.
[12] 王虹力,郭斌,刘思聪,刘佳琪,仵允港,於志文. 边端融合的终端情境自适应深度感知模型[J]. 浙江大学学报(工学版), 2021, 55(4): 626-638.
[13] 张腾,蒋鑫龙,陈益强,陈前,米涛免,陈彪. 基于腕部姿态的帕金森病用药后开-关期检测[J]. 浙江大学学报(工学版), 2021, 55(4): 639-647.
[14] 徐利锋,黄海帆,丁维龙,范玉雷. 基于改进DenseNet的水果小目标检测[J]. 浙江大学学报(工学版), 2021, 55(2): 377-385.
[15] 陈涵娟,达飞鹏,盖绍彦. 基于竞争注意力融合的深度三维点云分类网络[J]. 浙江大学学报(工学版), 2021, 55(12): 2342-2351.