Please wait a minute...
浙江大学学报(工学版)  2019, Vol. 53 Issue (9): 1728-1740    DOI: 10.3785/j.issn.1008-973X.2019.09.012
计算机科学与人工智能     
采用卷积自编码器网络的图像增强算法
王万良(),杨小涵,赵燕伟,高楠,吕闯,张兆娟
浙江工业大学 计算机科学与技术学院,机械工程学院,浙江 杭州 310023
Image enhancement algorithm with convolutional auto-encoder network
Wan-liang WANG(),Xiao-han YANG,Yan-wei ZHAO,Nan GAO,Chuang LV,Zhao-juan ZHANG
School of Computer Science and Technology, College of Mechanical Engineering, Zhejiang University of Technology, Hangzhou 310023, China
 全文: PDF(2256 KB)   HTML
摘要:

将图像增强方法低光网络(LLNet)应用于实际场景下的彩色图像时会产生大量冗余参数,为此基于LLNet提出卷积自编码器网络(CAENet)的图像增强方法. 将LLNet方法中的低光处理模块与网络训练衔接在一起;采用卷积网络代替传统自编码器的编码和解码方式. 实验结果表明:CAENet能够有效节约时间成本,减少网络参数,使网络训练更加高效,得到更好的图像低维表示. 在Corel5k数据集上的实验效果表明,CAENet在减少网络参数的同时,能有效提高图像光感和色感;在高分辨率数据集上的实验结果表明,针对图像细节方面,CAENet能够保留细节不失真;针对含噪低光图像,CAENet能在增强图像的同时达到去噪的效果,证明CAENet具有较强的鲁棒性.

关键词: 图像处理图像增强深度学习卷积神经网络降噪自编码器    
Abstract:

When the image enhancement method LLNet (the low-light net) was applied to three-channel images, there're a lot of redundant parameters. To solve this problem, a framework called CAENet (convolutional auto-encoder network) was proposed. Firstly, CAENet combined a low light processing module with a network training module. Secondly, in the encoding and decoding stages, CAENet used a convolutional network to replace the traditional fully connected network. The experimental results show that connecting low-light processing modules with network training can effectively save time costs. At the same time, the use of convolutional networks can reduce network parameters, making network training more efficient, and obtain better low-dimensional representation of images. The experimental results on the Corel5k dataset show that CAENet can effectively improve the image light perception and color perception while reducing network parameters. The experimental results on high-resolution datasets show that CAENet can preserve details for image details without distortion. In addition, for the noisy low-light image, CAENet can enhance the image while achieving the denoising effect, which proves that CAENet has strong robustness.

Key words: image processing    image enhancement    deep learning    convolutional neural network    denoising auto-encoder
收稿日期: 2019-01-24 出版日期: 2019-09-12
CLC:  TP 311.1  
作者简介: 王万良(1957—),男,教授,从事人工智能、机器自动化、网络控制研究. orcid.org/0000-0002-1552-5075. E-mail: zjutwwl@zjut.edu.cn
服务  
把本文推荐给朋友
加入引用管理器
E-mail Alert
作者相关文章  
王万良
杨小涵
赵燕伟
高楠
吕闯
张兆娟

引用本文:

王万良,杨小涵,赵燕伟,高楠,吕闯,张兆娟. 采用卷积自编码器网络的图像增强算法[J]. 浙江大学学报(工学版), 2019, 53(9): 1728-1740.

Wan-liang WANG,Xiao-han YANG,Yan-wei ZHAO,Nan GAO,Chuang LV,Zhao-juan ZHANG. Image enhancement algorithm with convolutional auto-encoder network. Journal of ZheJiang University (Engineering Science), 2019, 53(9): 1728-1740.

链接本文:

http://www.zjujournals.com/eng/CN/10.3785/j.issn.1008-973X.2019.09.012        http://www.zjujournals.com/eng/CN/Y2019/V53/I9/1728

图 1  低光网络(LLNet)结构图
图 2  LLNet模型结构示意图
图 3  采用卷积自编码器的图像增强网络(CAENet)结构示意图
图 4  流形学习方法下的图像增强原理图
图 5  LLNet低光处理阶段示意图
图 6  传统自编码器神经元全连接方式示意图
图 7  三通道彩色图像的卷积过程示意图
图 8  卷积编码以及反卷积解码示意图
图 9  CAENet训练过程
图 10  图像维度变换过程
图 11  CAENet伪代码流程图
图 12  CAENet训练中的图片特征逐步提取
图 13  不同算法在Corel5k数据集上的处理效果对比
图 14  细节增强测试实验中的原始图像与低光图像
图 15  不同算法对岩石低光图像中岩石细节的增强效果对比
图 16  不同算法对湖面低光图像中水面倒影细节的增强效果对比
图 17  不同算法对建筑低光图像中门窗细节的增强效果对比
SSIM HE CLAHE MSRCR AMSRCR MSRCP SRIE LIME CAENet
岩石细节图像 0.412 4 0.659 6 0.546 8 0.398 9 0.370 7 0.734 5 0.545 1 0.875 9
湖面细节图像 0.804 3 0.689 2 0.701 3 0.725 6 0.600 8 0.775 3 0.700 0 0.827 8
建筑细节图像 0.703 9 0.758 5 0.622 3 0.633 8 0.555 1 0.645 6 0.694 3 0.835 8
表 1  不同图像增强算法与CAENet算法的结构相似度指标对比
NIQE HE CLAHE MSRCR AMSRCR MSRCP SRIE LIME CAENet
岩石细节图像 7.900 6 8.459 4 8.163 6 7.825 8 7.179 6 8.970 7 8.259 4 7.428 4
湖面细节图像 4.687 2 5.135 0 5.317 0 4.610 2 4.665 8 4.151 3 5.714 1 4.526 7
建筑细节图像 5.086 8 5.061 9 5.357 1 5.603 4 5.186 1 4.827 3 5.681 6 3.864 8
表 2  不同图像增强算法与CAENet算法的自然图像质量评价指标对比
图 18  低光含噪图像
图 19  不同算法对高斯噪声的处理效果
图 20  不同算法对椒盐噪声的处理效果
图 21  不同算法对斑点噪声的处理效果
图 22  不同算法对泊松噪声的处理效果
图 23  不同图像增强算法评价指标的对比
1 GUO X, LI Y, LING H LIME: low-light image enhancement via illumination map estimation[J]. IEEE Transactions on Image Processing, 2017, 26 (2): 982- 993
doi: 10.1109/TIP.2016.2639450
2 吕永标, 赵建伟, 曹飞龙 基于复合卷积神经网络的图像去噪算法[J]. 模式识别与人工智能, 2017, 30 (2): 97- 105
LV Yong-biao, ZHAO Jian-wei, CAO Fei-long Image denoising algorithm based on composite convolution neural network[J]. Pattern Recognition and Artificial Intelligence, 2017, 30 (2): 97- 105
3 RIVERA A R, RYU B, CHAE O Content-aware dark image enhancement through channel division[J]. IEEE Transactions on Image Processing, 2012, 21 (9): 3967- 3980
doi: 10.1109/TIP.2012.2198667
4 SINGH K, KAPOOR R image enhancement using exposure based sub image histogram equalization[J]. Pattern Recognition Letters, 2014, 36: 10- 14
doi: 10.1016/j.patrec.2013.08.024
5 REZA A M Realization of the contrast limited adaptive histogram equalization (CLAHE) for real-time image enhancement[J]. Journal of VLSI Signal Processing Systems for Signal, Image and Video Technology, 2004, 38 (1): 35- 44
doi: 10.1023/B:VLSI.0000028532.53893.82
6 方帅, 杨静荣, 曹洋, 等 图像引导滤波的局部多尺度Retinex算法[J]. 中国图象图形学报, 2012, 17 (7): 748- 755
FANG Shuai, YANG Jing-rong, CAO Yang, et al A local multiscale Retinex algorithm for image guided filtering[J]. Journal of Image and Graphics, 2012, 17 (7): 748- 755
7 JOBSON D J, RAHMAN Z, WOODELL G A Properties and performance of a center/surround retinex[J]. IEEE Transactions on Image Processing, 1997, 6 (3): 451- 462
doi: 10.1109/83.557356
8 汪荣贵, 张新彤, 张璇, 等 基于Zernike矩的新型Retinex图像增强方法研究[J]. 中国图象图形学报, 2011, 16 (3): 310- 315
WANG Rong-giui, ZHANG Xin-tong, ZHANG Xuan, et al Research on the new Retinex image enhancment method based on zernike moment[J]. Journal of Image and Graphics, 2011, 16 (3): 310- 315
9 WANG J, LU K, XUE J, et al Single image dehazing based on the physical model and MSRCR algorithm[J]. IEEE Transactions on Circuits and Systems for Video Technology, 2017, PP (99): 1- 1
10 PARTHASARATHY S, SANKARAN P. An automated multi scale retinex with color restoration for image enhancement [C] // IEEE 2012 National Conference on Communications (NCC). Kharagpur: IEEE, 2012: 1-5.
11 PETRO A B, SBERT C, MOREL J M Multiscale Retinex[J]. Image Processing on Line, 2014, 71- 88
12 FU X, ZENG D, HUANG Y, et al. A weighted variational model for simultaneous reflectance and illumination estimation [C] // Computer Vision and Pattern Recognition (CVPR). Las Vegas, Nevada, USA: IEEE, 2016.
13 黄立威, 江碧涛, 吕守业, 等 基于深度学习的推荐系统研究综述[J]. 计算机学报, 2018, 1- 29
HUANG Li-wei, JIANG Bi-tao, LV Shou-ye, et al Review of the recommendation system based on deep learning[J]. Chinese Journal of Computers, 2018, 1- 29
doi: 10.11897/SP.J.1016.2018.00001
14 SHEN L, YUE Z, FENG F, et al. Msr-net: low-light image enhancement using deep convolutional network [J]. ArXiv Preprint. ArXiv: 1711. 02488, 2017.
15 LI C, GUO J, PORIKLI F, et al LightenNet: a convolutional neural network for weakly illuminated image enhancement[J]. Pattern Recognition Letters, 2018, 104: 15- 22
doi: 10.1016/j.patrec.2018.01.010
16 王万良. 人工智能及其应用: 第3版[M]. 北京: 高等教育出版社, 2015: 319.
17 LORE K G, AKINTAYO A, SARKAR S LLNet: A deep autoencoder approach to natural low-Light image enhancement[J]. Pattern Recognition, 2017, 61: 650- 662
doi: 10.1016/j.patcog.2016.06.008
18 黄健航, 雷迎科 基于边际Fisher深度自编码器的电台指纹特征提取[J]. 模式识别与人工智能, 2017, 30 (11): 1030- 1038
HUANG Jian-hang, LEI Ying-ke Radio fingerprint feature extraction based on marginal fisher depth autoencoder[J]. Pattern Recognition and Artificial Intelligence, 2017, 30 (11): 1030- 1038
19 VINCENT P, LAROCHELLE H, BRNGIO Y, et al. Extracting and composing robust features with denoising autoencoders [C] // Proceedings of the 25th International Conference on Machine Learning (ICML2008). Helsinki: ACM, 2008: 1096-1103.
20 MASCI J, MEIER U, CIRESAN D, et al. Stacked convolutional auto-encoders for hierarchical feature extraction [C] // International Conference on Artificial Neural Networks (ICANN). Springer, Berlin, Heidelberg: 2011: 52-59.
21 袁非牛, 章琳, 史劲亭,等. 自编码神经网络理论及应用综述[J]. 计算机学报, 2019, 42(1): 205-232.
YUAN Fei-niu, ZHANG Lin, SHI Jin-ting,et al.Self-encoding neural network theory and application review [J]. Chinese Journal of Computers, 2019, 42(1): 205-232.
22 VINCENT P, LAROCHELLE H, LAJOIE I, et al Stacked denoising autoencoders: learning useful representations in a deep network with a local denoising criterion[J]. Journal of Machine Learning Research, 2010, 11 (12): 3371- 3408
23 丁毅, 李玉惠, 李勃. 基于图像不同亮度区域特征的Gamma矫正方法[J]. 计算机技术与发展, 2016, 26(6): 37–39+45.
DING Yi, LI Yu-hui, LI Bo. Gamma correction method based on features of different brightness regions of images[J]. Computer Technology and Development, 2016, 26(6): 37-39.
24 张菲菲, 谢伟, 石强, 等 人眼视觉感知驱动的梯度域低照度图像对比度增强[J]. 计算机辅助设计与图形学学报, 2014, 26 (11): 1981- 1988
ZHANG Fei-fei, XIE Wei, SHI Qiang, et al Contrast enhancement of gradient domain low illumination image driven by human visual perception[J]. Journal of Computer-Aided Design and Computer Graphics, 2014, 26 (11): 1981- 1988
25 张林, 程华, 房一泉 基于卷积神经网络的链接表示及预测方法[J]. 浙江大学学报: 工学版, 2018, 52 (3): 552- 559
ZHANG Lin, CHENG Hua, FANG Yi-quan Link representation and prediction method based on convolutional neural network[J]. Journal of Zhejiang University: Engineering Science, 2018, 52 (3): 552- 559
26 GU J, WANG Z, KUEN J, et al Recent advances in convolutional neural networks[J]. Pattern Recognition, 2018, 77: 354- 377
doi: 10.1016/j.patcog.2017.10.013
27 ZEILER M D, KRISHNAN D, TATLOR G W, et al. Deconvolutional networks [C] // Computer Vision and Pattern Recognition (CVPR). Las Vegas: IEEE, 2010: 2528-2535.
28 袁公萍, 汤一平, 韩旺明, 等 基于深度卷积神经网络的车型识别方法[J]. 浙江大学学报: 工学版, 2018, 52 (4): 694- 702
YUAN Gong-ping, TANG Yi-ping, HAN Wang-ming, et al A vehicle identification method based on deep convolution neural network[J]. Journal of Zhejiang University: Engineering Science, 2018, 52 (4): 694- 702
29 LOZA A, BULL D R, HILL P R, et al Automatic contrast enhancement of low-light images based on local statistics of wavelet coefficients[J]. Digital Signal Processing, 2013, 23 (6): 1856- 1866
doi: 10.1016/j.dsp.2013.06.002
30 YE X, WANG L, XING H, et al. Denoising hybrid noises in image with stacked autoencoder [C] // Information and Automation (ICIA), Lijiang: IEEE, 2015: 2720-2724.
[1] 程训,余建波. 基于机器视觉的加工刀具磨损监测方法[J]. 浙江大学学报(工学版), 2021, 55(5): 896-904.
[2] 许佳辉,王敬昌,陈岭,吴勇. 基于图神经网络的地表水水质预测模型[J]. 浙江大学学报(工学版), 2021, 55(4): 601-607.
[3] 王虹力,郭斌,刘思聪,刘佳琪,仵允港,於志文. 边端融合的终端情境自适应深度感知模型[J]. 浙江大学学报(工学版), 2021, 55(4): 626-638.
[4] 张腾,蒋鑫龙,陈益强,陈前,米涛免,陈彪. 基于腕部姿态的帕金森病用药后开-关期检测[J]. 浙江大学学报(工学版), 2021, 55(4): 639-647.
[5] 郑英杰,吴松荣,韦若禹,涂振威,廖进,刘东. 基于目标图像FCM算法的地铁定位点匹配及误报排除方法[J]. 浙江大学学报(工学版), 2021, 55(3): 586-593.
[6] 雍子叶,郭继昌,李重仪. 融入注意力机制的弱监督水下图像增强算法[J]. 浙江大学学报(工学版), 2021, 55(3): 555-562.
[7] 徐利锋,黄海帆,丁维龙,范玉雷. 基于改进DenseNet的水果小目标检测[J]. 浙江大学学报(工学版), 2021, 55(2): 377-385.
[8] 许豪灿,李基拓,陆国栋. 由LeNet-5从单张着装图像重建三维人体[J]. 浙江大学学报(工学版), 2021, 55(1): 153-161.
[9] 黄毅鹏,胡冀苏,钱旭升,周志勇,赵文露,马麒,沈钧康,戴亚康. SE-Mask-RCNN:多参数MRI前列腺癌分割方法[J]. 浙江大学学报(工学版), 2021, 55(1): 203-212.
[10] 陈巧红,陈翊,李文书,贾宇波. 多尺度SE-Xception服装图像分类[J]. 浙江大学学报(工学版), 2020, 54(9): 1727-1735.
[11] 郑浦,白宏阳,李伟,郭宏伟. 复杂背景下的小目标检测算法[J]. 浙江大学学报(工学版), 2020, 54(9): 1777-1784.
[12] 周登文,田金月,马路遥,孙秀秀. 基于多级特征并联的轻量级图像语义分割[J]. 浙江大学学报(工学版), 2020, 54(8): 1516-1524.
[13] 张彦楠,黄小红,马严,丛群. 基于深度学习的录音文本分类方法[J]. 浙江大学学报(工学版), 2020, 54(7): 1264-1271.
[14] 明涛,王丹,郭继昌,李锵. 基于多尺度通道重校准的乳腺癌病理图像分类[J]. 浙江大学学报(工学版), 2020, 54(7): 1289-1297.
[15] 闫旭,范晓亮,郑传潘,臧彧,王程,程明,陈龙彪. 基于图卷积神经网络的城市交通态势预测算法[J]. 浙江大学学报(工学版), 2020, 54(6): 1147-1155.