Please wait a minute...
Journal of ZheJiang University (Engineering Science)  2019, Vol. 53 Issue (9): 1728-1740    DOI: 10.3785/j.issn.1008-973X.2019.09.012
Computer Science and Artificial Intelligence     
Image enhancement algorithm with convolutional auto-encoder network
Wan-liang WANG(),Xiao-han YANG,Yan-wei ZHAO,Nan GAO,Chuang LV,Zhao-juan ZHANG
School of Computer Science and Technology, College of Mechanical Engineering, Zhejiang University of Technology, Hangzhou 310023, China
Download: HTML     PDF(2256KB) HTML
Export: BibTeX | EndNote (RIS)      

Abstract  

When the image enhancement method LLNet (the low-light net) was applied to three-channel images, there're a lot of redundant parameters. To solve this problem, a framework called CAENet (convolutional auto-encoder network) was proposed. Firstly, CAENet combined a low light processing module with a network training module. Secondly, in the encoding and decoding stages, CAENet used a convolutional network to replace the traditional fully connected network. The experimental results show that connecting low-light processing modules with network training can effectively save time costs. At the same time, the use of convolutional networks can reduce network parameters, making network training more efficient, and obtain better low-dimensional representation of images. The experimental results on the Corel5k dataset show that CAENet can effectively improve the image light perception and color perception while reducing network parameters. The experimental results on high-resolution datasets show that CAENet can preserve details for image details without distortion. In addition, for the noisy low-light image, CAENet can enhance the image while achieving the denoising effect, which proves that CAENet has strong robustness.



Key wordsimage processing      image enhancement      deep learning      convolutional neural network      denoising auto-encoder     
Received: 24 January 2019      Published: 12 September 2019
CLC:  TP 311.1  
Cite this article:

Wan-liang WANG,Xiao-han YANG,Yan-wei ZHAO,Nan GAO,Chuang LV,Zhao-juan ZHANG. Image enhancement algorithm with convolutional auto-encoder network. Journal of ZheJiang University (Engineering Science), 2019, 53(9): 1728-1740.

URL:

http://www.zjujournals.com/eng/10.3785/j.issn.1008-973X.2019.09.012     OR     http://www.zjujournals.com/eng/Y2019/V53/I9/1728


采用卷积自编码器网络的图像增强算法

将图像增强方法低光网络(LLNet)应用于实际场景下的彩色图像时会产生大量冗余参数,为此基于LLNet提出卷积自编码器网络(CAENet)的图像增强方法. 将LLNet方法中的低光处理模块与网络训练衔接在一起;采用卷积网络代替传统自编码器的编码和解码方式. 实验结果表明:CAENet能够有效节约时间成本,减少网络参数,使网络训练更加高效,得到更好的图像低维表示. 在Corel5k数据集上的实验效果表明,CAENet在减少网络参数的同时,能有效提高图像光感和色感;在高分辨率数据集上的实验结果表明,针对图像细节方面,CAENet能够保留细节不失真;针对含噪低光图像,CAENet能在增强图像的同时达到去噪的效果,证明CAENet具有较强的鲁棒性.


关键词: 图像处理,  图像增强,  深度学习,  卷积神经网络,  降噪自编码器 
Fig.1 Structure diagram of low light net (LLNet)
Fig.2 Diagram of module structure of LLNet
Fig.3 Schematic diagram of convolutional auto-encoder network (CAENet)
Fig.4 Image enhancement schematic diagram under manifold learning method
Fig.5 Diagram of low light processing stage of LLNet
Fig.6 Diagram of full connection of traditional autoencoder
Fig.7 Diagram of convolutional process of three-channel image
Fig.8 Diagram of convolutional coding and deconvolutional decoding
Fig.9 Process of CAENet training
Fig.10 Process of change in image dimensions
Fig.11 Flow chart of CAENet pseudo code
Fig.12 Image feature extraction step by step using CAENet algorithm
Fig.13 Effect comparison of different algorithms on Corel5k dataset
Fig.14 Original image and low light image in detail enhancement test
Fig.15 Comparison of enhancement effects of different algorithms on rock detail in rock low light images
Fig.16 Comparison of enhancement effects of different algorithms on reflection of water surface in low-light images of lakes
Fig.17 Comparison of enhancement effects of different algorithms on detail of doors and windows in low-light images of buildings
SSIM HE CLAHE MSRCR AMSRCR MSRCP SRIE LIME CAENet
岩石细节图像 0.412 4 0.659 6 0.546 8 0.398 9 0.370 7 0.734 5 0.545 1 0.875 9
湖面细节图像 0.804 3 0.689 2 0.701 3 0.725 6 0.600 8 0.775 3 0.700 0 0.827 8
建筑细节图像 0.703 9 0.758 5 0.622 3 0.633 8 0.555 1 0.645 6 0.694 3 0.835 8
Tab.1 Comparison of structural similarity index between different image enhancement algorithms and CAENet algorithm
NIQE HE CLAHE MSRCR AMSRCR MSRCP SRIE LIME CAENet
岩石细节图像 7.900 6 8.459 4 8.163 6 7.825 8 7.179 6 8.970 7 8.259 4 7.428 4
湖面细节图像 4.687 2 5.135 0 5.317 0 4.610 2 4.665 8 4.151 3 5.714 1 4.526 7
建筑细节图像 5.086 8 5.061 9 5.357 1 5.603 4 5.186 1 4.827 3 5.681 6 3.864 8
Tab.2 Comparison of natural image quality evaluator between different image enhancement algorithms and CAENet algorithm
Fig.18 Low light noisy images
Fig.19 Effect of different algorithms on Gaussian noise
Fig.20 Effect of different algorithms on salt and pepper noise
Fig.21 Effect of different algorithms on speckle noise
Fig.22 Effect of different algorithms on poisson noise
Fig.23 Comparison of evaluation indexes by different image enhancement algorithms
[1]   GUO X, LI Y, LING H LIME: low-light image enhancement via illumination map estimation[J]. IEEE Transactions on Image Processing, 2017, 26 (2): 982- 993
doi: 10.1109/TIP.2016.2639450
[2]   吕永标, 赵建伟, 曹飞龙 基于复合卷积神经网络的图像去噪算法[J]. 模式识别与人工智能, 2017, 30 (2): 97- 105
LV Yong-biao, ZHAO Jian-wei, CAO Fei-long Image denoising algorithm based on composite convolution neural network[J]. Pattern Recognition and Artificial Intelligence, 2017, 30 (2): 97- 105
[3]   RIVERA A R, RYU B, CHAE O Content-aware dark image enhancement through channel division[J]. IEEE Transactions on Image Processing, 2012, 21 (9): 3967- 3980
doi: 10.1109/TIP.2012.2198667
[4]   SINGH K, KAPOOR R image enhancement using exposure based sub image histogram equalization[J]. Pattern Recognition Letters, 2014, 36: 10- 14
doi: 10.1016/j.patrec.2013.08.024
[5]   REZA A M Realization of the contrast limited adaptive histogram equalization (CLAHE) for real-time image enhancement[J]. Journal of VLSI Signal Processing Systems for Signal, Image and Video Technology, 2004, 38 (1): 35- 44
doi: 10.1023/B:VLSI.0000028532.53893.82
[6]   方帅, 杨静荣, 曹洋, 等 图像引导滤波的局部多尺度Retinex算法[J]. 中国图象图形学报, 2012, 17 (7): 748- 755
FANG Shuai, YANG Jing-rong, CAO Yang, et al A local multiscale Retinex algorithm for image guided filtering[J]. Journal of Image and Graphics, 2012, 17 (7): 748- 755
[7]   JOBSON D J, RAHMAN Z, WOODELL G A Properties and performance of a center/surround retinex[J]. IEEE Transactions on Image Processing, 1997, 6 (3): 451- 462
doi: 10.1109/83.557356
[8]   汪荣贵, 张新彤, 张璇, 等 基于Zernike矩的新型Retinex图像增强方法研究[J]. 中国图象图形学报, 2011, 16 (3): 310- 315
WANG Rong-giui, ZHANG Xin-tong, ZHANG Xuan, et al Research on the new Retinex image enhancment method based on zernike moment[J]. Journal of Image and Graphics, 2011, 16 (3): 310- 315
[9]   WANG J, LU K, XUE J, et al Single image dehazing based on the physical model and MSRCR algorithm[J]. IEEE Transactions on Circuits and Systems for Video Technology, 2017, PP (99): 1- 1
[10]   PARTHASARATHY S, SANKARAN P. An automated multi scale retinex with color restoration for image enhancement [C] // IEEE 2012 National Conference on Communications (NCC). Kharagpur: IEEE, 2012: 1-5.
[11]   PETRO A B, SBERT C, MOREL J M Multiscale Retinex[J]. Image Processing on Line, 2014, 71- 88
[12]   FU X, ZENG D, HUANG Y, et al. A weighted variational model for simultaneous reflectance and illumination estimation [C] // Computer Vision and Pattern Recognition (CVPR). Las Vegas, Nevada, USA: IEEE, 2016.
[13]   黄立威, 江碧涛, 吕守业, 等 基于深度学习的推荐系统研究综述[J]. 计算机学报, 2018, 1- 29
HUANG Li-wei, JIANG Bi-tao, LV Shou-ye, et al Review of the recommendation system based on deep learning[J]. Chinese Journal of Computers, 2018, 1- 29
doi: 10.11897/SP.J.1016.2018.00001
[14]   SHEN L, YUE Z, FENG F, et al. Msr-net: low-light image enhancement using deep convolutional network [J]. ArXiv Preprint. ArXiv: 1711. 02488, 2017.
[15]   LI C, GUO J, PORIKLI F, et al LightenNet: a convolutional neural network for weakly illuminated image enhancement[J]. Pattern Recognition Letters, 2018, 104: 15- 22
doi: 10.1016/j.patrec.2018.01.010
[16]   王万良. 人工智能及其应用: 第3版[M]. 北京: 高等教育出版社, 2015: 319.
[17]   LORE K G, AKINTAYO A, SARKAR S LLNet: A deep autoencoder approach to natural low-Light image enhancement[J]. Pattern Recognition, 2017, 61: 650- 662
doi: 10.1016/j.patcog.2016.06.008
[18]   黄健航, 雷迎科 基于边际Fisher深度自编码器的电台指纹特征提取[J]. 模式识别与人工智能, 2017, 30 (11): 1030- 1038
HUANG Jian-hang, LEI Ying-ke Radio fingerprint feature extraction based on marginal fisher depth autoencoder[J]. Pattern Recognition and Artificial Intelligence, 2017, 30 (11): 1030- 1038
[19]   VINCENT P, LAROCHELLE H, BRNGIO Y, et al. Extracting and composing robust features with denoising autoencoders [C] // Proceedings of the 25th International Conference on Machine Learning (ICML2008). Helsinki: ACM, 2008: 1096-1103.
[20]   MASCI J, MEIER U, CIRESAN D, et al. Stacked convolutional auto-encoders for hierarchical feature extraction [C] // International Conference on Artificial Neural Networks (ICANN). Springer, Berlin, Heidelberg: 2011: 52-59.
[21]   袁非牛, 章琳, 史劲亭,等. 自编码神经网络理论及应用综述[J]. 计算机学报, 2019, 42(1): 205-232.
YUAN Fei-niu, ZHANG Lin, SHI Jin-ting,et al.Self-encoding neural network theory and application review [J]. Chinese Journal of Computers, 2019, 42(1): 205-232.
[22]   VINCENT P, LAROCHELLE H, LAJOIE I, et al Stacked denoising autoencoders: learning useful representations in a deep network with a local denoising criterion[J]. Journal of Machine Learning Research, 2010, 11 (12): 3371- 3408
[23]   丁毅, 李玉惠, 李勃. 基于图像不同亮度区域特征的Gamma矫正方法[J]. 计算机技术与发展, 2016, 26(6): 37–39+45.
DING Yi, LI Yu-hui, LI Bo. Gamma correction method based on features of different brightness regions of images[J]. Computer Technology and Development, 2016, 26(6): 37-39.
[24]   张菲菲, 谢伟, 石强, 等 人眼视觉感知驱动的梯度域低照度图像对比度增强[J]. 计算机辅助设计与图形学学报, 2014, 26 (11): 1981- 1988
ZHANG Fei-fei, XIE Wei, SHI Qiang, et al Contrast enhancement of gradient domain low illumination image driven by human visual perception[J]. Journal of Computer-Aided Design and Computer Graphics, 2014, 26 (11): 1981- 1988
[25]   张林, 程华, 房一泉 基于卷积神经网络的链接表示及预测方法[J]. 浙江大学学报: 工学版, 2018, 52 (3): 552- 559
ZHANG Lin, CHENG Hua, FANG Yi-quan Link representation and prediction method based on convolutional neural network[J]. Journal of Zhejiang University: Engineering Science, 2018, 52 (3): 552- 559
[26]   GU J, WANG Z, KUEN J, et al Recent advances in convolutional neural networks[J]. Pattern Recognition, 2018, 77: 354- 377
doi: 10.1016/j.patcog.2017.10.013
[27]   ZEILER M D, KRISHNAN D, TATLOR G W, et al. Deconvolutional networks [C] // Computer Vision and Pattern Recognition (CVPR). Las Vegas: IEEE, 2010: 2528-2535.
[28]   袁公萍, 汤一平, 韩旺明, 等 基于深度卷积神经网络的车型识别方法[J]. 浙江大学学报: 工学版, 2018, 52 (4): 694- 702
YUAN Gong-ping, TANG Yi-ping, HAN Wang-ming, et al A vehicle identification method based on deep convolution neural network[J]. Journal of Zhejiang University: Engineering Science, 2018, 52 (4): 694- 702
[29]   LOZA A, BULL D R, HILL P R, et al Automatic contrast enhancement of low-light images based on local statistics of wavelet coefficients[J]. Digital Signal Processing, 2013, 23 (6): 1856- 1866
doi: 10.1016/j.dsp.2013.06.002
[30]   YE X, WANG L, XING H, et al. Denoising hybrid noises in image with stacked autoencoder [C] // Information and Automation (ICIA), Lijiang: IEEE, 2015: 2720-2724.
[1] Xun CHENG,Jian-bo YU. Monitoring method for machining tool wear based on machine vision[J]. Journal of ZheJiang University (Engineering Science), 2021, 55(5): 896-904.
[2] Jia-hui XU,Jing-chang WANG,Ling CHEN,Yong WU. Surface water quality prediction model based on graph neural network[J]. Journal of ZheJiang University (Engineering Science), 2021, 55(4): 601-607.
[3] Hong-li WANG,Bin GUO,Si-cong LIU,Jia-qi LIU,Yun-gang WU,Zhi-wen YU. End context-adaptative deep sensing model with edge-end collaboration[J]. Journal of ZheJiang University (Engineering Science), 2021, 55(4): 626-638.
[4] Teng ZHANG,Xin-long JIANG,Yi-qiang CHEN,Qian CHEN,Tao-mian MI,Piu CHAN. Wrist attitude-based Parkinson's disease ON/OFF state assessment after medication[J]. Journal of ZheJiang University (Engineering Science), 2021, 55(4): 639-647.
[5] Ying-jie ZHENG,Song-rong WU,Ruo-yu WEI,Zhen-wei TU,Jin LIAO,Dong LIU. Metro location point matching and false alarm elimination based on FCM algorithm of target image[J]. Journal of ZheJiang University (Engineering Science), 2021, 55(3): 586-593.
[6] Zi-ye YONG,Ji-chang GUO,Chong-yi LI. weakly supervised underwater image enhancement algorithm incorporating attention mechanism[J]. Journal of ZheJiang University (Engineering Science), 2021, 55(3): 555-562.
[7] Li-feng XU,Hai-fan HUANG,Wei-long DING,Yu-lei FAN. Detection of small fruit target based on improved DenseNet[J]. Journal of ZheJiang University (Engineering Science), 2021, 55(2): 377-385.
[8] Hao-can XU,Ji-tuo LI,Guo-dong LU. Reconstruction of three-dimensional human bodies from single image by LeNet-5[J]. Journal of ZheJiang University (Engineering Science), 2021, 55(1): 153-161.
[9] Yi-peng HUANG,Ji-su HU,Xu-sheng QIAN,Zhi-yong ZHOU,Wen-lu ZHAO,Qi MA,Jun-kang SHEN,Ya-kang DAI. SE-Mask-RCNN: segmentation method for prostate cancer on multi-parametric MRI[J]. Journal of ZheJiang University (Engineering Science), 2021, 55(1): 203-212.
[10] Qiao-hong CHEN,YI CHEN,Wen-shu Li,Yu-bo JIA. Clothing image classification based on multi-scale SE-Xception[J]. Journal of ZheJiang University (Engineering Science), 2020, 54(9): 1727-1735.
[11] Pu ZHENG,Hong-yang BAI,Wei LI,Hong-wei GUO. Small target detection algorithm in complex background[J]. Journal of ZheJiang University (Engineering Science), 2020, 54(9): 1777-1784.
[12] Deng-wen ZHOU,Jin-yue TIAN,Lu-yao MA,Xiu-xiu SUN. Lightweight image semantic segmentation based on multi-level feature cascaded network[J]. Journal of ZheJiang University (Engineering Science), 2020, 54(8): 1516-1524.
[13] Yan-nan ZHANG,Xiao-hong HUANG,Yan MA,Qun CONG. Method with recording text classification based on deep learning[J]. Journal of ZheJiang University (Engineering Science), 2020, 54(7): 1264-1271.
[14] Tao MING,Dan WANG,Ji-chang GUO,Qiang LI. Breast cancer histopathological image classification using multi-scale channel squeeze-and-excitation model[J]. Journal of ZheJiang University (Engineering Science), 2020, 54(7): 1289-1297.
[15] Xu YAN,Xiao-liang FAN,Chuan-pan ZHENG,Yu ZANG,Cheng WANG,Ming CHENG,Long-biao CHEN. Urban traffic flow prediction algorithm based on graph convolutional neural networks[J]. Journal of ZheJiang University (Engineering Science), 2020, 54(6): 1147-1155.