|
|
Underwater image enhancement algorithm based on GAN and multi-level wavelet CNN |
Pei-zhi WEN1(),Jun-mou CHEN1,Yan-nan XIAO1,Ya-yuan WEN2,Wen-ming HUANG1 |
1. School of Computer Science and Information Security, Guilin University of Electronic Technology, Guilin 541004, China 2. College of Electronic Engineering, Guangxi Normal University, Guilin 541004, China |
|
|
Abstract An underwater image enhancement algorithm was proposed based on generative adversarial networks (GAN) and improved convolutional neural networks (CNN) in order to solve the problems of haze blurring and color distortion of underwater image. Generative adversarial network was used to synthesize underwater images to effectively expand the paired underwater data set. The underwater image was decomposed by multi-scale wavelet transform without losing the feature resolution. Then, combined with CNN, the compact learning method was used to extract features from multi-scale images, and skip connection was used to prevent gradient dispersion. Finally, the fog blur effect of the underwater image was resolved. In order to improve the color correction ability of the model and overcome the problem of color distortion of underwater images, the correlation between different channels of color images was learned by using the style cost function. Experimental results show that, in subjective visual and objective indicators, the proposed algorithm is superior to the contrast algorithm in comprehensive performance and robustness.
|
Received: 29 March 2021
Published: 03 March 2022
|
|
基于生成式对抗网络和多级小波包卷积网络的水下图像增强算法
为了解决水下图像的雾模糊和偏色问题,针对水下图像成像模型提出基于生成式对抗网络(GAN)和改进卷积神经网络(CNN)的水下图像增强算法. 利用生成式对抗网络合成水下图像,以对配对式水下图像数据集进行有效扩充. 利用多级小波变换,以不丢失特征分辨率的方式对水下图像进行多尺度分解,然后结合卷积神经网络利用紧凑式学习方式对多尺度图像进行特征提取,并利用跳跃连接以防止梯度弥散,克服水下图像的雾模糊效应. 利用风格代价函数学习彩色图像各通道间的相关性,提高模型的色彩校正能力,克服水下图像色彩失真的问题. 实验结果表明,相较对比算法,在主观视觉和客观指标上,本研究所提算法拥有更优秀的综合性能及鲁棒性.
关键词:
图像处理,
水下图像增强,
多级小波变换,
卷积神经网络,
生成式对抗网络
|
|
[1] |
STANKIEWICZ P, TAN Y T, KOBILAROV M Adaptive s-ampling with an autonomous underwater vehicle in static marine environments[J]. Journal of Field Robotics, 2021, 38 (4): 572- 597
doi: 10.1002/rob.22005
|
|
|
[2] |
RUMSON A. Development of autonomous subsea pipeline inspection capabilities[C]// Global Oceans 2020: Singapore–US Gulf Coast. Biloxi: IEEE, 2020: 1-6.
|
|
|
[3] |
CEJKA J, BRUNO F, SKARLATOS D, et al Detecting square markers in underwater environments[J]. Remote Sensing, 2019, 11 (4): 23
|
|
|
[4] |
ANCUTI C, ANCUTI C O, HABER T, et al. Enhancing underwater images and videos by fusion[C]// 2012 IEEE Conference on Computer Vision and Pattern Recognition. Providence: IEEE, 2012: 81-88.
|
|
|
[5] |
HUANG D, WANG Y, SONG W, et al. Shallow-water image enhancement using relative global histogram stretching based on adaptive parameter acquisition[C]// International Conference on Multimedia Modeling. Bangkok: Springer, 2018: 453-465.
|
|
|
[6] |
DREWS P, NASCIMENTO E, MORAES F, et al. Transmission estimation in underwater single images[C]// Proceedings of the IEEE International Conference on Computer Vision Workshops. Sydney: IEEE, 2013: 825-830.
|
|
|
[7] |
PENG Y, COSMAN P C Underwater image restoration based on image blurriness and light absorption[J]. IEEE Transactions on Image Processing, 2017, 26 (4): 1579- 1594
doi: 10.1109/TIP.2017.2663846
|
|
|
[8] |
SONG W, WANG Y, HUANG D, et al. A rapid scene depth estimation model based on underwater light attenuation prior for underwater image restoration[C]// Pacific Rim Conference on Multimedia. Hefei: Springer, 2018: 678-688.
|
|
|
[9] |
LI J, SKINNER K A, EUSTICE R M, et al WaterGAN: uns-upervised generative network to enable real-time color correction of monocular underwater images[J]. IEEE Robotics and Automation Letters, 2018, 3 (1): 387- 394
|
|
|
[10] |
LI C Y, GUO J C, GUO C L Emerging from water: underwater image color correction based on weakly supervised color transfer[J]. IEEE Signal Processing Letters, 2018, 25 (3): 323- 327
doi: 10.1109/LSP.2018.2792050
|
|
|
[11] |
FABBRI C, ISLAM M J, SATTAR J. Enhancing underwater imagery using generative adversarial networks[C]// 2018 IEEE International Conference on Robotics and Automation. Brisbane: IEEE, 2018: 7159-7165.
|
|
|
[12] |
WANG N, ZHOU Y, HAN F, et al. UWGAN: underwater GAN for real-world underwater color restoration and dehazing [EB/OL]. (2019-12-21). https://arxiv.org/ftp/arxiv/papers/1912/1912.10269.pdf.
|
|
|
[13] |
LIU P, ZHANG H, ZHANG K, et al. Multi-level wavelet-CNN for image restoration[C]// Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition Workshops. Salt Lake City: IEEE, 2018: 773-782.
|
|
|
[14] |
JAFFE J S Computer modeling and the design of optimal underwater imaging systems[J]. IEEE Journal of Oceanic Engineering, 1990, 15 (2): 101- 111
doi: 10.1109/48.50695
|
|
|
[15] |
AKKAYNAK D, TREIBITZ T. A revised underwater image formation model[C]// Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition. Salt Lake City: IEEE, 2018: 6723-6732.
|
|
|
[16] |
AKKAYNAK D, TREIBITZ T, SHLESINGER T, et al. What is the space of attenuation coefficients in underwater computer vision?[C]// Proceedings of the IEEE Conference on Com-puter Vision and Pattern Recognition. Honolulu: IEEE, 2017: 4931-4940.
|
|
|
[17] |
GALDRAN A, PARDO D, PICON A, et al Automatic red channel underwater image restoration[J]. Journal of Visual Communication and Image Representation, 2015, 26: 132- 145
doi: 10.1016/j.jvcir.2014.11.006
|
|
|
[18] |
ZHAO X W, JIN T, QU S Deriving inherent optical properties from background color and underwater image enhancement[J]. Ocean Engineering, 2015, 94: 163- 172
doi: 10.1016/j.oceaneng.2014.11.036
|
|
|
[19] |
AKKAYNAK D, TREIBITZ T. Sea-thru: a method for removing water from underwater images[C]// Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Reco-gnition. Long Beach: IEEE, 2019: 1682-1691.
|
|
|
[20] |
GODARD C, MAC AODHA O, FIRMAN M, et al. Digging into self-supervised monocular depth estimation[C]// Proceedings of the IEEE/CVF International Conference on Computer Vision. Seoul: IEEE, 2019: 3828-3838.
|
|
|
[21] |
WEI K, FU Y, YANG J, et al. A physics-based noise formation model for extreme low-light raw denoising [C]// Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition. Seattle: IEEE, 2020: 2758-2767.
|
|
|
[22] |
RADFORD A, METZ L, CHINTALA S. Unsupervised representation learning with deep convolutional generative adversarial networks [EB/OL]. (2015-11-19). https://arxiv.org/pdf/1511.06434.pdf.
|
|
|
[23] |
RONNEBERGER O, FISCHER P, BROX T. U-net: convolutional networks for biomedical image segmentation[C]// International Conference on Medical Image Computing and Computer-assisted Intervention. Munich: Springer, 2015: 234-241.
|
|
|
[24] |
ODENA A, DUMOULIN V, OLAH C. Deconvolution and checkerboard artifacts [EB/OL]. [2021-03-01]. https://distill.pub/2016/deconv-checkerboard/.
|
|
|
[25] |
GATYS L A, ECKER A S, BETHGE M. A neural algorithm of artistic style [EB/OL]. (2015-08-26). https://arxiv.org/pdf/1508.06576.pdf.
|
|
|
[26] |
ISLAM M J, XIA Y, SATTAR J Fast underwater image enh-ancement for improved visual perception[J]. IEEE Robotics and Automation Letters, 2020, 5 (2): 3227- 3234
doi: 10.1109/LRA.2020.2974710
|
|
|
[27] |
SILBERMAN N, FERGUS R. Indoor scene segmentation using a structured light sensor[C]// 2011 IEEE International Conference on Computer Vision Workshops. Barcelona: IEEE, 2011: 601-608.
|
|
|
[28] |
SILBERMAN N, HOIEM D, KOHLI P, et al. Indoor segme-ntation and support inference from rgbd images[C]// European Conference on Computer Vision. Florence: Springer, 2012: 746-760.
|
|
|
[29] |
URPC竞赛项目: 水下目标检测 [DS/OL]. [2019-8-8]. http://www.cnurpc.org/a/xwjrz/2019/0808/129.html.
|
|
|
[30] |
YANG M, SOWMYA A An underwater color image quality evaluation metric[J]. IEEE Transactions on Image Processing, 2015, 24 (12): 6062- 6071
doi: 10.1109/TIP.2015.2491020
|
|
|
[31] |
PANETTA K, GAO C, AGAIAN S Human-visual-system-inspired underwater image quality measures[J]. IEEE Journal of Oceanic Engineering, 2015, 41 (3): 541- 551
|
|
|
[32] |
MITTAL A, SOUNDARARAJAN R, BOVIK A C Making a “completely blind” image quality analyzer[J]. IEEE Signal Processing Letters, 2013, 20 (3): 209- 212
doi: 10.1109/LSP.2012.2227726
|
|
|
|
Viewed |
|
|
|
Full text
|
|
|
|
|
Abstract
|
|
|
|
|
Cited |
|
|
|
|
|
Shared |
|
|
|
|
|
Discussed |
|
|
|
|