Please wait a minute...
浙江大学学报(工学版)  2022, Vol. 56 Issue (2): 213-224    DOI: 10.3785/j.issn.1008-973X.2022.02.001
计算机与控制工程     
基于生成式对抗网络和多级小波包卷积网络的水下图像增强算法
温佩芝1(),陈君谋1,肖雁南1,温雅媛2,黄文明1
1. 桂林电子科技大学 计算机与信息安全学院,广西 桂林 541004
2. 广西师范大学 电子工程学院,广西 桂林 541004
Underwater image enhancement algorithm based on GAN and multi-level wavelet CNN
Pei-zhi WEN1(),Jun-mou CHEN1,Yan-nan XIAO1,Ya-yuan WEN2,Wen-ming HUANG1
1. School of Computer Science and Information Security, Guilin University of Electronic Technology, Guilin 541004, China
2. College of Electronic Engineering, Guangxi Normal University, Guilin 541004, China
 全文: PDF(1464 KB)   HTML
摘要:

为了解决水下图像的雾模糊和偏色问题,针对水下图像成像模型提出基于生成式对抗网络(GAN)和改进卷积神经网络(CNN)的水下图像增强算法. 利用生成式对抗网络合成水下图像,以对配对式水下图像数据集进行有效扩充. 利用多级小波变换,以不丢失特征分辨率的方式对水下图像进行多尺度分解,然后结合卷积神经网络利用紧凑式学习方式对多尺度图像进行特征提取,并利用跳跃连接以防止梯度弥散,克服水下图像的雾模糊效应. 利用风格代价函数学习彩色图像各通道间的相关性,提高模型的色彩校正能力,克服水下图像色彩失真的问题. 实验结果表明,相较对比算法,在主观视觉和客观指标上,本研究所提算法拥有更优秀的综合性能及鲁棒性.

关键词: 图像处理水下图像增强多级小波变换卷积神经网络生成式对抗网络    
Abstract:

An underwater image enhancement algorithm was proposed based on generative adversarial networks (GAN) and improved convolutional neural networks (CNN) in order to solve the problems of haze blurring and color distortion of underwater image. Generative adversarial network was used to synthesize underwater images to effectively expand the paired underwater data set. The underwater image was decomposed by multi-scale wavelet transform without losing the feature resolution. Then, combined with CNN, the compact learning method was used to extract features from multi-scale images, and skip connection was used to prevent gradient dispersion. Finally, the fog blur effect of the underwater image was resolved. In order to improve the color correction ability of the model and overcome the problem of color distortion of underwater images, the correlation between different channels of color images was learned by using the style cost function. Experimental results show that, in subjective visual and objective indicators, the proposed algorithm is superior to the contrast algorithm in comprehensive performance and robustness.

Key words: image processing    underwater image enhancement    multi-level wavelet transform    convolutional neural networks    generative adversarial networks
收稿日期: 2021-03-29 出版日期: 2022-03-03
CLC:  TP 391  
基金资助: 国家自然科学基金资助项目(62027826);广西图像图形与智能处理重点实验室培育基地开放基金资助项目(GIIP2011)
作者简介: 温佩芝(1963—),女,教授,从事图像处理研究. orcid.org/0000-0002-3920-5930. E-mail: wpzsia@163.com
服务  
把本文推荐给朋友
加入引用管理器
E-mail Alert
作者相关文章  
温佩芝
陈君谋
肖雁南
温雅媛
黄文明

引用本文:

温佩芝,陈君谋,肖雁南,温雅媛,黄文明. 基于生成式对抗网络和多级小波包卷积网络的水下图像增强算法[J]. 浙江大学学报(工学版), 2022, 56(2): 213-224.

Pei-zhi WEN,Jun-mou CHEN,Yan-nan XIAO,Ya-yuan WEN,Wen-ming HUANG. Underwater image enhancement algorithm based on GAN and multi-level wavelet CNN. Journal of ZheJiang University (Engineering Science), 2022, 56(2): 213-224.

链接本文:

https://www.zjujournals.com/eng/CN/10.3785/j.issn.1008-973X.2022.02.001        https://www.zjujournals.com/eng/CN/Y2022/V56/I2/213

图 1  水下图像生成模型
图 2  水下图像生成模型的参数估计器
图 3  参数估计器的卷积模块
图 4  水下图像增强模型
图 5  本研究算法与其他算法在URPC数据集上的视觉增强效果对比
图 6  本研究算法与其他算法在EUVP数据集上的视觉增强效果对比
图 7  本研究算法与UWGAN算法实验图像细节对比
算法 UIQM UCIQE NIQE
原始图像 2.787 0.432 6.973
IBLA 2.612 0.558 7.220
UDCP 2.073 0.523 6.924
ULAP 2.394 0.524 6.717
RGHS 2.752 0.575 7.164
Sea-thru 3.022 0.533 6.844
UWGAN 2.981 0.490 6.225
FunieGAN 3.054 0.494 10.300
FunieGAN-up 2.915 0.479 7.744
本研究算法 3.341 0.525 5.866
本研究算法-无色彩校正 3.021 0.497 5.988
表 1  本研究算法和其他算法在URPC数据集上的各项指标对比
算法 SSIM PSNR UIQM UCIQE NIQE
原始图像 0.814 29.320 2.549 0.550 8.192
IBLA 0.464 15.040 1.045 0.698 11.930
UDCP 0.734 19.430 2.028 0.588 7.958
ULAP 0.694 23.190 2.213 0.577 8.073
RGHS 0.778 26.660 2.337 0.598 8.084
Sea-thru 0.772 21.380 2.593 0.591 8.123
UWGAN 0.880 29.020 3.345 0.554 7.294
FunieGAN 0.796 26.300 3.236 0.559 11.820
FunieGAN-up 0.768 26.200 3.029 0.557 8.259
本研究算法 0.932 30.610 3.196 0.570 7.212
本研究算法-无色彩校正 0.851 28.910 3.003 0.553 7.558
表 2  本研究算法和其他算法在EUVP数据集上的各项指标对比
算法 t/s 算法 t/s
IBLA 5.595 UWGAN 0.010
UDCP 2.411 FunieGAN 0.017
ULAP 0.380 FunieGAN-up 0.017
RGHS 1.005 本研究算法 0.025
Sea-thru 2.910 本研究算法-无色彩校正 0.025
表 3  各算法每张图像平均处理时间
1 STANKIEWICZ P, TAN Y T, KOBILAROV M Adaptive s-ampling with an autonomous underwater vehicle in static marine environments[J]. Journal of Field Robotics, 2021, 38 (4): 572- 597
doi: 10.1002/rob.22005
2 RUMSON A. Development of autonomous subsea pipeline inspection capabilities[C]// Global Oceans 2020: Singapore–US Gulf Coast. Biloxi: IEEE, 2020: 1-6.
3 CEJKA J, BRUNO F, SKARLATOS D, et al Detecting square markers in underwater environments[J]. Remote Sensing, 2019, 11 (4): 23
4 ANCUTI C, ANCUTI C O, HABER T, et al. Enhancing underwater images and videos by fusion[C]// 2012 IEEE Conference on Computer Vision and Pattern Recognition. Providence: IEEE, 2012: 81-88.
5 HUANG D, WANG Y, SONG W, et al. Shallow-water image enhancement using relative global histogram stretching based on adaptive parameter acquisition[C]// International Conference on Multimedia Modeling. Bangkok: Springer, 2018: 453-465.
6 DREWS P, NASCIMENTO E, MORAES F, et al. Transmission estimation in underwater single images[C]// Proceedings of the IEEE International Conference on Computer Vision Workshops. Sydney: IEEE, 2013: 825-830.
7 PENG Y, COSMAN P C Underwater image restoration based on image blurriness and light absorption[J]. IEEE Transactions on Image Processing, 2017, 26 (4): 1579- 1594
doi: 10.1109/TIP.2017.2663846
8 SONG W, WANG Y, HUANG D, et al. A rapid scene depth estimation model based on underwater light attenuation prior for underwater image restoration[C]// Pacific Rim Conference on Multimedia. Hefei: Springer, 2018: 678-688.
9 LI J, SKINNER K A, EUSTICE R M, et al WaterGAN: uns-upervised generative network to enable real-time color correction of monocular underwater images[J]. IEEE Robotics and Automation Letters, 2018, 3 (1): 387- 394
10 LI C Y, GUO J C, GUO C L Emerging from water: underwater image color correction based on weakly supervised color transfer[J]. IEEE Signal Processing Letters, 2018, 25 (3): 323- 327
doi: 10.1109/LSP.2018.2792050
11 FABBRI C, ISLAM M J, SATTAR J. Enhancing underwater imagery using generative adversarial networks[C]// 2018 IEEE International Conference on Robotics and Automation. Brisbane: IEEE, 2018: 7159-7165.
12 WANG N, ZHOU Y, HAN F, et al. UWGAN: underwater GAN for real-world underwater color restoration and dehazing [EB/OL]. (2019-12-21). https://arxiv.org/ftp/arxiv/papers/1912/1912.10269.pdf.
13 LIU P, ZHANG H, ZHANG K, et al. Multi-level wavelet-CNN for image restoration[C]// Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition Workshops. Salt Lake City: IEEE, 2018: 773-782.
14 JAFFE J S Computer modeling and the design of optimal underwater imaging systems[J]. IEEE Journal of Oceanic Engineering, 1990, 15 (2): 101- 111
doi: 10.1109/48.50695
15 AKKAYNAK D, TREIBITZ T. A revised underwater image formation model[C]// Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition. Salt Lake City: IEEE, 2018: 6723-6732.
16 AKKAYNAK D, TREIBITZ T, SHLESINGER T, et al. What is the space of attenuation coefficients in underwater computer vision?[C]// Proceedings of the IEEE Conference on Com-puter Vision and Pattern Recognition. Honolulu: IEEE, 2017: 4931-4940.
17 GALDRAN A, PARDO D, PICON A, et al Automatic red channel underwater image restoration[J]. Journal of Visual Communication and Image Representation, 2015, 26: 132- 145
doi: 10.1016/j.jvcir.2014.11.006
18 ZHAO X W, JIN T, QU S Deriving inherent optical properties from background color and underwater image enhancement[J]. Ocean Engineering, 2015, 94: 163- 172
doi: 10.1016/j.oceaneng.2014.11.036
19 AKKAYNAK D, TREIBITZ T. Sea-thru: a method for removing water from underwater images[C]// Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Reco-gnition. Long Beach: IEEE, 2019: 1682-1691.
20 GODARD C, MAC AODHA O, FIRMAN M, et al. Digging into self-supervised monocular depth estimation[C]// Proceedings of the IEEE/CVF International Conference on Computer Vision. Seoul: IEEE, 2019: 3828-3838.
21 WEI K, FU Y, YANG J, et al. A physics-based noise formation model for extreme low-light raw denoising [C]// Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition. Seattle: IEEE, 2020: 2758-2767.
22 RADFORD A, METZ L, CHINTALA S. Unsupervised representation learning with deep convolutional generative adversarial networks [EB/OL]. (2015-11-19). https://arxiv.org/pdf/1511.06434.pdf.
23 RONNEBERGER O, FISCHER P, BROX T. U-net: convolutional networks for biomedical image segmentation[C]// International Conference on Medical Image Computing and Computer-assisted Intervention. Munich: Springer, 2015: 234-241.
24 ODENA A, DUMOULIN V, OLAH C. Deconvolution and checkerboard artifacts [EB/OL]. [2021-03-01]. https://distill.pub/2016/deconv-checkerboard/.
25 GATYS L A, ECKER A S, BETHGE M. A neural algorithm of artistic style [EB/OL]. (2015-08-26). https://arxiv.org/pdf/1508.06576.pdf.
26 ISLAM M J, XIA Y, SATTAR J Fast underwater image enh-ancement for improved visual perception[J]. IEEE Robotics and Automation Letters, 2020, 5 (2): 3227- 3234
doi: 10.1109/LRA.2020.2974710
27 SILBERMAN N, FERGUS R. Indoor scene segmentation using a structured light sensor[C]// 2011 IEEE International Conference on Computer Vision Workshops. Barcelona: IEEE, 2011: 601-608.
28 SILBERMAN N, HOIEM D, KOHLI P, et al. Indoor segme-ntation and support inference from rgbd images[C]// European Conference on Computer Vision. Florence: Springer, 2012: 746-760.
29 URPC竞赛项目: 水下目标检测 [DS/OL]. [2019-8-8]. http://www.cnurpc.org/a/xwjrz/2019/0808/129.html.
30 YANG M, SOWMYA A An underwater color image quality evaluation metric[J]. IEEE Transactions on Image Processing, 2015, 24 (12): 6062- 6071
doi: 10.1109/TIP.2015.2491020
31 PANETTA K, GAO C, AGAIAN S Human-visual-system-inspired underwater image quality measures[J]. IEEE Journal of Oceanic Engineering, 2015, 41 (3): 541- 551
32 MITTAL A, SOUNDARARAJAN R, BOVIK A C Making a “completely blind” image quality analyzer[J]. IEEE Signal Processing Letters, 2013, 20 (3): 209- 212
doi: 10.1109/LSP.2012.2227726
[1] 何立,庞善民. 结合年龄监督和人脸先验的语音-人脸图像重建[J]. 浙江大学学报(工学版), 2022, 56(5): 1006-1016.
[2] 王云灏,孙铭会,辛毅,张博宣. 基于压电薄膜传感器的机器人触觉识别系统[J]. 浙江大学学报(工学版), 2022, 56(4): 702-710.
[3] 陈彤,郭剑锋,韩心中,谢学立,席建祥. 基于生成对抗模型的可见光-红外图像匹配方法[J]. 浙江大学学报(工学版), 2022, 56(1): 63-74.
[4] 任松,朱倩雯,涂歆玥,邓超,王小书. 基于深度学习的公路隧道衬砌病害识别方法[J]. 浙江大学学报(工学版), 2022, 56(1): 92-99.
[5] 郑英杰,吴松荣,韦若禹,涂振威,廖进,刘东. 基于目标图像FCM算法的地铁定位点匹配及误报排除方法[J]. 浙江大学学报(工学版), 2021, 55(3): 586-593.
[6] 徐铸业,赵小强,蒋红梅. 基于点分布模型的3D模型拟合方法[J]. 浙江大学学报(工学版), 2021, 55(12): 2373-2381.
[7] 刘近贞,叶方方,熊慧. 基于卷积神经网络的多类运动想象脑电信号识别[J]. 浙江大学学报(工学版), 2021, 55(11): 2054-2066.
[8] 陈巧红,陈翊,李文书,贾宇波. 多尺度SE-Xception服装图像分类[J]. 浙江大学学报(工学版), 2020, 54(9): 1727-1735.
[9] 周登文,田金月,马路遥,孙秀秀. 基于多级特征并联的轻量级图像语义分割[J]. 浙江大学学报(工学版), 2020, 54(8): 1516-1524.
[10] 张彦楠,黄小红,马严,丛群. 基于深度学习的录音文本分类方法[J]. 浙江大学学报(工学版), 2020, 54(7): 1264-1271.
[11] 闫旭,范晓亮,郑传潘,臧彧,王程,程明,陈龙彪. 基于图卷积神经网络的城市交通态势预测算法[J]. 浙江大学学报(工学版), 2020, 54(6): 1147-1155.
[12] 沈宗礼,余建波. 基于迁移学习与深度森林的晶圆图缺陷识别[J]. 浙江大学学报(工学版), 2020, 54(6): 1228-1239.
[13] 汪海晋,尹宗宇,柯臻铮,郭英杰,董辉跃. 基于一维卷积神经网络的螺旋铣刀具磨损监测[J]. 浙江大学学报(工学版), 2020, 54(5): 931-939.
[14] 杨萍,王丹,康子健,李童,付利华,余悦任. 基于模式识别和集成CNN-LSTM的阵发性房颤预测模型[J]. 浙江大学学报(工学版), 2020, 54(5): 1039-1048.
[15] 金列俊,詹建明,陈俊华,王涛. 基于一维卷积神经网络的钻杆故障诊断[J]. 浙江大学学报(工学版), 2020, 54(3): 467-474.