Please wait a minute...
浙江大学学报(工学版)
浙江大学学报(工学版)  2021, Vol. 55 Issue (3): 555-562    DOI: 10.3785/j.issn.1008-973X.2021.03.016
计算机与控制工程     
融入注意力机制的弱监督水下图像增强算法
雍子叶1(),郭继昌1,*(),李重仪2
1. 天津大学 电气自动化与信息工程学院，天津 300072
2. 香港城市大学 电脑科学学院，香港 999077
weakly supervised underwater image enhancement algorithm incorporating attention mechanism
Zi-ye YONG1(),Ji-chang GUO1,*(),Chong-yi LI2
1. School of Electrical Automation and Information Engineering, Tianjin University, Tianjin 300072, China
2. Department of Computer Science, City University of Hong Kong, Hong Kong 999077, China
 全文: PDF(1290 KB)   HTML
摘要：

基于监督学习的水下图像增强算法中所需成对训练样本获得困难，为此提出一种融入注意力机制的弱监督水下图像增强算法. 根据不同波长的光在水中传播时衰减程度不同的物理特性，计算红通道衰减图，并将依赖红通道衰减图引导的注意力模块融入生成网络，提高生成网络修正水下图像色偏的性能；设计对抗损失函数和结构相似性损失函数相结合的多项联合损失函数，在修正水下图像色偏的同时保留更多图像细节；在全局和局部两个尺度下优化提出的弱监督水下图像增强网络模型. 实验结果表明，所提算法在主观视觉质量和客观评价指标上都优于比较算法，可以有效地提高水下图像清晰度.
关键词： 水下图像;  图像增强;  弱监督学习;  注意力机制;  清晰度    
Abstract:

The supervised underwater image enhancement algorithms need paired training image samples that are difficult to be obtained in some uncontrolled scenarios such as underwater scenarios. A weakly supervised underwater image enhancement algorithm incorporating attention mechanism was proposed. Firstly, the red channel attenuation map was calculated according to the characteristics that the light with different wavelengths suffers from different attenuation when it propagates in water. After that, the attention module guided by the calculated red channel attenuation map was integrated into the generator, which effectively improved the performance of the generator in terms of correcting the color deviation of underwater images. In addition, a multiple joint loss function, including an adversarial loss and a structural similarity loss, was designed, which retained more image details while correcting color deviation of underwater images. Finally, the underwater image enhancement network was optimized under global and local scales. Experimental results show that the proposed algorithm is better than the competing algorithms in both subjective visual quality and objective evaluation index, and thus can effectively improve the visibility of underwater images.
Key words: underwater image    image enhancement    weakly supervised learning    attention mechanism    clarity
收稿日期: 2019-12-30 出版日期: 2021-04-25
CLC:  TP 391  
基金资助: 国家自然科学基金资助项目（61771334）
通讯作者: 郭继昌     E-mail: 18175288288@163.com;jcguo@tju.edu.cn
作者简介: 雍子叶（1996—），女，硕士生，从事水下图像清晰化研究. orcid.org/0000-0002-8186-3873. E-mail： 18175288288@163.com
服务　　
把本文推荐给朋友
加入引用管理器
E-mail Alert
作者相关文章　　
雍子叶
郭继昌
李重仪

引用本文:

雍子叶,郭继昌,李重仪. 融入注意力机制的弱监督水下图像增强算法[J]. 浙江大学学报(工学版), 2021, 55(3): 555-562.

Zi-ye YONG,Ji-chang GUO,Chong-yi LI. weakly supervised underwater image enhancement algorithm incorporating attention mechanism. Journal of ZheJiang University (Engineering Science), 2021, 55(3): 555-562.

链接本文:

http://www.zjujournals.com/eng/CN/10.3785/j.issn.1008-973X.2021.03.016    或    http://www.zjujournals.com/eng/CN/Y2021/V55/I3/555

图 1  生成对抗网络模型
图 2  融入注意力机制的弱监督水下图像增强算法流程
图 3  融入注意力机制的弱监督水下图像增强算法核心网络结构
图 4  本文算法与经典算法实验结果对比
图 5  本文算法与经典算法实验图像细节对比
算法 UIQM UIConM UCIQE Entropy
文献[12] 5.109 5 0.797 1 0.638 5 7.472 1
文献[9] 5.259 6 0.855 5 0.579 9 7.043 4
文献[10] 5.427 1 0.916 5 0.541 0 7.473 6
文献[21] 5.455 0 0.907 9 0.564 5 7.557 7
本文 5.517 7 0.926 9 0.583 9 7.608 9
表 1  各算法不同客观指标对比结果
图 6  消融实验对比结果
机制 UIQM UIConM UCIQE UICM Entropy
融入注意力 5.517 7 0.926 9 0.583 9 2.838 2 7.608 9
未融入注意力 5.060 9 0.847 8 0.516 4 3.020 9 7.271 7
表 2  消融实验不同客观指标对比结果
1 SCHETTINI R, CORCHS S Underwater image processing: state of the art of restoration and image enhancement methods[J]. EURASIP Journal on Advances in Signal Processing, 2010, 2010 (1): 1- 15
2 WANG J, SONG Z J, LI C Y, et al Research progress of underwater image enhancement methods[J]. Journal of Ocean Technology, 2016, 35 (2): 76- 82
3 ISHIKAWA N, SUGIURA S, HANZO L Subcarrier-index modulation aided OFDM—will it work?[J]. IEEE Access, 2016, 4: 2580- 2593
doi: 10.1109/ACCESS.2016.2568040
4 BIANCO N C, MOHAN A, EUSTICE R M. Initial results in underwater single image dehazing [C]// OCEANS 2010 MTS. Seattle: IEEE, 2010, 27(3): 1-8.
5 SINGH K, KAPOOR R, SINHA S K Enhancement of low exposure images via recursive histogram equalization algorithms[J]. Optik-International Journal for Light and Electron Optics, 2015, 126 (20): 2619- 2625
doi: 10.1016/j.ijleo.2015.06.060
6 ZHANG S, WANG T, DONG J Y, et al Underwater image enhancement via extended multi-scale Retinex[J]. Neurocomputing, 2017, 245: 1- 9
doi: 10.1016/j.neucom.2017.03.029
7 LI J, SKINNER K A, EUSTICE R M, et al WaterGAN: unsupervised generative network to enable real-time color correction of monocular underwater images[J]. IEEE Robotics and Automation Letters, 2018, 3 (1): 387- 394
8 CHEN X, YU J, KONG S, et al Towards real-time advancement of underwater visual quality with GAN[J]. IEEE Transactions on Industrial Electronics, 2019, 66 (12): 9350- 9359
doi: 10.1109/TIE.2019.2893840
9 ZHU J Y, PARK T, ISOLA P, et al. Unpaired image-to-image translation using cycle-consistent adversarial networks [C]// 2017 IEEE International Conference on Computer Vision (ICCV). Venice: IEEE, 2017: 2242-2251.
10 LI C, GUO J, GUO C Emerging from water: underwater image color correction based on weakly supervised color transfer[J]. IEEE Signal Processing Letters, 2018, 25 (3): 323- 327
doi: 10.1109/LSP.2018.2792050
11 DUNTLEY S Q Light in the sea[J]. Journal of the Optical Society of America, 1963, 53 (2): 214- 233
doi: 10.1364/JOSA.53.000214
12 LI C, GUO J Underwater image enhancement by dehazing and color correction[J]. Journal of Electronic Imaging, 2015, 24 (3): 1- 10
13 GOODFELLOW I, POUGET-ABADIE J, MIRZA M, et al. Generative adversarial nets [C]// 28th Conference on Neural Information Processing Systems. Montreal: NIPS, 2014: 2672-2680.
14 RONNEBERGER O, FISCHER P, BROX T. U-Net: convolutional networks for biomedical image segmentation [C]// International Conference on Medical Image Computing and Computer-Assisted Intervention. Miccai: MICCAI, 2015: 234-241.
15 LIU D, WEN B, LIU X, et al. When image denoising meets high-level vision tasks: a deep learning approach [C]// 27th International Joint Conference on Artificial Intelligence. Stockholm: IJCAI, 2018: 842-848.
16 JIANG Y, GONG X, LIU D, et al. EnlightenGAN: deep light enhancement without paired supervision [EB/OL]. [2019-06-17]. https://arxiv.org/abs/1906.06972.
17 ALEXIA J M. The relativistic discriminator: a key element missing from standard gan [EB/OL]. [2018-09-10]. https://arxiv.org/abs/1807.00734.
18 MAO X, LI Q, XIE H, et al. Least squares generative adversarial networks [C]// 2017 IEEE International Conference on Computer Vision (ICCV). Venice: IEEE, 2017, 4: 2813-2821.
19 ZHAO H, GALLO O, FROSIO I, et al Loss functions for image restoration with neural networks[J]. IEEE Transactions on Computational Imaging, 2017, 3 (1): 47- 57
20 LI C, GUO C, REN W, et al An underwater image enhancement benchmark dataset and beyond[J]. IEEE Transactions on Image Processing, 2020, 29 (1): 4376- 4389
21 FABBRI C, ISLAM M J, SATTAR J. Enhancing underwater imagery using generative adversarial networks [C]// 2018 IEEE International Conference on Robotics and Automation (ICRA). Brisbane: IEEE, 2018: 7159-7165.
22 ISLAM M J, XIA Y, SATTAR J Fast underwater image enhancement for improved visual perception[J]. IEEE Robotics and Automation Letters, 2020, 5 (2): 3227- 3234
23 PANETTA K, GAO C, AGAIAN S Human-visual-system-inspired underwater image quality measures[J]. IEEE Journal of Oceanic Engineering, 2016, 41 (3): 541- 551
[1] 程训,余建波. 基于机器视觉的加工刀具磨损监测方法[J]. 浙江大学学报(工学版), 2021, 55(5): 896-904.
[2] 刘创,梁军. 基于注意力机制的车辆运动轨迹预测[J]. 浙江大学学报(工学版), 2020, 54(6): 1156-1163.
[3] 张岩,郭斌,王倩茹,张靖,於志文. SeqRec：基于长期偏好和即时兴趣的序列推荐模型[J]. 浙江大学学报(工学版), 2020, 54(6): 1177-1184.
[4] 赵小虎,尹良飞,赵成龙. 基于全局?局部特征和自适应注意力机制的图像语义描述算法[J]. 浙江大学学报(工学版), 2020, 54(1): 126-134.
[5] 王万良,杨小涵,赵燕伟,高楠,吕闯,张兆娟. 采用卷积自编码器网络的图像增强算法[J]. 浙江大学学报(工学版), 2019, 53(9): 1728-1740.
[6] 董月,冯华君,徐之海,陈跃庭,李奇. Attention Res-Unet: 一种高效阴影检测算法[J]. 浙江大学学报(工学版), 2019, 53(2): 373-381.
[7] 郭宝震, 左万利, 王英. 采用词向量注意力机制的双路卷积神经网络句子分类模型[J]. 浙江大学学报(工学版), 2018, 52(9): 1729-1737.
[8] 黄松, 易本顺. 基于自适应透射率比的水下图像复原算法[J]. 浙江大学学报(工学版), 2018, 52(1): 166-173.
[9] 张天煜, 冯华君, 徐之海, 李奇, 陈跃庭. 基于强边缘宽度直方图的图像清晰度指标[J]. J4, 2014, 48(2): 312-320.
[10] 刘涛, 赵巨峰, 徐之海, 冯华君, 陈慧芳. 基于卡尔曼滤波的红外图像增强算法[J]. J4, 2012, 46(8): 1534-1539.
[11] 张红虎,谢辉. 评价室内汉语清晰度中STI计权方法适用性检验[J]. J4, 2012, 46(3): 463-469.
[12] 张红虎, 张三明. 镜面与扩散反射界面球形空间语言清晰度比较[J]. J4, 2010, 44(4): 812-818.
[13] 张赞超, 夏顺仁. 全自动尿液图像识别技术[J]. J4, 2009, 43(5): 832-838.
[14] 喻琪 夏顺仁 丛卫华 张峰山 杨晓帆.  基于小波系数相关性和模糊理论的声纳图像处理[J]. J4, 2008, 42(12): 2151-2155.
[15] 张赞超 夏顺仁 段会龙. 基于细胞神经网络的尿沉渣图像分割[J]. J4, 2008, 42(12): 2139-2144.