weakly supervised underwater image enhancement algorithm incorporating attention mechanism

doi:10.3785/j.issn.1008-973X.2021.03.016

Journal of ZheJiang University (Engineering Science)

2021, Vol. 55

Issue (3): 555-562 DOI: 10.3785/j.issn.1008-973X.2021.03.016

weakly supervised underwater image enhancement algorithm incorporating attention mechanism

Zi-ye YONG1(

),Ji-chang GUO1,*(

),Chong-yi LI2

1. School of Electrical Automation and Information Engineering, Tianjin University, Tianjin 300072, China
2. Department of Computer Science, City University of Hong Kong, Hong Kong 999077, China

Download:

HTML

PDF(1290KB) HTML
Export: BibTeX | EndNote (RIS)

Abstract

The supervised underwater image enhancement algorithms need paired training image samples that are difficult to be obtained in some uncontrolled scenarios such as underwater scenarios. A weakly supervised underwater image enhancement algorithm incorporating attention mechanism was proposed. Firstly, the red channel attenuation map was calculated according to the characteristics that the light with different wavelengths suffers from different attenuation when it propagates in water. After that, the attention module guided by the calculated red channel attenuation map was integrated into the generator, which effectively improved the performance of the generator in terms of correcting the color deviation of underwater images. In addition, a multiple joint loss function, including an adversarial loss and a structural similarity loss, was designed, which retained more image details while correcting color deviation of underwater images. Finally, the underwater image enhancement network was optimized under global and local scales. Experimental results show that the proposed algorithm is better than the competing algorithms in both subjective visual quality and objective evaluation index, and thus can effectively improve the visibility of underwater images.

Key words： underwater image image enhancement weakly supervised learning attention mechanism clarity

Received: 30 December 2019 Published: 25 April 2021

CLC:

TP 391

Fund: 国家自然科学基金资助项目（61771334）

Corresponding Authors: Ji-chang GUO E-mail: 18175288288@163.com;jcguo@tju.edu.cn

	Service
	E-mail this article
	Add to my bookshelf
	Add to citation manager
	E-mail Alert
	RSS
	Articles by authors
	Zi-ye YONG
	Ji-chang GUO
	Chong-yi LI

Cite this article:

Zi-ye YONG,Ji-chang GUO,Chong-yi LI. weakly supervised underwater image enhancement algorithm incorporating attention mechanism. Journal of ZheJiang University (Engineering Science), 2021, 55(3): 555-562.

URL:

http://www.zjujournals.com/eng/10.3785/j.issn.1008-973X.2021.03.016 OR http://www.zjujournals.com/eng/Y2021/V55/I3/555

融入注意力机制的弱监督水下图像增强算法

基于监督学习的水下图像增强算法中所需成对训练样本获得困难，为此提出一种融入注意力机制的弱监督水下图像增强算法. 根据不同波长的光在水中传播时衰减程度不同的物理特性，计算红通道衰减图，并将依赖红通道衰减图引导的注意力模块融入生成网络，提高生成网络修正水下图像色偏的性能；设计对抗损失函数和结构相似性损失函数相结合的多项联合损失函数，在修正水下图像色偏的同时保留更多图像细节；在全局和局部两个尺度下优化提出的弱监督水下图像增强网络模型. 实验结果表明，所提算法在主观视觉质量和客观评价指标上都优于比较算法，可以有效地提高水下图像清晰度.

关键词： 水下图像, 图像增强, 弱监督学习, 注意力机制, 清晰度

Fig.1 Generative adversarial network

Fig.2 Flowchart of weakly supervised underwater image enhancement algorithm incorporating attention mechanism

Fig.3 Network architecture of weakly supervised underwater image enhancement algorithm incorporating attention mechanism

Fig.4 Experimental results of proposed algorithm compared with classical algorithms

Fig.5 Experimental image details of proposed algorithm compared with classical algorithms

Tab.1 Comparison results of different objective indexes in each algorithm

Fig.6 Comparison results of ablation experiment

Tab.2 Comparison results of different objective indexes in ablation experiment


[1]	SCHETTINI R, CORCHS S Underwater image processing: state of the art of restoration and image enhancement methods[J]. EURASIP Journal on Advances in Signal Processing, 2010, 2010 (1): 1- 15

[2]	WANG J, SONG Z J, LI C Y, et al Research progress of underwater image enhancement methods[J]. Journal of Ocean Technology, 2016, 35 (2): 76- 82

[3]	ISHIKAWA N, SUGIURA S, HANZO L Subcarrier-index modulation aided OFDM—will it work?[J]. IEEE Access, 2016, 4: 2580- 2593 doi: 10.1109/ACCESS.2016.2568040

[4]	BIANCO N C, MOHAN A, EUSTICE R M. Initial results in underwater single image dehazing [C]// OCEANS 2010 MTS. Seattle: IEEE, 2010, 27(3): 1-8.

[5]	SINGH K, KAPOOR R, SINHA S K Enhancement of low exposure images via recursive histogram equalization algorithms[J]. Optik-International Journal for Light and Electron Optics, 2015, 126 (20): 2619- 2625 doi: 10.1016/j.ijleo.2015.06.060

[6]	ZHANG S, WANG T, DONG J Y, et al Underwater image enhancement via extended multi-scale Retinex[J]. Neurocomputing, 2017, 245: 1- 9 doi: 10.1016/j.neucom.2017.03.029

[7]	LI J, SKINNER K A, EUSTICE R M, et al WaterGAN: unsupervised generative network to enable real-time color correction of monocular underwater images[J]. IEEE Robotics and Automation Letters, 2018, 3 (1): 387- 394

[8]	CHEN X, YU J, KONG S, et al Towards real-time advancement of underwater visual quality with GAN[J]. IEEE Transactions on Industrial Electronics, 2019, 66 (12): 9350- 9359 doi: 10.1109/TIE.2019.2893840

[9]	ZHU J Y, PARK T, ISOLA P, et al. Unpaired image-to-image translation using cycle-consistent adversarial networks [C]// 2017 IEEE International Conference on Computer Vision (ICCV). Venice: IEEE, 2017: 2242-2251.

[10]	LI C, GUO J, GUO C Emerging from water: underwater image color correction based on weakly supervised color transfer[J]. IEEE Signal Processing Letters, 2018, 25 (3): 323- 327 doi: 10.1109/LSP.2018.2792050

[11]	DUNTLEY S Q Light in the sea[J]. Journal of the Optical Society of America, 1963, 53 (2): 214- 233 doi: 10.1364/JOSA.53.000214

[12]	LI C, GUO J Underwater image enhancement by dehazing and color correction[J]. Journal of Electronic Imaging, 2015, 24 (3): 1- 10

[13]	GOODFELLOW I, POUGET-ABADIE J, MIRZA M, et al. Generative adversarial nets [C]// 28th Conference on Neural Information Processing Systems. Montreal: NIPS, 2014: 2672-2680.

[14]	RONNEBERGER O, FISCHER P, BROX T. U-Net: convolutional networks for biomedical image segmentation [C]// International Conference on Medical Image Computing and Computer-Assisted Intervention. Miccai: MICCAI, 2015: 234-241.

[15]	LIU D, WEN B, LIU X, et al. When image denoising meets high-level vision tasks: a deep learning approach [C]// 27th International Joint Conference on Artificial Intelligence. Stockholm: IJCAI, 2018: 842-848.

[16]	JIANG Y, GONG X, LIU D, et al. EnlightenGAN: deep light enhancement without paired supervision [EB/OL]. [2019-06-17]. https://arxiv.org/abs/1906.06972.

[17]	ALEXIA J M. The relativistic discriminator: a key element missing from standard gan [EB/OL]. [2018-09-10]. https://arxiv.org/abs/1807.00734.

[18]	MAO X, LI Q, XIE H, et al. Least squares generative adversarial networks [C]// 2017 IEEE International Conference on Computer Vision (ICCV). Venice: IEEE, 2017, 4: 2813-2821.

[19]	ZHAO H, GALLO O, FROSIO I, et al Loss functions for image restoration with neural networks[J]. IEEE Transactions on Computational Imaging, 2017, 3 (1): 47- 57

[20]	LI C, GUO C, REN W, et al An underwater image enhancement benchmark dataset and beyond[J]. IEEE Transactions on Image Processing, 2020, 29 (1): 4376- 4389

[21]	FABBRI C, ISLAM M J, SATTAR J. Enhancing underwater imagery using generative adversarial networks [C]// 2018 IEEE International Conference on Robotics and Automation (ICRA). Brisbane: IEEE, 2018: 7159-7165.

[22]	ISLAM M J, XIA Y, SATTAR J Fast underwater image enhancement for improved visual perception[J]. IEEE Robotics and Automation Letters, 2020, 5 (2): 3227- 3234

[23]	PANETTA K, GAO C, AGAIAN S Human-visual-system-inspired underwater image quality measures[J]. IEEE Journal of Oceanic Engineering, 2016, 41 (3): 541- 551

[1]	Xun CHENG,Jian-bo YU. Monitoring method for machining tool wear based on machine vision[J]. Journal of ZheJiang University (Engineering Science), 2021, 55(5): 896-904.

[2]	Chuang LIU,Jun LIANG. Vehicle motion trajectory prediction based on attention mechanism[J]. Journal of ZheJiang University (Engineering Science), 2020, 54(6): 1156-1163.

[3]	Yan ZHANG,Bin GUO,Qian-ru WANG,Jing ZHANG,Zhi-wen YU. SeqRec: sequential-based recommendation model with long-term preference and instant interest[J]. Journal of ZheJiang University (Engineering Science), 2020, 54(6): 1177-1184.

[4]	Xiao-hu ZHAO,Liang-fei YIN,Cheng-long ZHAO. Image captioning based on global-local feature and adaptive-attention[J]. Journal of ZheJiang University (Engineering Science), 2020, 54(1): 126-134.

[5]	Wan-liang WANG,Xiao-han YANG,Yan-wei ZHAO,Nan GAO,Chuang LV,Zhao-juan ZHANG. Image enhancement algorithm with convolutional auto-encoder network[J]. Journal of ZheJiang University (Engineering Science), 2019, 53(9): 1728-1740.

[6]	GUO Bao-zhen, ZUO Wan-li, WANG Ying. Double CNN sentence classification model with attention mechanism of word embeddings[J]. Journal of ZheJiang University (Engineering Science), 2018, 52(9): 1729-1737.

[7]	HUANG Song, YI Ben-shun. Underwater image restoration algorithm based on adaptive transmission ratio[J]. Journal of ZheJiang University (Engineering Science), 2018, 52(1): 166-173.

[8]	LIU Tao, ZHAO Ju-feng, XU Zhi-hai, FENG Hua-jun,CHEN Hui-fang. Enhancement algorithm for infrared images based on Kalman filter[J]. Journal of ZheJiang University (Engineering Science), 2012, 46(8): 1534-1539.

[9]	ZHANG Zan-Chao, JIA Shun-Ren. Automatic urinary image processing[J]. Journal of ZheJiang University (Engineering Science), 2009, 43(5): 832-838.

Viewed

Full text

Abstract

Cited

Shared

Discussed