|
|
Underwater image enhancement algorithm via fusing reverse medium transmission map |
Jian-zhao ZHANG( ),Ji-chang GUO*( ),Yu-dong WANG |
School of Electrical and Information Engineering, Tianjin University, Tianjin 300072, China |
|
|
Abstract An end-to-end underwater image enhancement algorithm via fusing reverse medium transmission map was proposed to resolve the problem of quality degradation in the underwater images. The original RGB map and the reverse medium transmission map obtained by the traditional method were fed into two encoders of the proposed two-stream convolutional neural network. The information of two images was fully integrated through the cross-modality feature fusion module, and the network could learn the characteristics of underwater optical images well. The feature expression capabilities could be further strengthened through the feature enhancement modules. The decoder and encoder were connected via a residual decoding module to supply the RGB features. The underwater image enhancement algorithm via fusing reverse medium transmission map processes information from coarse to fine through cross-modality and cross-scale information fusion, and finally outputs an enhanced RGB image. The experimental results showed that the proposed algorithm could effectively improve the visual quality of underwater images. Taking into account both subjective evaluation and objective evaluation, the proposed algorithm was better than the six competing algorithms of white balance WB and histogram equalization HE, Water-Net, UGAN, UWCNN, Ucolor.
|
Received: 23 May 2022
Published: 09 May 2023
|
|
Fund: 国家自然科学基金资助项目(62171315) |
Corresponding Authors:
Ji-chang GUO
E-mail: 2020234133@tju.edu.cn;jcguo@tju.edu.cn
|
基于融合逆透射率图的水下图像增强算法
针对水下图像质量退化严重的问题,提出一种端到端的基于融合逆透射率图的水下图像增强算法. 将原始RGB图像和基于传统方法得到的逆透射率图分别输入到双流卷积神经网络的2个编码器中;通过跨模态特征融合模块使得2种图像信息充分融合互补,让网络更好地学习到水下光学成像的特点;通过特征增强模块,增强特征的表达能力;通过残差解码模块连接解码器和编码器,以补充和丰富RGB特征. 通过逆透射率图的水下图像增强算法以及跨模态跨尺度的信息融合,由粗到细地进行逐级处理,最终输出增强后的RGB图像. 实验结果表明,所提算法能够有效地提升水下图像视觉质量. 综合主观评价和客观评价,所提算法优于所对比的白平衡WB、直方图均衡化HE、Water-Net、UGAN、UWCNN、Ucolor 6种算法.
关键词:
逆透射率图,
水下图像增强,
水下光学成像,
双流卷积神经网络,
特征融合
|
|
[1] |
IQBAL K, ODETAYO M, JAMES A, et al. Enhancing the low quality images using unsupervised colour correction method [C]// Proceedings of IEEE International Conference on Systems, Man and Cybernetics: IEEE, 2010: 1703-1709.
|
|
|
[2] |
ANCUTI C, ANCUTI C O, HABER T. Enhancing underwater images and videos by fusion [C]// Proceedings of IEEE Conference on Computer Vision and Pattern Recognition (CVPR). Providence RI: IEEE, 2012: 81-88.
|
|
|
[3] |
HITAM M S, KUALA T, YUSSOF W, et al. Mixture contrast limited adaptive histogram equalization for underwater image enhancement [C]// Proceedings of International Conference on Computer Applications Technology. Sousse, Tunisia, 2013: 1-5.
|
|
|
[4] |
DREWSJR P, NASCIMENTO E R, BOTELHO S S C, et al Underwater depth estimation and image restoration based on singleimages[J]. IEEE Computer Graphics and Applications, 2016, 36 (2): 24- 35
doi: 10.1109/MCG.2016.26
|
|
|
[5] |
LI C, GUO J, CHEN S, et al. Underwater image restoration based on minimum information loss principle and optical properties of underwater imaging [C]// IEEE International Conference on Image Processing (ICIP). Phoenix: IEEE, 2016: 1993-1997.
|
|
|
[6] |
LI C, GUO J, CONG R, et al Underwater image enhancement by dehazing with minimum information loss and histogram distribution prior[J]. IEEE Transactions on Image Processing, 2016, 25 (12): 5664- 5677
doi: 10.1109/TIP.2016.2612882
|
|
|
[7] |
AKKAYNAK D, TREIBITZ T. A revised underwater image formation model [C]// Proceedings of IEEE Conference on Computer Vision and Pattern Recognition (CVPR). Salt Lake City: IEEE, 2018: 6723-6732.
|
|
|
[8] |
AKKAYNAK D, TREIBITZ T. Sea-Thru: a method for removing water from underwater images [C]// Proceedings of IEEE Conference on Computer Vision and Pattern Recognition (CVPR). Long Beach: IEEE, 2019: 1682-1691.
|
|
|
[9] |
LI J, SKINNER K A, EUSTICE M, et al WaterGAN: unsupervised generative networkto enable real-time color correction of monocular underwater images[J]. IEEE Robotics and Automation Letters, 2018, 3 (1): 387- 394
|
|
|
[10] |
FABBRI C, ISLAM M J, SATTAR J. Enhancing underwater imagery using generative adversarial networks [C]// IEEE International Conference on Robotics and Automation (ICRA). Brisbane: IEEE, 2018: 7159-7165.
|
|
|
[11] |
ANWAR S, LI C, PORIKLI F. Deep underwater image enhancemen [EB/OL]. [2018-07-10]. https://arxiv.org/abs/1807.03528.
|
|
|
[12] |
LI C, ANWAR S, PORIKLI F Underwater scene prior inspired deep underwater image and video enhancement[J]. Pattern Recognition, 2019, 98 (1): 107038
|
|
|
[13] |
LI C, GUO C, REN W, et al An underwater image enhancement benchmark dataset and beyond[J]. IEEE Transactions on Image Processing, 2020, 29: 4376- 4389
doi: 10.1109/TIP.2019.2955241
|
|
|
[14] |
LIU Y C, CHAN W H, CHEN Y Q Automatic white balance for digital still camera[J]. IEEE Transactions on Consumer Electronics, 1995, 41 (3): 460- 466
doi: 10.1109/30.468045
|
|
|
[15] |
HUMMEL R Image enhancement by histogram transformation[J]. IEEE Computer Graphics and Image Processing, 1977, 6 (2): 184- 195
doi: 10.1016/S0146-664X(77)80011-7
|
|
|
[16] |
WANG Y, GUO J, GAO H UIEC^2-Net: CNN-based underwater image enhancement using two color space[J]. Signal Processing: Image Communication, 2021, 96: 116250
doi: 10.1016/j.image.2021.116250
|
|
|
[17] |
LI C, ANWAR S, HOU J, et al Underwater image enhancement via medium transmission-guided multi-color space embedding[J]. IEEE Transactions on Image Processing, 2021, 30: 4985- 5000
doi: 10.1109/TIP.2021.3076367
|
|
|
[18] |
LU H, LI Y, ZHANG L Contrast enhancement for images in turbid water[J]. Journal of the Optical Society of America A, 2015, 32 (5): 886- 893
doi: 10.1364/JOSAA.32.000886
|
|
|
[19] |
PENG Y T, K CAO, P C COSMAN Generalization of the dark channel prior for single image restoration[J]. IEEE Transactions on Image Processing, 2018, 27 (6): 2856- 2868
doi: 10.1109/TIP.2018.2813092
|
|
|
[20] |
LI C, CONG R, PIAO Y, et al. RGB-D salient object detection with cross-modality modulation and selection [C]// European Conference on Computer Vision (ECCV). Cham: Springer, 2020: 225-241.
|
|
|
[21] |
HE K, ZHANG X, REN S, et al. Deep residual learning for image recognition [C]// Proceedings of IEEE Conference on Computer Vision and Pattern Recognition (CVPR). Las Vegas: IEEE, 2016: 770-778.
|
|
|
[22] |
ZHAO H, GALLO O, FROSIO I, et al Loss functions for image restoration with neural networks[J]. IEEE Transactions on Computational Imaging, 2016, 3 (1): 47- 57
|
|
|
[23] |
JOHNSON J, ALAHI A, LI F F. Perceptual losses for real-timestyle transfer and super-resolution [C]// European Conference on Computer Vision (ECCV). Cham: Springer, 2016: 694-711.
|
|
|
[24] |
LIU W, RABINOVICH A, BERG A C, Parsenet: looking wider to see better [EB/OL]. [2015-06-15]. https://arxiv.org/abs/1506.04579.
|
|
|
[25] |
PANETTA K, GAO C, AGAIAN S Human-visual-system inspired underwater image quality measures[J]. IEEE Journal of Oceanic Engineering, 2016, 41 (3): 541- 551
doi: 10.1109/JOE.2015.2469915
|
|
|
|
Viewed |
|
|
|
Full text
|
|
|
|
|
Abstract
|
|
|
|
|
Cited |
|
|
|
|
|
Shared |
|
|
|
|
|
Discussed |
|
|
|
|