An end-to-end underwater image enhancement algorithm via fusing reverse medium transmission map was proposed to resolve the problem of quality degradation in the underwater images. The original RGB map and the reverse medium transmission map obtained by the traditional method were fed into two encoders of the proposed two-stream convolutional neural network. The information of two images was fully integrated through the cross-modality feature fusion module, and the network could learn the characteristics of underwater optical images well. The feature expression capabilities could be further strengthened through the feature enhancement modules. The decoder and encoder were connected via a residual decoding module to supply the RGB features. The underwater image enhancement algorithm via fusing reverse medium transmission map processes information from coarse to fine through cross-modality and cross-scale information fusion, and finally outputs an enhanced RGB image. The experimental results showed that the proposed algorithm could effectively improve the visual quality of underwater images. Taking into account both subjective evaluation and objective evaluation, the proposed algorithm was better than the six competing algorithms of white balance WB and histogram equalization HE, Water-Net, UGAN, UWCNN, Ucolor.
Fig.5Subjective visual comparison of each algorithm
算法
MSE ↓
PSNR↑
SSIM↑
Entropy↑
UCIQE↑
UIQM↑
原图
4 144.966 6
14.494 5
0.695 4
6.090 0
0.494 2
1.882 9
WB[14]
4 246.773 4
14.517 4
0.686 2
5.981 7
0.477 4
1.860 5
HE[15]
3 885.061 3
14.303 6
0.737 2
7.141 3
0.604 4
2.461 3
Water-Net[13]
3 417.155 3
15.761 2
0.768 7
6.787 9
0.540 5
2.409 3
UGAN[10]
2 266.487 8
16.263 6
0.662 5
7.121 4
0.559 7
3.020 2
UWCNN[12]
2 159.017 0
17.045 3
0.767 0
6.530 7
0.549 4
4.125 1
Ucolor[17]
723.786 5
23.239 3
0.871 8
7.260 6
0.513 2
2.575 7
URMT-Net
636.919 4
22.933 8
0.886 2
7.355 7
0.564 0
2.868 5
Tab.1Objective evaluation indexes of each algorithm on synthesized datasets
算法
MSE ↓
PSNR↑
SSIM↑
Entropy↑
UCIQE↑
UIQM↑
原图
1 324.763 9
18.258 0
0.815 0
6.745 0
0.504 4
2.840 7
WB[14]
1 455.735 0
17.926 1
0.804 1
6.761 9
0.542 9
2.873 1
HE[15]
1 164.861 4
18.863 8
0.874 9
7.817 3
0.664 0
3.620 7
Water-Net[13]
838.973 3
21.394 8
0.894 8
7.151 8
0.580 7
3.587 2
UGAN[10]
558.296 5
21.303 1
0.769 1
7.466 8
0.616 2
4.015 7
UWCNN[12]
2 804.303 1
15.089 1
0.710 8
6.309 1
0.483 7
2.983 2
Ucolor[17]
519.691 7
22.752 8
0.906 7
7.242 1
0.569 3
4.563 4
URMT-Net
489.336 1
22.305 0
0.902 1
7.484 7
0.599 0
3.786 2
Tab.2Objective evaluation indexes of each algorithm on real datasets
Fig.6Subjective visual comparison of ablation experiment
算法
MSE ↓
PSNR↑
SSIM↑
Entropy↑
UCIQE↑
UIQM↑
无特征融合模块
674.632 2
22.567 8
0.882 6
7.347 2
0.559 0
2.870 2
无特征增强模块
678.910 3
22.576 5
0.881 4
7.368 9
0.562 2
2.879 1
无残差解码模块
722.180 6
22.336 1
0.879 0
7.332 0
0.561 8
2.857 9
URMT-Net
636.919 4
22.933 8
0.886 2
7.355 7
0.564 0
2.868 5
Tab.3Objective evaluation indexes of ablation experiment on synthesized datasets
算法
MSE ↓
PSNR↑
SSIM↑
Entropy↑
UCIQE↑
UIQM↑
无特征融合模块
486.392 1
22.090 4
0.901 1
7.450 7
0.590 2
3.806 7
无特征增强模块
531.255 5
21.826 0
0.897 8
7.441 7
0.588 3
3.742 0
无残差解码模块
515.029 9
22.258 4
0.899 7
7.452 9
0.590 8
3.804 3
URMT-Net
489.336 1
22.305 0
0.902 1
7.484 7
0.599 0
3.786 2
Tab.4Objective evaluation indexes of ablation experiment on real datasets
[1]
IQBAL K, ODETAYO M, JAMES A, et al. Enhancing the low quality images using unsupervised colour correction method [C]// Proceedings of IEEE International Conference on Systems, Man and Cybernetics: IEEE, 2010: 1703-1709.
[2]
ANCUTI C, ANCUTI C O, HABER T. Enhancing underwater images and videos by fusion [C]// Proceedings of IEEE Conference on Computer Vision and Pattern Recognition (CVPR). Providence RI: IEEE, 2012: 81-88.
[3]
HITAM M S, KUALA T, YUSSOF W, et al. Mixture contrast limited adaptive histogram equalization for underwater image enhancement [C]// Proceedings of International Conference on Computer Applications Technology. Sousse, Tunisia, 2013: 1-5.
[4]
DREWSJR P, NASCIMENTO E R, BOTELHO S S C, et al Underwater depth estimation and image restoration based on singleimages[J]. IEEE Computer Graphics and Applications, 2016, 36 (2): 24- 35
doi: 10.1109/MCG.2016.26
[5]
LI C, GUO J, CHEN S, et al. Underwater image restoration based on minimum information loss principle and optical properties of underwater imaging [C]// IEEE International Conference on Image Processing (ICIP). Phoenix: IEEE, 2016: 1993-1997.
[6]
LI C, GUO J, CONG R, et al Underwater image enhancement by dehazing with minimum information loss and histogram distribution prior[J]. IEEE Transactions on Image Processing, 2016, 25 (12): 5664- 5677
doi: 10.1109/TIP.2016.2612882
[7]
AKKAYNAK D, TREIBITZ T. A revised underwater image formation model [C]// Proceedings of IEEE Conference on Computer Vision and Pattern Recognition (CVPR). Salt Lake City: IEEE, 2018: 6723-6732.
[8]
AKKAYNAK D, TREIBITZ T. Sea-Thru: a method for removing water from underwater images [C]// Proceedings of IEEE Conference on Computer Vision and Pattern Recognition (CVPR). Long Beach: IEEE, 2019: 1682-1691.
[9]
LI J, SKINNER K A, EUSTICE M, et al WaterGAN: unsupervised generative networkto enable real-time color correction of monocular underwater images[J]. IEEE Robotics and Automation Letters, 2018, 3 (1): 387- 394
[10]
FABBRI C, ISLAM M J, SATTAR J. Enhancing underwater imagery using generative adversarial networks [C]// IEEE International Conference on Robotics and Automation (ICRA). Brisbane: IEEE, 2018: 7159-7165.
[11]
ANWAR S, LI C, PORIKLI F. Deep underwater image enhancemen [EB/OL]. [2018-07-10]. https://arxiv.org/abs/1807.03528.
[12]
LI C, ANWAR S, PORIKLI F Underwater scene prior inspired deep underwater image and video enhancement[J]. Pattern Recognition, 2019, 98 (1): 107038
[13]
LI C, GUO C, REN W, et al An underwater image enhancement benchmark dataset and beyond[J]. IEEE Transactions on Image Processing, 2020, 29: 4376- 4389
doi: 10.1109/TIP.2019.2955241
[14]
LIU Y C, CHAN W H, CHEN Y Q Automatic white balance for digital still camera[J]. IEEE Transactions on Consumer Electronics, 1995, 41 (3): 460- 466
doi: 10.1109/30.468045
[15]
HUMMEL R Image enhancement by histogram transformation[J]. IEEE Computer Graphics and Image Processing, 1977, 6 (2): 184- 195
doi: 10.1016/S0146-664X(77)80011-7
[16]
WANG Y, GUO J, GAO H UIEC^2-Net: CNN-based underwater image enhancement using two color space[J]. Signal Processing: Image Communication, 2021, 96: 116250
doi: 10.1016/j.image.2021.116250
[17]
LI C, ANWAR S, HOU J, et al Underwater image enhancement via medium transmission-guided multi-color space embedding[J]. IEEE Transactions on Image Processing, 2021, 30: 4985- 5000
doi: 10.1109/TIP.2021.3076367
[18]
LU H, LI Y, ZHANG L Contrast enhancement for images in turbid water[J]. Journal of the Optical Society of America A, 2015, 32 (5): 886- 893
doi: 10.1364/JOSAA.32.000886
[19]
PENG Y T, K CAO, P C COSMAN Generalization of the dark channel prior for single image restoration[J]. IEEE Transactions on Image Processing, 2018, 27 (6): 2856- 2868
doi: 10.1109/TIP.2018.2813092
[20]
LI C, CONG R, PIAO Y, et al. RGB-D salient object detection with cross-modality modulation and selection [C]// European Conference on Computer Vision (ECCV). Cham: Springer, 2020: 225-241.
[21]
HE K, ZHANG X, REN S, et al. Deep residual learning for image recognition [C]// Proceedings of IEEE Conference on Computer Vision and Pattern Recognition (CVPR). Las Vegas: IEEE, 2016: 770-778.
[22]
ZHAO H, GALLO O, FROSIO I, et al Loss functions for image restoration with neural networks[J]. IEEE Transactions on Computational Imaging, 2016, 3 (1): 47- 57
[23]
JOHNSON J, ALAHI A, LI F F. Perceptual losses for real-timestyle transfer and super-resolution [C]// European Conference on Computer Vision (ECCV). Cham: Springer, 2016: 694-711.
[24]
LIU W, RABINOVICH A, BERG A C, Parsenet: looking wider to see better [EB/OL]. [2015-06-15]. https://arxiv.org/abs/1506.04579.
