1. School of Electrical and Information Engineering, Tianjin University, Tianjin 300072, China 2. Shanghai Artificial Intelligence Laboratory, Shanghai 200232, China 3. School of Microelectronics and Communication Engineering, Chongqing University, Chongqing 401331, China
A large scale real scene stereo low-light image dataset SLL10K was proposed. There were 12 658 pairs of unreferenced stereo low-illumination images and 205 pairs of referenced stereo images contained in the dataset. The images in the SLL10K dataset cover a wealth of lighting, time, and scene. FCNet, a stereo low-light image enhancement network based on feature fusion and consistency loss was proposed. The feature fusion module was used to fully integrate intra-monocular and inter-stereo features, and the consistency loss function was used to maintain the consistency between images before and after enhancement. Experiments on the SLL10K dataset and darkening KITTI dataset show that the images with FCNet obtain better performance on low-light image enhancement and object detection than the monocular enhancement method.
Jia-wen LIAO,Yan-wei PANG,Jing NIE,Han-qing SUN,Jia-le CAO. Stereo low-light enhancement based on feature fusion and consistency loss. Journal of ZheJiang University (Engineering Science), 2023, 57(12): 2456-2466.
Fig.2Sample image pairs for training and testing sets in outdoor scene of SLL10K
Fig.3Sample of low-light image pairs and reference image image pairs in indoor scene of SLL10K
Fig.4Structure diagram of FCNet
Fig.5Structure diagram of stereo inter-intra-feature fusion module
测试集
SIIFF
SC损失
BRISQUE
NIQE
PIQE
LOE
PSNR/dB
SSIM
LPIPS
室外
×
×
25.684 8
2.832 7
32.569 0
807.7
—
—
—
√
×
23.546 3
2.687 0
30.476 3
696.3
—
—
—
√
√
23.095 1
2.641 1
28.111 5
679.2
—
—
—
室内
×
×
27.663 3
3.694 9
38.553 7
2 738.0
11.230 5
0.360 5
0.633 5
√
×
24.973 8
3.500 8
34.091 6
2 628.4
11.224 1
0.375 7
0.623 4
√
√
22.531 7
3.390 1
31.279 4
2 593.4
11.222 1
0.406 2
0.609 2
Tab.1Ablation experiment of FCNet’s different modules on two test sets
测试集
nm
BRISQUE
NIQE
PIQE
LOE
PSNR/dB
SSIM
LPIPS
室外
1
23.310 1
2.651 0
28.553 6
688.8
—
—
—
2
23.095 1
2.641 1
28.111 5
679.2
—
—
—
室内
1
22.656 7
3.414 7
32.062 5
2 609.4
11.119 0
0.363 7
0.639 8
2
22.531 7
3.390 1
31.279 4
2 593.4
11.222 1
0.406 2
0.609 2
Tab.2Ablation experiment of FCNet’s module number variation on two test sets
测试集
卷积类别
BRISQUE
NIQE
PIQE
LOE
PSNR/dB
SSIM
LPIPS
室外
3×3
23.732 9
3.253 5
31.648 9
753.7
—
—
—
1×9和9×1
23.095 1
2.641 1
28.111 5
679.2
—
—
—
室内
3×3
24.625 2
3.816 4
41.876 4
2 655.2
11.068 3
0.317 7
0.651 6
1×9和9×1
22.531 7
3.390 1
31.279 4
2 593.4
11.222 1
0.406 2
0.609 2
Tab.3Ablation experiment of convolution variation on two test sets
方法
左目
右目
BRISQUE
NIQE
PIQE
LOE
BRISQUE
NIQE
PIQE
LOE
RetinexNet[11]
24.640 7
4.257 3
36.454 5
1 741.5
24.420 5
4.270 1
36.276 1
1752.7
ISSR[24]
28.392 8
2.815 1
27.923 2
753.0
28.903 5
2.810 7
28.525 2
724.2
GLAD[25]
23.628 8
3.280 8
28.633 4
590.8
23.729 0
3.268 0
28.310 2
584.8
DVENet[16]
23.100 2
2.931 7
28.175 1
791.9
22.706 7
2.872 3
27.531 7
740.2
ZeroDCE++[17]
25.684 8
2.832 7
32.569 0
807.7
25.560 2
2.857 2
32.118 0
814.4
RUAS[26]
29.713 1
3.746 6
30.512 2
2 520.0
29.736 9
3.675 5
30.863 7
2 300.6
FCNet
23.095 1
2.641 1
28.111 5
679.2
22.689 8
2.632 3
27.374 5
669.9
Tab.4Indicators comparison of different image enhancement methods on SLL10K outdoor dataset
方法
左目
右目
BRISQUE
NIQE
PIQE
LOE
PSNR
SSIM
LPIPS
BRISQUE
NIQE
PIQE
LOE
PSNR
SSIM
LPIPS
RetinexNet[11]
34.042 9
5.592 2
49.155 6
3 241.0
11.640 2
0.222 1
0.812 2
33.562 7
5.596 0
48.305 5
3 102.3
11.211 8
0.238 0
0.798 2
ISSR[24]
23.900 7
2.751 8
29.193 1
2 599.6
8.858 8
0.258 8
0.674 5
24.626 0
2.950 9
28.848 0
2 577.8
8.199 8
0.261 9
0.660 3
GLAD[25]
23.139 6
3.673 4
42.574 7
2 547.2
12.875 1
0.229 0
0.666 0
25.854 5
3.581 8
41.646 0
2 515.4
12.174 0
0.247 5
0.648 3
DVENet[16]
23.075 7
3.405 7
32.115 1
2 595.8
9.026 0
0.246 0
0.656 9
22.189 2
3.391 6
30.236 3
2 543.2
8.595 1
0.248 9
0.643 7
ZeroDCE++[17]
27.663 3
3.694 9
38.553 7
2 738.0
11.230 5
0.360 5
0.725 4
27.262 2
3.649 7
38.050 0
2 671.1
10.410 9
0.363 1
0.713 0
RUAS[26]
24.693 4
3.361 7
35.548 8
2 671.3
9.864 7
0.380 4
0.710 2
24.375 1
3.305 0
33.077 0
2 581.7
9.234 6
0.369 1
0.693 1
FCNet
22.531 7
3.390 1
31.279 4
2 593.4
11.222 1
0.406 2
0.609 2
21.273 6
3.329 7
29.109 7
2 539.9
10.407 3
0.407 3
0.598 0
Tab.5Indicators comparison of different image enhancement methods on SLL10K indoor dataset
Fig.6Processing effects comparison of different image enhancement methods on four scene images
Fig.7Comparison of monocular and stereo enhancement of indoor images
%
方法
$ {\mathrm{A}\mathrm{P}}_{{\rm{2d}}} $
$ {\mathrm{A}\mathrm{P}}_{{\rm{ori}}} $
$ {\mathrm{A}\mathrm{P}}_{{\rm{bev}}} $
$ {\mathrm{A}\mathrm{P}}_{{\rm{3d}}} $
原始
88.86
86.90
64.54
50.49
低光照
70.48
68.14
48.16
34.43
RetinexNet[11]
69.70
67.71
48.99
36.21
ISSR[23]
70.73
68.12
46.20
33.45
GLAD[24]
88.30
86.12
57.23
45.05
ZeroDCE++[17]
88.23
86.03
57.12
44.98
RUAS[25]
87.81
84.24
55.27
42.32
FCNet
88.51
86.55
57.72
45.45
Tab.6Object detection results of different image enhancement methods
Fig.8Comparison of parallax map between monocular and stereo image enhancement methods
[1]
WANG Y, WAN R, YANG W, et al. Low-light image enhancement with normalizing flow [C]// Proceedings of the AAAI Conference on Artificial Intelligence. [S.l.]: AAAI, 2022: 2604-2612.
[2]
LV F, LI Y, LU F Attention guided low-light image enhancement with a large scale low-light simulation dataset[J]. International Journal of Computer Vision, 2021, 129: 2175- 2193
doi: 10.1007/s11263-021-01466-8
[3]
WANG L, LIU Z, SIU W, et al Lightening network for low-light image enhancement[J]. IEEE Transactions on Image Processing, 2020, 29: 7984- 7996
doi: 10.1109/TIP.2020.3008396
[4]
ZHOU S, LI C, LOY C C. LEDNet: joint low-light enhancement and deblurring in the dark [C]// Proceedings of the European Conference on Computer Vision. [S.l.]: Springer, 2022: 573-589.
[5]
ZHENG C, SHI D, SHI W. Adaptive unfolding total variation network for low-light image enhancement [C]// Proceedings of the IEEE/CVF International Conference on Computer Vision. Montreal: IEEE, 2021: 4439-4448.
[6]
WU W, WENG J, ZHANG P, et al. URetinex-Net: retinex-based deep unfolding network for low-light image enhancement [C]// Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition. New Orleans: IEEE, 2022: 5901-5910.
[7]
HAO S, HAN X, GUO Y, et al Low-light image enhancement with semi-decoupled decomposition[J]. IEEE Transactions on Multimedia, 2020, 22 (12): 3025- 3038
doi: 10.1109/TMM.2020.2969790
[8]
GUO C, LI C, GUO J, et al. Zero-reference deep curve estimation for low-light image enhancement [C]// Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition. Seattle: IEEE, 2020: 1777–1786.
[9]
MA L, MA T, LIU R, et al. Toward fast, flexible, and robust low-light image enhancement [C]// Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition. New Orleans: IEEE, 2022: 5627-5636.
[10]
ZHANG Z, ZHENG H, HONG R, et al. Deep color consistent network for low-light image enhancement [C]// Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition. New Orleans: IEEE, 2022: 1889-1898.
[11]
WEI C, WANG W, YANG W, et al. Deep retinex decomposition for low-light enhancement [C]// Proceedings of the 29th British Machine Vision Conference. Newcastle Upon Tyne: [s.n.], 2018.
[12]
CHEN C, CHEN Q, XU J, et al. Learning to see in the dark [C]// Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition. Salt Lake City: IEEE, 2018: 3291-3300.
[13]
PANG Y, NIE J, XIE J, et al. BidNet: binocular image dehazing without explicit disparity estimation [C]// Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition. Seattle: IEEE, 2020: 5931-5940.
[14]
ZHOU S, ZHANG J, ZUO W, et al. DAVANet: stereo deblurring with view aggregation [C]// Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition. Long Beach: IEEE, 2019: 10996-11005.
[15]
ZHANG K, LUO W, REN W, et al. Beyond monocular deraining: stereo image deraining via semantic understanding [C]// Proceedings of the European Conference on Computer Vision. [S.l.]: Springer, 2020: 71-89.
[16]
HUANG J, FU X, XIAO Z, et al Low-light stereo image enhancement[J]. IEEE Transactions on Multimedia, 2022, 25: 2978- 2992
[17]
LI C, GUO C, LOY C C Learning to enhance low-light image via zero-reference deep curve estimation[J]. IEEE Transactions on Pattern Analysis and Machine Intelligence, 2022, 44 (8): 4225- 4238
[18]
WANG L, WANG Y, LIANG Z, et al. Learning parallax attention for stereo image super-resolution [C]// Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition. Long Beach: IEEE, 2019: 12250-12259.
[19]
MITTAL A, MOORTHY A K, BOVIK A C No-reference image quality assessment in the spatial domain[J]. IEEE Transactions on Image Processing, 2012, 21 (12): 4695- 4708
doi: 10.1109/TIP.2012.2214050
[20]
MITTAL A, SOUNDARARAJAN R, BOVIK A C Making a “completely blind” image quality analyzer[J]. IEEE Signal Processing Letters, 2012, 20 (3): 209- 212
[21]
VENKATANATH N, PRANEETH D, BH M C, et al. Blind image quality evaluation using perception based features [C]// Proceedings of the Twenty First National Conference on Communications. Mumbai: IEEE, 2015: 1-6.
[22]
WANG S, ZHENG J, HU H, et al Naturalness preserved enhancement algorithm for non-uniform illumination images[J]. IEEE Transactions on Image Processing, 2013, 22 (9): 3538- 3548
doi: 10.1109/TIP.2013.2261309
[23]
ZHANG R, ISOLA P, EFROS A A, et al. The unreasonable effectiveness of deep features as a perceptual metric [C]// Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition. Salt Lake City: IEEE, 2018: 586-595.
[24]
FAN M, WANG W, YANG W, et al. Integrating semantic segmentation and retinex model for low-light image enhancement [C]// Proceedings of the 28th ACM International Conference on Multimedia. [S.l.]: ACM, 2020: 2317-2325.
[25]
WANG W, WEI C, YANG W, et al. GLADNet: low-light enhancement network with global awareness [C]// Proceedings of the IEEE International Conference on Automatic Face and Gesture Recognition. Xi’an: IEEE, 2018: 751-755.
[26]
LIU R, MA L, ZHANG J, et al. Retinex-inspired unrolling with cooperative prior architecture search for low-light image enhancement [C]// Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition. Nashville: IEEE, 2021: 10561-10570.
[27]
GEIGER A, LENZ P, URTASUN R. Are we ready for autonomous driving? The KITTI vision benchmark suite [C]// Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition. Providence: IEEE, 2012: 3354-3361.
[28]
彭国福, 林正浩 图像处理中Gamma校正的研究和实现[J]. 电子工程师, 2006, 32 (2): 30- 32 PENG Guo-fu, LIN Zheng-hao A study on Gamma correction and its implementation in image processing[J]. Electronic Engineer, 2006, 32 (2): 30- 32
doi: 10.3969/j.issn.1674-4888.2006.02.008
