The tone mapping operators (TMO) aims at reproducing the visual perception of high dynamic range (HDR) images on the low dynamic range media. A histogram based tone mapping algorithm was proposed to use image segmentation and fusion. The HDR images were segmented into non-overlapping rectangle blocks. In each block, the tone mapping problem was regarded as the K-means clustering problem based on the image histogram with a proposed loss function. The derived mapping function was adaptively adjusted considering the uniformity of the block. The detail-enhancing effect of the mapping function was then restrained by the block uniformity for less arising artefacts. An image fusion scheme was presented to deal with the artefacts on the segmentation boundaries. With the form of a bilateral filter, the fusion scheme took into account both the difference of position and luminance, ensuring the naturalness of the tone-mapped results. The proposed algorithm was evaluated to use the tone-mapped image quality index and the results illustrated a better image quality compared with the typical tone mapping methods. A balance between the detail enhancement and the global appearance preserving has also been achieved with?an?impressed?execution?efficiency?among?local?TMOs.
Hao JIANG,Hai-song XU. Histogram based tone mapping algorithm using image segmentation and fusion. Journal of ZheJiang University (Engineering Science), 2022, 56(11): 2224-2231.
Fig.1Flowchart of histogram based tone mapping algorithm using image segmentation and fusion
Fig.2Tone mapped results of different curves for same block
Fig.3Fusion results corresponding to different σd values
Fig.4Fusion results corresponding to different σs values
图像序号
Q
k均值聚类方法
本研究算法
1
0.67
0.88
2
0.53
0.82
3
0.65
0.87
4
0.65
0.87
5
0.76
0.98
6
0.70
0.90
7
0.76
0.95
8
0.75
0.97
9
0.59
0.84
均值
0.67
0.90
Tab.1TMQI scores of proposed algorithm with and without image segmentation
Fig.5Tone mapped results of proposed algorithm with and without image segmentation
图像
Q
Drago et al.[4]
Kimet al.[22]
Reinhard et al.[23]
Khan et al.[8]
Krawcyzk et al.[24]
Shan et al.[25]
Liang et al.[11]
Liet al.[26]
本研究算法
1
0.76
0.72
0.72
0.81
0.73
0.73
0.81
0.91
0.88
2
0.70
0.68
0.65
0.79
0.68
0.60
0.75
0.89
0.82
3
0.77
0.78
0.73
0.88
0.74
0.75
0.82
0.87
0.87
4
0.76
0.78
0.73
0.85
0.71
0.74
0.80
0.85
0.87
5
0.88
0.84
0.79
0.88
0.85
0.88
0.86
0.73
0.98
6
0.85
0.84
0.73
0.88
0.77
0.76
0.86
0.81
0.90
7
0.91
0.88
0.79
0.83
0.88
0.87
0.84
0.71
0.95
8
0.86
0.87
0.78
0.87
0.82
0.84
0.85
0.72
0.97
9
0.72
0.82
0.71
0.79
0.70
0.72
0.83
0.80
0.84
均值
0.80
0.80
0.74
0.84
0.76
0.77
0.82
0.81
0.90
Tab.2TMQI scores of different algorithms
图像
S
Drago et al.[4]
Kimet al.[22]
Reinhard et al.[23]
Khan et al.[8]
Krawcyzk et al.[24]
Shan et al.[25]
Liang et al.[11]
Liet al.[26]
本研究算法
1
0.78
0.70
0.69
0.70
0.73
0.72
0.66
0.73
0.87
2
0.61
0.55
0.49
0.70
0.55
0.37
0.57
0.68
0.74
3
0.79
0.81
0.73
0.75
0.74
0.77
0.64
0.72
0.83
4
0.77
0.79
0.71
0.69
0.64
0.74
0.62
0.71
0.82
5
0.87
0.88
0.85
0.73
0.90
0.88
0.70
0.69
0.93
6
0.70
0.71
0.65
0.73
0.64
0.67
0.59
0.67
0.78
7
0.87
0.87
0.86
0.73
0.88
0.88
0.67
0.66
0.89
8
0.86
0.87
0.84
0.75
0.87
0.87
0.69
0.70
0.92
9
0.63
0.74
0.61
0.66
0.50
0.63
0.56
0.61
0.74
均值
0.76
0.77
0.72
0.72
0.72
0.73
0.63
0.69
0.83
Tab.3Structure-fidelity scores of different algorithms
图像
N
Drago et al.[4]
Kimet al.[22]
Reinhard et al.[23]
Khan et al.[8]
Krawcyzk et al.[24]
Shan et al.[25]
Liang et al.[11]
Liet al.[26]
本研究算法
1
0.05
0.01
0.00
0.33
0.01
0.01
0.41
0.85
0.44
2
0.01
0.01
0.00
0.22
0.02
0.00
0.26
0.86
0.33
3
0.05
0.08
0.01
0.62
0.00
0.01
0.48
0.64
0.45
4
0.04
0.07
0.00
0.56
0.02
0.01
0.45
0.52
0.44
5
0.46
0.23
0.06
0.66
0.24
0.41
0.60
0.03
0.97
6
0.55
0.45
0.05
0.72
0.22
0.15
0.84
0.38
0.74
7
0.61
0.44
0.06
0.36
0.41
0.38
0.52
0.00
0.85
8
0.36
0.38
0.05
0.60
0.17
0.25
0.57
0.01
0.93
9
0.04
0.34
0.03
0.30
0.13
0.06
0.74
0.45
0.45
均值
0.24
0.22
0.03
0.49
0.14
0.14
0.54
0.42
0.62
Tab.4Naturalness scores of different algorithms
Fig.6Examples for comparison of tone mapping algorithms
算法
平均运行时间/s
Drago et al.[4]
5.61
Kimet al.[22]
2.08
Reinhard et al.[23]
2.73
Khan et al.[8]
8.95
Krawcyzk et al.[24]
11.90
Shan et al.[25]
169.01
Liang et al.[11]
68.07
Liet al.[26]
183.38
本研究算法
15.64
Tab.5Average operation time of compared algorithms
[1]
REINHARD E, HEIDRICH W, DEBEVEC P, et al. High dynamic range imaging: acquisition, display, and image-based lighting [M]. Morgan: Kaufmann, 2010: 1-18.
[2]
方竞宇. 高动态范围彩色图像捕获与显示方法及技术研究[D]. 杭州: 浙江大学, 2016: 78-80. FANG Jing-yu. Study on methodology and technology of acquisition and rendering for high dynamic range color images [D]. Hangzhou: Zhejiang University, 2016: 78-80.
[3]
WARD G A contrast-based scale factor for luminance display[J]. Graphics Gems, 1994, 4: 415- 421
[4]
DRAGO F, MYSZKOWSKI K, ANNEN T, et al. Adaptive logarithmic mapping for displaying high contrast scenes [C]// Computer Graphics Forum. Oxford, UK: Blackwell Publishing, Inc, 2003: 419-426.
[5]
TUMBLIN J, RUSHMEIER H Tone reproduction for realistic images[J]. IEEE Computer Graphics and Applications, 1993, 13 (6): 42- 48
doi: 10.1109/38.252554
[6]
SCHLICK C. Quantization techniques for visualization of high dynamic range pictures, photorealistic rendering techniques [M]. Berlin, Heidelberg: Springer, 1995: 7-20.
[7]
MA K, YEGANEH H, ZENG K, et al High dynamic range image compression by optimizing tone mapped image quality index[J]. IEEE Transactions on Image Processing, 2015, 24 (10): 3086- 3097
doi: 10.1109/TIP.2015.2436340
[8]
KHAN I R, AZIZ W, SHIM S O Tone-mapping using perceptual-quantizer and image histogram[J]. IEEE Access, 2020, 8: 31350- 31358
doi: 10.1109/ACCESS.2020.2973273
[9]
AMBALATHANKANDY P, IKEBE M, YOSHIDA T, et al An adaptive global and local tone mapping algorithm implemented on FPGA[J]. IEEE Transactions on Circuits and Systems for Video Technology, 2019, 30 (9): 3015- 3028
[10]
ZHANG X S, YANG K F, ZHOU J, et al Retina inspired tone mapping method for high dynamic range images[J]. Optics Express, 2020, 28 (5): 5953- 5964
doi: 10.1364/OE.380555
[11]
LIANG Z, XU J, ZHANG D, et al. A hybrid l1-l0 layer decomposition model for tone mapping [C]// Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition. Salt Lake City, Utah, 2018: 4758-4766.
[12]
FAHIM M A N I, JUNG H Y Fast single-image HDR tone-mapping by avoiding base layer extraction[J]. Multidisciplinary Digital Publishing Institute, 2020, 20 (16): 4378
[13]
NGUYEN N H, VAN V T, LEE C Human visual system model-based optimized tone mapping of high dynamic range images[J]. IEEE Access, 2021, 9: 127343- 127355
doi: 10.1109/ACCESS.2021.3112046
[14]
DUAN J, BRESSAN M, DANCE C, et al Tone-mapping high dynamic range images by novel histogram adjustment[J]. Pattern Recognition, 2010, 43 (5): 1847- 1862
doi: 10.1016/j.patcog.2009.12.006
[15]
OSKARSSON M Temporally consistent tone mapping of images and video using optimal k-means clustering[J]. Journal of Mathematical Imaging and Vision, 2017, 57 (2): 225- 238
doi: 10.1007/s10851-016-0677-1
[16]
LARSON G W, RUSHMEIER H, PIATKO C A visibility matching tone reproduction operator for high dynamic range scenes[J]. IEEE Transactions on Visualization and Computer Graphics, 1997, 3 (4): 291- 306
doi: 10.1109/2945.646233
[17]
WANG H, SONG M Ckmeans. 1d. dp: optimal k-means clustering in one dimension by dynamic programming[J]. The R Journal, 2011, 3 (2): 29
doi: 10.32614/RJ-2011-015
[18]
石雅婷, 顾晓娟 人眼亮度感知特性在高动态范围影像中的应用[J]. 现代电影技术, 2021, 9: 9- 16 SHI Ya-ting, GU Xiao-juan The application of human lightness perception in high dynamic range images[J]. Advanced Motion Picture Technology, 2021, 9: 9- 16
doi: 10.3969/j.issn.1673-3215.2021.09.002
[19]
徐海松. 颜色信息工程第2版 [M]. 杭州: 浙江大学出版社, 2015: 50-54. XU Hai-song. Color information engineering and the second edition [M]. Hangzhou: Zhejiang University Press, 2015: 50-54.
[20]
PANETTA K, KEZEBOU L, OLUDARE V, et al TMO-Net: a parameter-free tone mapping operator using generative adversarial network, and performance benchmarking on large scale HDR dataset[J]. IEEE Access, 2021, 9: 39500- 39517
doi: 10.1109/ACCESS.2021.3064295
[21]
YEGANEH H, WANG Z Objective quality assessment of tone-mapped images[J]. IEEE Transactions on Image Processing, 2012, 22 (2): 657- 667
[22]
KIM M H, KAUTZ J. Consistent tone reproduction [C]// Proceedings of the Tenth IASTED International Conference on Computer Graphics and Imaging. ACTA Press Anaheim, 2008: 152-159.
[23]
REINHARD E, STARK M, SHIRLEY P, et al. Photographic tone reproduction for digital images [C]// Proceedings of the 29th Annual Conference on Computer Graphics and Interactive Techniques. San Antonio Texas, 2002: 267-276.
[24]
KRAWCZYK G, MYSZKOWSKI K, SEIDEL H P. Lightness perception in tone reproduction for high dynamic range images [C]// Computer Graphics Forum. North Holland, 2005: 635-646.
[25]
SHAN Q, JIA J, BROWN M S Globally optimized linear windowed tone mapping[J]. IEEE Transactions on Visualization and Computer Graphics, 2009, 16 (4): 663- 675
[26]
LI H, JIA X, ZHANG L Clustering based content and color adaptive tone mapping[J]. Computer Vision and Image Understanding, 2018, 168: 37- 49
doi: 10.1016/j.cviu.2017.11.001
