Please wait a minute...
浙江大学学报(工学版)
浙江大学学报(工学版)  2022, Vol. 56 Issue (11): 2224-2231    DOI: 10.3785/j.issn.1008-973X.2022.11.013
计算机技术     
基于直方图与图像分块融合的阶调映射算法
蒋昊(),徐海松*()
浙江大学 光电科学与工程学院, 现代光学仪器国家重点实验室，浙江 杭州 310027
Histogram based tone mapping algorithm using image segmentation and fusion
Hao JIANG(),Hai-song XU*()
State Key Laboratory of Modern Optical Instrumentation, College of Optical Science and Engineering, Zhejiang University, Hangzhou 310027, China
 全文: PDF(5565 KB)   HTML
摘要：

阶调映射算法（TMO）旨在将高动态范围（HDR）图像以最符合人眼感知的形式复现于低动态范围显示设备上. 为此基于图像分块与融合，提出一种直方图调整的阶调映射算法. 高动态范围图像被分为若干非重合的长方形区域，在每一块区域中阶调映射问题被视为基于直方图的K均值聚类问题并且建立相应的求解目标函数. 各区域中解得的映射函数根据该区域的均匀性进行调整，以避免对比度增强过度，从而减少伪影. 最后，提出一种双边滤波形式的图像融合策略以保证区域边界的平滑性，兼顾位置及亮度差，进而提升映射结果的自然性. 在实验中，采用阶调映射图像质量指标来对算法进行客观评估，结果表明所提算法相对于经典的阶调映射方法有更好的阶调映射效果，在细节增强与全局外貌保留间达到平衡，并在局部阶调映射算法中具有较好的运算效率.
关键词： 图像处理;  阶调映射;  高动态范围图像;  直方图调整;  图像融合;  K均值聚类    
Abstract:

The tone mapping operators (TMO) aims at reproducing the visual perception of high dynamic range (HDR) images on the low dynamic range media. A histogram based tone mapping algorithm was proposed to use image segmentation and fusion. The HDR images were segmented into non-overlapping rectangle blocks. In each block, the tone mapping problem was regarded as the K-means clustering problem based on the image histogram with a proposed loss function. The derived mapping function was adaptively adjusted considering the uniformity of the block. The detail-enhancing effect of the mapping function was then restrained by the block uniformity for less arising artefacts. An image fusion scheme was presented to deal with the artefacts on the segmentation boundaries. With the form of a bilateral filter, the fusion scheme took into account both the difference of position and luminance, ensuring the naturalness of the tone-mapped results. The proposed algorithm was evaluated to use the tone-mapped image quality index and the results illustrated a better image quality compared with the typical tone mapping methods. A balance between the detail enhancement and the global appearance preserving has also been achieved with?an?impressed?execution?efficiency?among?local?TMOs.
Key words: image processing    tone mapping    high dynamic range image    histogram adjustment    image fusion    K-means clustering
收稿日期: 2022-02-08 出版日期: 2022-12-02
CLC:  TN 27  
通讯作者: 徐海松     E-mail: zjujhh@163.com;chsxu@zju.edu.cn
作者简介: 蒋昊（1997—），男，硕士生，从事阶调映射算法研究. orcid.org/ 0000-0001-5701-6319. E-mail： zjujhh@163.com
服务　　
把本文推荐给朋友
加入引用管理器
E-mail Alert
作者相关文章　　
蒋昊
徐海松

引用本文:

蒋昊,徐海松. 基于直方图与图像分块融合的阶调映射算法[J]. 浙江大学学报(工学版), 2022, 56(11): 2224-2231.

Hao JIANG,Hai-song XU. Histogram based tone mapping algorithm using image segmentation and fusion. Journal of ZheJiang University (Engineering Science), 2022, 56(11): 2224-2231.

链接本文:

https://www.zjujournals.com/eng/CN/10.3785/j.issn.1008-973X.2022.11.013    或    https://www.zjujournals.com/eng/CN/Y2022/V56/I11/2224

图 1  基于直方图与图像分块融合的阶调映射算法的流程图
图 2  相同区域应用不同映射曲线的结果
图 3  不同距离衰减因子取值的融合效果示例
图 4  不同亮度衰减因子取值的融合效果示例
图像序号 Q
k均值聚类方法 本研究算法
1 0.67 0.88
2 0.53 0.82
3 0.65 0.87
4 0.65 0.87
5 0.76 0.98
6 0.70 0.90
7 0.76 0.95
8 0.75 0.97
9 0.59 0.84
均值 0.67 0.90
表 1  图像分块与否的TMQI分数
图 5  图像分块与否的映射结果
图像 Q
Drago et al. [4] Kimet al. [22] Reinhard et al. [23] Khan et al. [8] Krawcyzk et al. [24] Shan et al. [25] Liang et al. [11] Liet al. [26] 本研究算法
1 0.76 0.72 0.72 0.81 0.73 0.73 0.81 0.91 0.88
2 0.70 0.68 0.65 0.79 0.68 0.60 0.75 0.89 0.82
3 0.77 0.78 0.73 0.88 0.74 0.75 0.82 0.87 0.87
4 0.76 0.78 0.73 0.85 0.71 0.74 0.80 0.85 0.87
5 0.88 0.84 0.79 0.88 0.85 0.88 0.86 0.73 0.98
6 0.85 0.84 0.73 0.88 0.77 0.76 0.86 0.81 0.90
7 0.91 0.88 0.79 0.83 0.88 0.87 0.84 0.71 0.95
8 0.86 0.87 0.78 0.87 0.82 0.84 0.85 0.72 0.97
9 0.72 0.82 0.71 0.79 0.70 0.72 0.83 0.80 0.84
均值 0.80 0.80 0.74 0.84 0.76 0.77 0.82 0.81 0.90
表 2  不同算法的TMQI分数
图像 S
Drago et al. [4] Kimet al. [22] Reinhard et al. [23] Khan et al. [8] Krawcyzk et al. [24] Shan et al. [25] Liang et al. [11] Liet al. [26] 本研究算法
1 0.78 0.70 0.69 0.70 0.73 0.72 0.66 0.73 0.87
2 0.61 0.55 0.49 0.70 0.55 0.37 0.57 0.68 0.74
3 0.79 0.81 0.73 0.75 0.74 0.77 0.64 0.72 0.83
4 0.77 0.79 0.71 0.69 0.64 0.74 0.62 0.71 0.82
5 0.87 0.88 0.85 0.73 0.90 0.88 0.70 0.69 0.93
6 0.70 0.71 0.65 0.73 0.64 0.67 0.59 0.67 0.78
7 0.87 0.87 0.86 0.73 0.88 0.88 0.67 0.66 0.89
8 0.86 0.87 0.84 0.75 0.87 0.87 0.69 0.70 0.92
9 0.63 0.74 0.61 0.66 0.50 0.63 0.56 0.61 0.74
均值 0.76 0.77 0.72 0.72 0.72 0.73 0.63 0.69 0.83
表 3  不同算法的结构保真性分数
图像 N
Drago et al. [4] Kimet al. [22] Reinhard et al. [23] Khan et al. [8] Krawcyzk et al. [24] Shan et al. [25] Liang et al. [11] Liet al. [26] 本研究算法
1 0.05 0.01 0.00 0.33 0.01 0.01 0.41 0.85 0.44
2 0.01 0.01 0.00 0.22 0.02 0.00 0.26 0.86 0.33
3 0.05 0.08 0.01 0.62 0.00 0.01 0.48 0.64 0.45
4 0.04 0.07 0.00 0.56 0.02 0.01 0.45 0.52 0.44
5 0.46 0.23 0.06 0.66 0.24 0.41 0.60 0.03 0.97
6 0.55 0.45 0.05 0.72 0.22 0.15 0.84 0.38 0.74
7 0.61 0.44 0.06 0.36 0.41 0.38 0.52 0.00 0.85
8 0.36 0.38 0.05 0.60 0.17 0.25 0.57 0.01 0.93
9 0.04 0.34 0.03 0.30 0.13 0.06 0.74 0.45 0.45
均值 0.24 0.22 0.03 0.49 0.14 0.14 0.54 0.42 0.62
表 4  不同算法的自然性分数
图 6  阶调映射算法对比示例
算法 平均运行时间/s
Drago et al. [4] 5.61
Kimet al. [22] 2.08
Reinhard et al. [23] 2.73
Khan et al. [8] 8.95
Krawcyzk et al. [24] 11.90
Shan et al. [25] 169.01
Liang et al. [11] 68.07
Liet al. [26] 183.38
本研究算法 15.64
表 5  对比算法的平均运行时间
1 REINHARD E, HEIDRICH W, DEBEVEC P, et al. High dynamic range imaging: acquisition, display, and image-based lighting [M]. Morgan: Kaufmann, 2010: 1-18.
2 方竞宇. 高动态范围彩色图像捕获与显示方法及技术研究[D]. 杭州: 浙江大学, 2016: 78-80.
FANG Jing-yu. Study on methodology and technology of acquisition and rendering for high dynamic range color images [D]. Hangzhou: Zhejiang University, 2016: 78-80.
3 WARD G A contrast-based scale factor for luminance display[J]. Graphics Gems, 1994, 4: 415- 421
4 DRAGO F, MYSZKOWSKI K, ANNEN T, et al. Adaptive logarithmic mapping for displaying high contrast scenes [C]// Computer Graphics Forum. Oxford, UK: Blackwell Publishing, Inc, 2003: 419-426.
5 TUMBLIN J, RUSHMEIER H Tone reproduction for realistic images[J]. IEEE Computer Graphics and Applications, 1993, 13 (6): 42- 48
doi: 10.1109/38.252554
6 SCHLICK C. Quantization techniques for visualization of high dynamic range pictures, photorealistic rendering techniques [M]. Berlin, Heidelberg: Springer, 1995: 7-20.
7 MA K, YEGANEH H, ZENG K, et al High dynamic range image compression by optimizing tone mapped image quality index[J]. IEEE Transactions on Image Processing, 2015, 24 (10): 3086- 3097
doi: 10.1109/TIP.2015.2436340
8 KHAN I R, AZIZ W, SHIM S O Tone-mapping using perceptual-quantizer and image histogram[J]. IEEE Access, 2020, 8: 31350- 31358
doi: 10.1109/ACCESS.2020.2973273
9 AMBALATHANKANDY P, IKEBE M, YOSHIDA T, et al An adaptive global and local tone mapping algorithm implemented on FPGA[J]. IEEE Transactions on Circuits and Systems for Video Technology, 2019, 30 (9): 3015- 3028
10 ZHANG X S, YANG K F, ZHOU J, et al Retina inspired tone mapping method for high dynamic range images[J]. Optics Express, 2020, 28 (5): 5953- 5964
doi: 10.1364/OE.380555
11 LIANG Z, XU J, ZHANG D, et al. A hybrid l1-l0 layer decomposition model for tone mapping [C]// Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition. Salt Lake City, Utah, 2018: 4758-4766.
12 FAHIM M A N I, JUNG H Y Fast single-image HDR tone-mapping by avoiding base layer extraction[J]. Multidisciplinary Digital Publishing Institute, 2020, 20 (16): 4378
13 NGUYEN N H, VAN V T, LEE C Human visual system model-based optimized tone mapping of high dynamic range images[J]. IEEE Access, 2021, 9: 127343- 127355
doi: 10.1109/ACCESS.2021.3112046
14 DUAN J, BRESSAN M, DANCE C, et al Tone-mapping high dynamic range images by novel histogram adjustment[J]. Pattern Recognition, 2010, 43 (5): 1847- 1862
doi: 10.1016/j.patcog.2009.12.006
15 OSKARSSON M Temporally consistent tone mapping of images and video using optimal k-means clustering[J]. Journal of Mathematical Imaging and Vision, 2017, 57 (2): 225- 238
doi: 10.1007/s10851-016-0677-1
16 LARSON G W, RUSHMEIER H, PIATKO C A visibility matching tone reproduction operator for high dynamic range scenes[J]. IEEE Transactions on Visualization and Computer Graphics, 1997, 3 (4): 291- 306
doi: 10.1109/2945.646233
17 WANG H, SONG M Ckmeans. 1d. dp: optimal k-means clustering in one dimension by dynamic programming[J]. The R Journal, 2011, 3 (2): 29
doi: 10.32614/RJ-2011-015
18 石雅婷, 顾晓娟 人眼亮度感知特性在高动态范围影像中的应用[J]. 现代电影技术, 2021, 9: 9- 16
SHI Ya-ting, GU Xiao-juan The application of human lightness perception in high dynamic range images[J]. Advanced Motion Picture Technology, 2021, 9: 9- 16
doi: 10.3969/j.issn.1673-3215.2021.09.002
19 徐海松. 颜色信息工程第2版 [M]. 杭州: 浙江大学出版社, 2015: 50-54.
XU Hai-song. Color information engineering and the second edition [M]. Hangzhou: Zhejiang University Press, 2015: 50-54.
20 PANETTA K, KEZEBOU L, OLUDARE V, et al TMO-Net: a parameter-free tone mapping operator using generative adversarial network, and performance benchmarking on large scale HDR dataset[J]. IEEE Access, 2021, 9: 39500- 39517
doi: 10.1109/ACCESS.2021.3064295
21 YEGANEH H, WANG Z Objective quality assessment of tone-mapped images[J]. IEEE Transactions on Image Processing, 2012, 22 (2): 657- 667
22 KIM M H, KAUTZ J. Consistent tone reproduction [C]// Proceedings of the Tenth IASTED International Conference on Computer Graphics and Imaging. ACTA Press Anaheim, 2008: 152-159.
23 REINHARD E, STARK M, SHIRLEY P, et al. Photographic tone reproduction for digital images [C]// Proceedings of the 29th Annual Conference on Computer Graphics and Interactive Techniques. San Antonio Texas, 2002: 267-276.
24 KRAWCZYK G, MYSZKOWSKI K, SEIDEL H P. Lightness perception in tone reproduction for high dynamic range images [C]// Computer Graphics Forum. North Holland, 2005: 635-646.
25 SHAN Q, JIA J, BROWN M S Globally optimized linear windowed tone mapping[J]. IEEE Transactions on Visualization and Computer Graphics, 2009, 16 (4): 663- 675
26 LI H, JIA X, ZHANG L Clustering based content and color adaptive tone mapping[J]. Computer Vision and Image Understanding, 2018, 168: 37- 49
doi: 10.1016/j.cviu.2017.11.001
[1] 温佩芝,陈君谋,肖雁南,温雅媛,黄文明. 基于生成式对抗网络和多级小波包卷积网络的水下图像增强算法[J]. 浙江大学学报(工学版), 2022, 56(2): 213-224.
[2] 陈彤,郭剑锋,韩心中,谢学立,席建祥. 基于生成对抗模型的可见光-红外图像匹配方法[J]. 浙江大学学报(工学版), 2022, 56(1): 63-74.
[3] 徐铸业,赵小强,蒋红梅. 基于点分布模型的3D模型拟合方法[J]. 浙江大学学报(工学版), 2021, 55(12): 2373-2381.
[4] 李瑛,成芳,赵志林. 采用结构光的大跨度销孔加工精度在线测量[J]. 浙江大学学报(工学版), 2020, 54(3): 557-565.
[5] 王万良,杨小涵,赵燕伟,高楠,吕闯,张兆娟. 采用卷积自编码器网络的图像增强算法[J]. 浙江大学学报(工学版), 2019, 53(9): 1728-1740.
[6] 王硕朋,杨鹏,孙昊,刘迈. 两级参考点匹配位置指纹声源定位方法[J]. 浙江大学学报(工学版), 2019, 53(6): 1198-1204.
[7] 周昊, 李宁, 李源, 赵梦豪, 岑可法. 富氧条件下乙醇喷雾燃烧特性的实验研究[J]. 浙江大学学报(工学版), 2018, 52(9): 1821-1827.
[8] 张承志, 冯华君, 徐之海, 李奇, 陈跃庭. 图像噪声方差分段估计法[J]. 浙江大学学报(工学版), 2018, 52(9): 1804-1810.
[9] 周佳立, 陈以军, 武敏. 基于FPGA监听的图像采集与预处理方法[J]. 浙江大学学报(工学版), 2018, 52(2): 398-405.
[10] 黄松, 易本顺. 基于自适应透射率比的水下图像复原算法[J]. 浙江大学学报(工学版), 2018, 52(1): 166-173.
[11] 陈德, 韩森, 苏谦, 韩霄. 基于抗滑降噪性能的沥青路面表面构造评价指标[J]. 浙江大学学报(工学版), 2017, 51(5): 896-903.
[12] 周昊, 马炜晨, 杨玉, 陈建中. 低氮煤粉旋流燃烧器火焰特性的研究[J]. 浙江大学学报(工学版), 2016, 50(4): 698-703.
[13] 吴一全,殷骏,朱丽,袁永明. 基于蜂群优化或分解的二维Arimoto灰度熵阈值分割[J]. 浙江大学学报(工学版), 2015, 49(9): 1625-1633.
[14] 陈阔, 冯华君, 徐之海, 李奇, 陈跃庭. 细节保持的快速曝光融合[J]. 浙江大学学报(工学版), 2015, 49(6): 1048-1054.
[15] 江燊煜,陈阔,徐之海,冯华君,李奇,陈跃庭. 基于曝光适度评价的多曝光图像融合方法[J]. 浙江大学学报(工学版), 2015, 49(3): 470-475.