Please wait a minute...
浙江大学学报(工学版)  2025, Vol. 59 Issue (3): 557-565    DOI: 10.3785/j.issn.1008-973X.2025.03.013
计算机技术     
基于光源感知的跨相机颜色一致性算法
杨敏航(),徐海松*(),黄益铭,叶正男,张云涛,胡兵
浙江大学 光电科学与工程学院,极端光学技术与仪器全国重点实验室,浙江 杭州 310027
Illumination-aware color consistency algorithm for cross-camera applications
Minhang YANG(),Haisong XU*(),Yiming HUANG,Zhengnan YE,Yuntao ZHANG,Bin HU
State Key Laboratory of Extreme Photonics and Instrumentation, College of Optical Science and Engineering, Zhejiang University, Hangzhou 310027, China
 全文: PDF(2302 KB)   HTML
摘要:

针对相机传感器光谱灵敏度和内置图像信号处理(ISP)模块差异导致的图像间颜色不一致的问题,提出2种基于光源感知的跨相机颜色一致性算法,即色温(CCT)感知的自适应分区映射算法和类别感知的自适应分类映射算法,用于将 RAW 图像从待处理相机的颜色空间转换到目标相机的颜色空间. 通过在不同色温和类型的典型光源下得到的训练算子来推导目标场景最终的映射算子,为不同照明场景建立特定的映射关系,进而实现跨相机图像的转换. 在实验中采用颜色差异指标来对算法进行客观评估,结果表明所提出的2种算法在色差和视觉效果方面均比现有经典方法展现出更好的性能,能够有效适应照明条件的多样性和复杂性,具有广泛的应用潜力.

关键词: 图像处理颜色一致性跨相机应用RAW图像光源感知    
Abstract:

The issue of color inconsistency between images was caused by the differences in camera sensor spectral sensitivities and built-in image signal processing (ISP) modules. Two illumination-aware color consistency algorithms for cross-camera applications, namely the correlated-color-temperatures (CCT)-aware adaptive partitioning mapping algorithm and the category-aware adaptive classification mapping algorithm, were proposed to convert the RAW image from the color space of the source camera to that of the target camera. Specifically, by deriving the final mapping operator for the target scene based on the training operators obtained under typical light sources of different CCTs and types, specific mapping relationships were established for different lighting scenarios, thereby achieving the cross-camera image conversion. The performance of the algorithms was objectively evaluated using color difference metrics in the experiments. Results illustrated that both of the proposed algorithms could achieve better performance in terms of color difference and visual effect compared to the existing classical methods, effectively adapting to the diversity and complexity of lighting conditions, thereby exhibiting their potential for wide applications.

Key words: image processing    color consistency    cross-camera applications    RAW image    illumination awareness
收稿日期: 2023-12-28 出版日期: 2025-03-10
CLC:  O 432  
基金资助: 中央高校基本科研业务费专项资金资助项目(S20220156).
通讯作者: 徐海松     E-mail: yangmh011899@163.com;chsxu@zju.edu.cn
作者简介: 杨敏航(1999—),女,硕士生,从事颜色一致性与颜色再现研究. orcid.org/0009-0005-7620-6747. E-mail:yangmh011899@163.com
服务  
把本文推荐给朋友
加入引用管理器
E-mail Alert
作者相关文章  
杨敏航
徐海松
黄益铭
叶正男
张云涛
胡兵

引用本文:

杨敏航,徐海松,黄益铭,叶正男,张云涛,胡兵. 基于光源感知的跨相机颜色一致性算法[J]. 浙江大学学报(工学版), 2025, 59(3): 557-565.

Minhang YANG,Haisong XU,Yiming HUANG,Zhengnan YE,Yuntao ZHANG,Bin HU. Illumination-aware color consistency algorithm for cross-camera applications. Journal of ZheJiang University (Engineering Science), 2025, 59(3): 557-565.

链接本文:

https://www.zjujournals.com/eng/CN/10.3785/j.issn.1008-973X.2025.03.013        https://www.zjujournals.com/eng/CN/Y2025/V59/I3/557

图 1  CCT-AAPM算法的框架图
图 2  CA-ACM算法的流程图
图 3  数据集中部分示例图像
照明设备光源E/luxCCT/K
多光源
标准灯箱
A15792826
CWF15414078
D6517216525
H15822347
TL8415404028
U3014962912
LED照明系统LED14632525
LED24743507
LED34774488
LED44705513
LED54746391
LED64757307
表 1  算法训练中照明光源的具体参数情况
图 4  训练集所包含光源的光谱功率分布
算法LED日光荧光灯钨丝灯
$\Delta E_{ab}^*$$\Delta E_{ab,{\text{ }}w/o{\text{ }}\Delta {L^*}}^*$$\Delta C_{ab}^*$$\Delta E_{ab}^*$$\Delta E_{ab,{\text{ }}w/o{\text{ }}\Delta {L^*}}^*$$\Delta C_{ab}^*$$\Delta E_{ab}^*$$\Delta E_{ab,{\text{ }}w/o{\text{ }}\Delta {L^*}}^*$$\Delta C_{ab}^*$$\Delta E_{ab}^*$$\Delta E_{ab,{\text{ }}w/o{\text{ }}\Delta {L^*}}^*$$\Delta C_{ab}^*$
智能手机超广角副摄像
头 → 智能手机主摄像头
基准4.783.682.666.914.724.184.763.182.254.603.332.44
文献[21]3.612.581.835.052.772.263.271.711.173.412.241.58
文献[16]6.525.753.315.713.481.984.573.251.856.655.743.17
文献[4]4.173.062.004.422.571.573.362.021.333.221.971.28
全局方法[21]3.652.561.744.642.311.623.421.801.193.512.231.53
CCT-AAPM3.262.221.474.282.041.323.261.621.073.001.801.14
CA-ACM3.332.251.494.051.901.173.341.701.152.971.771.12
智能手机主摄像头 →
尼康D3x 数码单反相机
基准5.594.073.244.042.662.137.885.825.619.317.416.91
文献[21]5.374.093.195.054.033.515.182.411.966.794.893.84
文献[16]6.905.463.496.004.812.9811.3910.036.3611.7610.126.36
文献[4]5.263.472.363.432.081.675.902.862.285.923.332.53
全局方法[21]4.322.741.893.352.041.655.202.542.105.593.382.57
CCT-AAPM3.952.511.652.741.551.144.922.051.495.082.671.67
CA-ACM4.022.541.702.731.531.144.912.021.555.042.601.60
表 2  4种照明条件下3台相机之间的颜色一致性实验结果
图 5  3台相机间所有测试图像的平均颜色差异结果
图 6  5幅示例图像的可视化结果
变换形式智能手机副摄 → 智能手机主摄智能手机主摄 → 尼康D3x相机
$\Delta E_{ab}^*$$\Delta E_{ab,{\text{ }}w/o{\text{ }}\Delta {L^*}}^*$$\Delta C_{ab}^*$$\Delta E_{ab}^*$$\Delta E_{ab,{\text{ }}w/o{\text{ }}\Delta {L^*}}^*$$\Delta C_{ab}^*$
基准5.043.682.776.144.513.89
线性3×33.332.001.314.012.261.52
多项式6×33.191.941.273.992.241.49
根多项式6×33.221.911.263.892.131.44
单应性变换3.852.541.774.832.922.01
表 3  不同变换形式的映射算子对CCT-AAPM算法性能的影响
图 7  用于组合算法性能评估的典型场景
典型场景$\Delta E_{ab}^*$$\Delta E_{ab,{\text{ }}w/o{\text{ }}\Delta {L^*}}^*$$\Delta C_{ab}^*$
CCT-AAPMCA-ACM组合算法CCT-AAPMCA-ACM组合算法CCT-AAPMCA-ACM组合算法
图像(a)4.254.393.963.533.553.252.692.762.44
图像(b)3.073.232.992.732.812.651.741.881.69
图像(c)2.322.422.212.052.061.791.091.130.92
图像(d)2.102.211.951.511.531.310.871.060.77
图像(e)2.702.842.561.241.321.150.910.990.84
图像(f)1.281.451.240.911.050.810.640.780.57
图像(g)2.072.161.981.291.311.110.860.860.74
图像(h)2.922.832.702.242.091.981.641.491.40
图像(i)3.193.072.912.272.202.051.711.681.47
图像(j)2.922.822.751.851.761.651.130.990.92
图像(k)3.263.123.102.572.442.341.491.321.27
表 4  CCT-AAPM、CA-ACM及其组合算法的颜色一致性测试结果
1 JIANG J, LIU D, GU J, et al. What is the space of spectral sensitivity functions for digital color cameras? [C]// IEEE Workshop on Applications of Computer Vision . Clearwater Beach: 2013: 168–179.
2 RAMANATH R, SNYDER W E, YOO Y, et al Color image processing pipeline[J]. IEEE Signal Processing Magazine, 2005, 22 (1): 34- 43
doi: 10.1109/MSP.2005.1407713
3 KARAIMER H C, BROWN M S. A software platform for manipulating the camera imaging pipeline [C]// Computer Vision–ECCV 2016: 14th European Conference . Amsterdam: Springer, 2016: 429–444.
4 GONG H, FINLAYSON G D, FISHER R B, et al 3D color homography model for photo-realistic color transfer re-coding[J]. The Visual Computer, 2019, 35 (3): 323- 333
doi: 10.1007/s00371-017-1462-x
5 ANIRUDTH N, PRASAD B P, JAIN A, et al. Robust photometric alignment for asymmetric camera system [C]// IEEE International Conference on Consumer Electronics . Las Vegas: IEEE, 2018: 1–4.
6 OLIVEIRA M, SAPPA A D, SANTOS V. Unsupervised local color correction for coarsely registered images [C]// IEEE Conference on Computer Vision and Pattern Recognition . Colorado Springs: IEEE, 2011: 201–208.
7 WANG H, DAI L, ZHANG X. Consistent segmentation based color correction for coarsely registered images [C]// 2nd IAPR Asian Conference on Pattern Recognition . Naha: IEEE, 2013: 319–324.
8 CHISTOV E, ALUTIS N, VATOLIN D, et al. Color mismatches in stereoscopic video: real-world dataset and deep correction method [EB/OL]. (2023-06-15)[2024-2-22]. https://arxiv.org/abs/2303.06657.
9 TIAN Q C, COHEN L D. Color correction in image stitching using histogram specification and global mapping [C]// 6th International Conference on Image Processing Theory, Tools and Applications . Oulu: IEEE, 2016: 1–6.
10 DING C Q, MA Z Multi-camera color correction via hybrid histogram matching[J]. IEEE Transactions on Circuits and Systems for Video Technology, 2021, 31 (9): 3327- 3337
doi: 10.1109/TCSVT.2020.3038484
11 HWANG Y, LEE J Y, KWEON I S, et al Probabilistic moving least squares with spatial constraints for nonlinear color transfer between images[J]. Computer Vision and Image Understanding, 2019, 180: 1- 12
doi: 10.1016/j.cviu.2018.11.001
12 LI Y, YIN H, YAO J, et al A unified probabilistic framework of robust and efficient color consistency correction for multiple images[J]. Journal of Photogrammetry and Remote Sensing, 2022, 190: 1- 24
doi: 10.1016/j.isprsjprs.2022.05.009
13 THANH O V, CANHAM T, VAZQUEZ-CORRAL J, et al. Color stabilization for multi-camera light-field imaging [C]// 2020 IEEE International Conference on Acoustics, Speech and Signal Processing . Barcelona: IEEE, 2020: 2148–2152.
14 GIL RODRÍGUEZ R, VAZQUEZ-CORRAL J, BERTALMÍO M Color matching images with unknown non-linear encodings[J]. IEEE Transactions on Image Processing, 2020, 29: 4435- 4444
doi: 10.1109/TIP.2020.2968766
15 OSKARSSON M. Robust image-to-image color transfer using optimal inlier maximization [C]// Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition . Nashville: IEEE, 2021: 786–795.
16 ZHAO Y F, FERGUSON S, ZHOU H Y, et al Color alignment for relative color constancy via non-standard references[J]. IEEE Transactions on Image Processing, 2022, 31: 6591- 6604
doi: 10.1109/TIP.2022.3214107
17 BARBERO-ÁLVAREZ M A, MENÉNDEZ J M, RODRIGO J A An adaptive colour calibration for crowdsourced images in heritage preservation science[J]. IEEE Access, 2020, 8: 185093- 185111
doi: 10.1109/ACCESS.2020.3017390
18 FANG F, GONG H, MACKIEWICZ M, et al. Colour correction toolbox [C]// Proceedings of 13th AIC Congress . Jeju: Korea Society of Color Studies, 2017: 13–18.
19 ZHAO S, LIU L, FENG Z, et al Colorimetric characterization of color imaging system based on kernel partial least squares[J]. Sensors, 2023, 23 (12): 5706
doi: 10.3390/s23125706
20 GONG R, WANG Q, SHAO X P, et al A color calibration method between different digital cameras[J]. Optik, 2016, 127 (6): 3281- 3285
doi: 10.1016/j.ijleo.2015.12.003
21 NGUYEN H M R, PRASAD D K, BROWN M S. Raw-to-raw: mapping between image sensor color responses [C]// Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition . Columbus: IEEE, 2014: 3398–3405.
22 AFIFI M, ABUOLAIM A. Semi-supervised raw-to-raw mapping [EB/OL]. (2021-09-06)[2023-12-20]. https://arxiv.org/abs/2106.13883.
23 KARAIMER H C, BROWN M S. Improving color reproduction accuracy on cameras [C]// Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition . Salt Lake City: IEEE, 2018: 6440–6449.
24 ROBERTSON A R Computation of correlated color temperature and distribution temperature[J]. Journal of the Optical Society of America, 1968, 58 (11): 1528- 1535
doi: 10.1364/JOSA.58.001528
25 BAXTER D, ROYER M, SMET K Modifications of the Robertson method for calculating correlated color temperature to improve accuracy and speed[J]. Leukos, 2024, 20 (1): 55- 66
doi: 10.1080/15502724.2023.2166060
26 ZHANG F Z, XU H S, WANG Z H Spectral design methods for multi-channel LED light sources based on differential evolution[J]. Applied optics, 2016, 55 (28): 7771- 7781
doi: 10.1364/AO.55.007771
27 ROBERTSON A R The CIE 1976 color-difference formulae[J]. Color Research and Application, 1977, 2 (1): 7- 11
doi: 10.1002/j.1520-6378.1977.tb00104.x
[1] 陈智超,杨杰,李凡,冯志成. 基于深度学习的列车运行环境感知关键算法研究综述[J]. 浙江大学学报(工学版), 2025, 59(1): 1-17.
[2] 赵小强,王泽,宋昭漾,蒋红梅. 基于动态注意力网络的图像超分辨率重建[J]. 浙江大学学报(工学版), 2023, 57(8): 1487-1494.
[3] 郭浩然,郭继昌,汪昱东. 面向水下场景的轻量级图像语义分割网络[J]. 浙江大学学报(工学版), 2023, 57(7): 1278-1286.
[4] 熊帆,陈田,卞佰成,刘军. 基于卷积循环神经网络的芯片表面字符识别[J]. 浙江大学学报(工学版), 2023, 57(5): 948-956.
[5] 温佩芝,陈君谋,肖雁南,温雅媛,黄文明. 基于生成式对抗网络和多级小波包卷积网络的水下图像增强算法[J]. 浙江大学学报(工学版), 2022, 56(2): 213-224.
[6] 蒋昊,徐海松. 基于直方图与图像分块融合的阶调映射算法[J]. 浙江大学学报(工学版), 2022, 56(11): 2224-2231.
[7] 陈彤,郭剑锋,韩心中,谢学立,席建祥. 基于生成对抗模型的可见光-红外图像匹配方法[J]. 浙江大学学报(工学版), 2022, 56(1): 63-74.
[8] 徐铸业,赵小强,蒋红梅. 基于点分布模型的3D模型拟合方法[J]. 浙江大学学报(工学版), 2021, 55(12): 2373-2381.
[9] 李瑛,成芳,赵志林. 采用结构光的大跨度销孔加工精度在线测量[J]. 浙江大学学报(工学版), 2020, 54(3): 557-565.
[10] 王万良,杨小涵,赵燕伟,高楠,吕闯,张兆娟. 采用卷积自编码器网络的图像增强算法[J]. 浙江大学学报(工学版), 2019, 53(9): 1728-1740.
[11] 周昊, 李宁, 李源, 赵梦豪, 岑可法. 富氧条件下乙醇喷雾燃烧特性的实验研究[J]. 浙江大学学报(工学版), 2018, 52(9): 1821-1827.
[12] 张承志, 冯华君, 徐之海, 李奇, 陈跃庭. 图像噪声方差分段估计法[J]. 浙江大学学报(工学版), 2018, 52(9): 1804-1810.
[13] 周佳立, 陈以军, 武敏. 基于FPGA监听的图像采集与预处理方法[J]. 浙江大学学报(工学版), 2018, 52(2): 398-405.
[14] 黄松, 易本顺. 基于自适应透射率比的水下图像复原算法[J]. 浙江大学学报(工学版), 2018, 52(1): 166-173.
[15] 陈德, 韩森, 苏谦, 韩霄. 基于抗滑降噪性能的沥青路面表面构造评价指标[J]. 浙江大学学报(工学版), 2017, 51(5): 896-903.