Please wait a minute...
Journal of ZheJiang University (Engineering Science)  2025, Vol. 59 Issue (3): 557-565    DOI: 10.3785/j.issn.1008-973X.2025.03.013
    
Illumination-aware color consistency algorithm for cross-camera applications
Minhang YANG(),Haisong XU*(),Yiming HUANG,Zhengnan YE,Yuntao ZHANG,Bin HU
State Key Laboratory of Extreme Photonics and Instrumentation, College of Optical Science and Engineering, Zhejiang University, Hangzhou 310027, China
Download: HTML     PDF(2302KB) HTML
Export: BibTeX | EndNote (RIS)      

Abstract  

The issue of color inconsistency between images was caused by the differences in camera sensor spectral sensitivities and built-in image signal processing (ISP) modules. Two illumination-aware color consistency algorithms for cross-camera applications, namely the correlated-color-temperatures (CCT)-aware adaptive partitioning mapping algorithm and the category-aware adaptive classification mapping algorithm, were proposed to convert the RAW image from the color space of the source camera to that of the target camera. Specifically, by deriving the final mapping operator for the target scene based on the training operators obtained under typical light sources of different CCTs and types, specific mapping relationships were established for different lighting scenarios, thereby achieving the cross-camera image conversion. The performance of the algorithms was objectively evaluated using color difference metrics in the experiments. Results illustrated that both of the proposed algorithms could achieve better performance in terms of color difference and visual effect compared to the existing classical methods, effectively adapting to the diversity and complexity of lighting conditions, thereby exhibiting their potential for wide applications.



Key wordsimage processing      color consistency      cross-camera applications      RAW image      illumination awareness     
Received: 28 December 2023      Published: 10 March 2025
CLC:  O 432  
Fund:  中央高校基本科研业务费专项资金资助项目(S20220156).
Corresponding Authors: Haisong XU     E-mail: yangmh011899@163.com;chsxu@zju.edu.cn
Cite this article:

Minhang YANG,Haisong XU,Yiming HUANG,Zhengnan YE,Yuntao ZHANG,Bin HU. Illumination-aware color consistency algorithm for cross-camera applications. Journal of ZheJiang University (Engineering Science), 2025, 59(3): 557-565.

URL:

https://www.zjujournals.com/eng/10.3785/j.issn.1008-973X.2025.03.013     OR     https://www.zjujournals.com/eng/Y2025/V59/I3/557


基于光源感知的跨相机颜色一致性算法

针对相机传感器光谱灵敏度和内置图像信号处理(ISP)模块差异导致的图像间颜色不一致的问题,提出2种基于光源感知的跨相机颜色一致性算法,即色温(CCT)感知的自适应分区映射算法和类别感知的自适应分类映射算法,用于将 RAW 图像从待处理相机的颜色空间转换到目标相机的颜色空间. 通过在不同色温和类型的典型光源下得到的训练算子来推导目标场景最终的映射算子,为不同照明场景建立特定的映射关系,进而实现跨相机图像的转换. 在实验中采用颜色差异指标来对算法进行客观评估,结果表明所提出的2种算法在色差和视觉效果方面均比现有经典方法展现出更好的性能,能够有效适应照明条件的多样性和复杂性,具有广泛的应用潜力.


关键词: 图像处理,  颜色一致性,  跨相机应用,  RAW图像,  光源感知 
Fig.1 Framework of CCT-AAPM algorithm
Fig.2 Procedure of CA-ACM algorithm
Fig.3 Example images from dataset
照明设备光源E/luxCCT/K
多光源
标准灯箱
A15792826
CWF15414078
D6517216525
H15822347
TL8415404028
U3014962912
LED照明系统LED14632525
LED24743507
LED34774488
LED44705513
LED54746391
LED64757307
Tab.1 Specific parameters of lighting sources during training phase of algorithm
Fig.4 Spectral power distributions of lighting conditions for training set
算法LED日光荧光灯钨丝灯
$\Delta E_{ab}^*$$\Delta E_{ab,{\text{ }}w/o{\text{ }}\Delta {L^*}}^*$$\Delta C_{ab}^*$$\Delta E_{ab}^*$$\Delta E_{ab,{\text{ }}w/o{\text{ }}\Delta {L^*}}^*$$\Delta C_{ab}^*$$\Delta E_{ab}^*$$\Delta E_{ab,{\text{ }}w/o{\text{ }}\Delta {L^*}}^*$$\Delta C_{ab}^*$$\Delta E_{ab}^*$$\Delta E_{ab,{\text{ }}w/o{\text{ }}\Delta {L^*}}^*$$\Delta C_{ab}^*$
智能手机超广角副摄像
头 → 智能手机主摄像头
基准4.783.682.666.914.724.184.763.182.254.603.332.44
文献[21]3.612.581.835.052.772.263.271.711.173.412.241.58
文献[16]6.525.753.315.713.481.984.573.251.856.655.743.17
文献[4]4.173.062.004.422.571.573.362.021.333.221.971.28
全局方法[21]3.652.561.744.642.311.623.421.801.193.512.231.53
CCT-AAPM3.262.221.474.282.041.323.261.621.073.001.801.14
CA-ACM3.332.251.494.051.901.173.341.701.152.971.771.12
智能手机主摄像头 →
尼康D3x 数码单反相机
基准5.594.073.244.042.662.137.885.825.619.317.416.91
文献[21]5.374.093.195.054.033.515.182.411.966.794.893.84
文献[16]6.905.463.496.004.812.9811.3910.036.3611.7610.126.36
文献[4]5.263.472.363.432.081.675.902.862.285.923.332.53
全局方法[21]4.322.741.893.352.041.655.202.542.105.593.382.57
CCT-AAPM3.952.511.652.741.551.144.922.051.495.082.671.67
CA-ACM4.022.541.702.731.531.144.912.021.555.042.601.60
Tab.2 Color consistency results of four illumination conditions for mapping among three cameras
Fig.5 Average color differences for all testing images among three cameras
Fig.6 Visualization results of five example images
变换形式智能手机副摄 → 智能手机主摄智能手机主摄 → 尼康D3x相机
$\Delta E_{ab}^*$$\Delta E_{ab,{\text{ }}w/o{\text{ }}\Delta {L^*}}^*$$\Delta C_{ab}^*$$\Delta E_{ab}^*$$\Delta E_{ab,{\text{ }}w/o{\text{ }}\Delta {L^*}}^*$$\Delta C_{ab}^*$
基准5.043.682.776.144.513.89
线性3×33.332.001.314.012.261.52
多项式6×33.191.941.273.992.241.49
根多项式6×33.221.911.263.892.131.44
单应性变换3.852.541.774.832.922.01
Tab.3 Effect of mapping operators with different transformation forms on performance of CCT-AAPM algorithm
Fig.7 Typical scenarios for evaluation of combination algorithm
典型场景$\Delta E_{ab}^*$$\Delta E_{ab,{\text{ }}w/o{\text{ }}\Delta {L^*}}^*$$\Delta C_{ab}^*$
CCT-AAPMCA-ACM组合算法CCT-AAPMCA-ACM组合算法CCT-AAPMCA-ACM组合算法
图像(a)4.254.393.963.533.553.252.692.762.44
图像(b)3.073.232.992.732.812.651.741.881.69
图像(c)2.322.422.212.052.061.791.091.130.92
图像(d)2.102.211.951.511.531.310.871.060.77
图像(e)2.702.842.561.241.321.150.910.990.84
图像(f)1.281.451.240.911.050.810.640.780.57
图像(g)2.072.161.981.291.311.110.860.860.74
图像(h)2.922.832.702.242.091.981.641.491.40
图像(i)3.193.072.912.272.202.051.711.681.47
图像(j)2.922.822.751.851.761.651.130.990.92
图像(k)3.263.123.102.572.442.341.491.321.27
Tab.4 Color consistency results of CCT-AAPM, CA-ACM and their combination algorithm
[1]   JIANG J, LIU D, GU J, et al. What is the space of spectral sensitivity functions for digital color cameras? [C]// IEEE Workshop on Applications of Computer Vision . Clearwater Beach: 2013: 168–179.
[2]   RAMANATH R, SNYDER W E, YOO Y, et al Color image processing pipeline[J]. IEEE Signal Processing Magazine, 2005, 22 (1): 34- 43
doi: 10.1109/MSP.2005.1407713
[3]   KARAIMER H C, BROWN M S. A software platform for manipulating the camera imaging pipeline [C]// Computer Vision–ECCV 2016: 14th European Conference . Amsterdam: Springer, 2016: 429–444.
[4]   GONG H, FINLAYSON G D, FISHER R B, et al 3D color homography model for photo-realistic color transfer re-coding[J]. The Visual Computer, 2019, 35 (3): 323- 333
doi: 10.1007/s00371-017-1462-x
[5]   ANIRUDTH N, PRASAD B P, JAIN A, et al. Robust photometric alignment for asymmetric camera system [C]// IEEE International Conference on Consumer Electronics . Las Vegas: IEEE, 2018: 1–4.
[6]   OLIVEIRA M, SAPPA A D, SANTOS V. Unsupervised local color correction for coarsely registered images [C]// IEEE Conference on Computer Vision and Pattern Recognition . Colorado Springs: IEEE, 2011: 201–208.
[7]   WANG H, DAI L, ZHANG X. Consistent segmentation based color correction for coarsely registered images [C]// 2nd IAPR Asian Conference on Pattern Recognition . Naha: IEEE, 2013: 319–324.
[8]   CHISTOV E, ALUTIS N, VATOLIN D, et al. Color mismatches in stereoscopic video: real-world dataset and deep correction method [EB/OL]. (2023-06-15)[2024-2-22]. https://arxiv.org/abs/2303.06657.
[9]   TIAN Q C, COHEN L D. Color correction in image stitching using histogram specification and global mapping [C]// 6th International Conference on Image Processing Theory, Tools and Applications . Oulu: IEEE, 2016: 1–6.
[10]   DING C Q, MA Z Multi-camera color correction via hybrid histogram matching[J]. IEEE Transactions on Circuits and Systems for Video Technology, 2021, 31 (9): 3327- 3337
doi: 10.1109/TCSVT.2020.3038484
[11]   HWANG Y, LEE J Y, KWEON I S, et al Probabilistic moving least squares with spatial constraints for nonlinear color transfer between images[J]. Computer Vision and Image Understanding, 2019, 180: 1- 12
doi: 10.1016/j.cviu.2018.11.001
[12]   LI Y, YIN H, YAO J, et al A unified probabilistic framework of robust and efficient color consistency correction for multiple images[J]. Journal of Photogrammetry and Remote Sensing, 2022, 190: 1- 24
doi: 10.1016/j.isprsjprs.2022.05.009
[13]   THANH O V, CANHAM T, VAZQUEZ-CORRAL J, et al. Color stabilization for multi-camera light-field imaging [C]// 2020 IEEE International Conference on Acoustics, Speech and Signal Processing . Barcelona: IEEE, 2020: 2148–2152.
[14]   GIL RODRÍGUEZ R, VAZQUEZ-CORRAL J, BERTALMÍO M Color matching images with unknown non-linear encodings[J]. IEEE Transactions on Image Processing, 2020, 29: 4435- 4444
doi: 10.1109/TIP.2020.2968766
[15]   OSKARSSON M. Robust image-to-image color transfer using optimal inlier maximization [C]// Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition . Nashville: IEEE, 2021: 786–795.
[16]   ZHAO Y F, FERGUSON S, ZHOU H Y, et al Color alignment for relative color constancy via non-standard references[J]. IEEE Transactions on Image Processing, 2022, 31: 6591- 6604
doi: 10.1109/TIP.2022.3214107
[17]   BARBERO-ÁLVAREZ M A, MENÉNDEZ J M, RODRIGO J A An adaptive colour calibration for crowdsourced images in heritage preservation science[J]. IEEE Access, 2020, 8: 185093- 185111
doi: 10.1109/ACCESS.2020.3017390
[18]   FANG F, GONG H, MACKIEWICZ M, et al. Colour correction toolbox [C]// Proceedings of 13th AIC Congress . Jeju: Korea Society of Color Studies, 2017: 13–18.
[19]   ZHAO S, LIU L, FENG Z, et al Colorimetric characterization of color imaging system based on kernel partial least squares[J]. Sensors, 2023, 23 (12): 5706
doi: 10.3390/s23125706
[20]   GONG R, WANG Q, SHAO X P, et al A color calibration method between different digital cameras[J]. Optik, 2016, 127 (6): 3281- 3285
doi: 10.1016/j.ijleo.2015.12.003
[21]   NGUYEN H M R, PRASAD D K, BROWN M S. Raw-to-raw: mapping between image sensor color responses [C]// Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition . Columbus: IEEE, 2014: 3398–3405.
[22]   AFIFI M, ABUOLAIM A. Semi-supervised raw-to-raw mapping [EB/OL]. (2021-09-06)[2023-12-20]. https://arxiv.org/abs/2106.13883.
[23]   KARAIMER H C, BROWN M S. Improving color reproduction accuracy on cameras [C]// Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition . Salt Lake City: IEEE, 2018: 6440–6449.
[24]   ROBERTSON A R Computation of correlated color temperature and distribution temperature[J]. Journal of the Optical Society of America, 1968, 58 (11): 1528- 1535
doi: 10.1364/JOSA.58.001528
[25]   BAXTER D, ROYER M, SMET K Modifications of the Robertson method for calculating correlated color temperature to improve accuracy and speed[J]. Leukos, 2024, 20 (1): 55- 66
doi: 10.1080/15502724.2023.2166060
[26]   ZHANG F Z, XU H S, WANG Z H Spectral design methods for multi-channel LED light sources based on differential evolution[J]. Applied optics, 2016, 55 (28): 7771- 7781
doi: 10.1364/AO.55.007771
[27]   ROBERTSON A R The CIE 1976 color-difference formulae[J]. Color Research and Application, 1977, 2 (1): 7- 11
doi: 10.1002/j.1520-6378.1977.tb00104.x
[1] Zhichao CHEN,Jie YANG,Fan LI,Zhicheng FENG. Review on deep learning-based key algorithm for train running environment perception[J]. Journal of ZheJiang University (Engineering Science), 2025, 59(1): 1-17.
[2] Xiao-qiang ZHAO,Ze WANG,Zhao-yang SONG,Hong-mei JIANG. Image super-resolution reconstruction based on dynamic attention network[J]. Journal of ZheJiang University (Engineering Science), 2023, 57(8): 1487-1494.
[3] Hao-ran GUO,Ji-chang GUO,Yu-dong WANG. Lightweight semantic segmentation network for underwater image[J]. Journal of ZheJiang University (Engineering Science), 2023, 57(7): 1278-1286.
[4] Fan XIONG,Tian CHEN,Bai-cheng BIAN,Jun LIU. Chip surface character recognition based on convolutional recurrent neural network[J]. Journal of ZheJiang University (Engineering Science), 2023, 57(5): 948-956.
[5] Pei-zhi WEN,Jun-mou CHEN,Yan-nan XIAO,Ya-yuan WEN,Wen-ming HUANG. Underwater image enhancement algorithm based on GAN and multi-level wavelet CNN[J]. Journal of ZheJiang University (Engineering Science), 2022, 56(2): 213-224.
[6] Hao JIANG,Hai-song XU. Histogram based tone mapping algorithm using image segmentation and fusion[J]. Journal of ZheJiang University (Engineering Science), 2022, 56(11): 2224-2231.
[7] Tong CHEN,Jian-feng GUO,Xin-zhong HAN,Xue-li XIE,Jian-xiang XI. Visible and infrared image matching method based on generative adversarial model[J]. Journal of ZheJiang University (Engineering Science), 2022, 56(1): 63-74.
[8] Zhu-ye XU,Xiao-qiang ZHAO,Hong-mei JIANG. 3D model fitting method based on point distribution model[J]. Journal of ZheJiang University (Engineering Science), 2021, 55(12): 2373-2381.
[9] Ying LI,Fang CHENG,Zhi-lin ZHAO. Machining precision online measurement of large span pin hole using structured light[J]. Journal of ZheJiang University (Engineering Science), 2020, 54(3): 557-565.
[10] Wan-liang WANG,Xiao-han YANG,Yan-wei ZHAO,Nan GAO,Chuang LV,Zhao-juan ZHANG. Image enhancement algorithm with convolutional auto-encoder network[J]. Journal of ZheJiang University (Engineering Science), 2019, 53(9): 1728-1740.
[11] ZHOU Hao, LI Ning, LI Yuan, ZHAO Meng-hao, CEN Ke-fa. Experimental study on ethanol spray combustion characteristics under oxy-fuel conditions[J]. Journal of ZheJiang University (Engineering Science), 2018, 52(9): 1821-1827.
[12] ZHANG Cheng-zhi, FENG Hua-jun, XU Zhi-hai, LI Qi, CHEN Yue-ting. Piecewise noise variance estimation of images based on wavelet transform[J]. Journal of ZheJiang University (Engineering Science), 2018, 52(9): 1804-1810.
[13] ZHOU Jia-li, CHEN Yi-jun, WU Min. Image acquisition and preprocessing method based on FPGA monitor[J]. Journal of ZheJiang University (Engineering Science), 2018, 52(2): 398-405.
[14] HUANG Song, YI Ben-shun. Underwater image restoration algorithm based on adaptive transmission ratio[J]. Journal of ZheJiang University (Engineering Science), 2018, 52(1): 166-173.
[15] ZHOU Hao, MA Wei chen, YANG Yu, CHEN Jian zhong. Investigation of flame characteristics of low NOx swirl burner[J]. Journal of ZheJiang University (Engineering Science), 2016, 50(4): 698-703.