|
|
|
| Continuous digital zooming algorithm of dual-focal camera based on texture restoration |
Jiong-hui SONG1( ),Qi LI1,*(),Jing WANG2,Zhi-hai XU1,Hua-jun FENG1,Yue-ting CHEN1 |
1. College of Optical Science and Engineering, Zhejiang University, Hangzhou 310027, China
2. Science and Technology on Optical Radiation Laboratory, Beijing 100854, China |
|
|
|
Abstract A continuous digital zooming algorithm mainly for dual-focal camera imaging system was proposed. The continuous digital zooming problem of dual-focal camera was divided into the feature transfer problem in the field of view of the long-focus camera and the texture repair problem outside the field of view of the long-focus camera. The texture information of long-focus camera image was used to restore short-focus camera image referring to contextual image restoration algorithm, and a similar network structure was used to reduce the visual difference inside and outside the field of view of the long-focus camera. Experimental results show that compared with other algorithms, the proposed algorithm can improve the quality of images, improve the problem of large gap between the texture details inside and outside the field of view of the long-focus camera, and has higher subjective resolution and better visual clarity. In addition, the algorithm has an obvious repair effect on the texture details outside the field of view of the long-focus camera and also performs well for the actual images.
|
|
Received: 03 August 2020
Published: 01 September 2021
|
|
|
| Fund: 装备预研重点实验室基金资助项目（61424080214）；十三五民用航天资助项目 |
|
Corresponding Authors:
Qi LI
E-mail: 21830051@zju.edu.cn;liqi@zju.edu.cn
|
基于纹理修复的双焦相机连续数字变焦算法
设计主要用于双焦相机成像系统的连续数字变焦算法. 该算法将双焦相机连续数字变焦问题拆分成长焦相机视场内的特征迁移问题和长焦相机视场外的纹理修复问题. 在实现细节上，该算法参考基于上下文语义的图像修复算法的思路，利用长焦相机图像的纹理信息修复短焦相机图像，并在2个问题上使用相似的网络结构来降低长焦相机视场内外细节视觉效果的差异. 实验结果表明，相比其他算法，所提算法能显著提升变焦图像的质量，有效改善长焦相机视场内外纹理细节差距较大的问题，在主客观评价上都具有更好的表现. 此外，该方法对于处于长焦相机视场外的纹理细节，具有明显的修复效果，对于实际拍摄的图像同样有效.
关键词：
双焦相机,
连续数字变焦,
上下文语义,
纹理修复,
图像修复
|
|
| [1] |
KEYS R Cubic convolution interpolation for digital image processing[J]. Speech and Signal Processing, 1981, 29 (6): 1153- 1160
doi: 10.1109/TASSP.1981.1163711
|
|
|
| [2] |
LI X New edge-directed interpolation[J]. IEEE Transactions on Image Processing, 2001, 10 (10): 1521- 1527
doi: 10.1109/83.951537
|
|
|
| [3] |
DONG C, LOY C, HE K, et al Image super-resolution using deep convolutional networks[J]. IEEE Transactions on Pattern Analysis and Machine Intelligence, 2016, 38 (2): 295- 307
doi: 10.1109/TPAMI.2015.2439281
|
|
|
| [4] |
KIM J, LEE J, LEE K. Accurate image super-resolution using very deep convolutional networks[C]// 2016 IEEE Conference on Computer Vision and Pattern Recognition (CVPR). Las Vegas: IEEE, 2016: 1646-1654.
|
|
|
| [5] |
LEDIG C, THEIS L, HUSZAR F, et al. Photo-realistic single image super-resolution using a generative adversarial network[C]// 2017 IEEE Conference on Computer Vision and Pattern Recognition (CVPR). Hawaii: IEEE, 2017: 4681-4690.
|
|
|
| [6] |
YU S, MOON B, KIM D, et al Continuous digital zooming of asymmetric dual camera images using registration and variational image restoration[J]. Multidimensional Systems and Signal Processing, 2018, 29 (4): 1959- 1987
doi: 10.1007/s11045-017-0534-4
|
|
|
| [7] |
MOON B, YU S, KO S, et al Continuous digital zooming using local self-similarity-based superresolution for an asymmetric dual camera system[J]. Journal of the Optical Society of America A-optics Image Science and Vision, 2017, 34 (6): 991- 1003
doi: 10.1364/JOSAA.34.000991
|
|
|
| [8] |
MA H, LI Q, XU Z, et al Photo-realistic continuous digital zooming for an asymmetrical dual camera system[J]. Optics and Laser Technology, 2019, 109: 110- 122
doi: 10.1016/j.optlastec.2018.07.056
|
|
|
| [9] |
赫贵然, 李奇, 冯华君, 等 基于CNN特征提取的双焦相机连续数字变焦[J]. 浙江大学学报: 工学版, 2019, 53 (6): 1182- 1189
HE Gui-ran, LI Qi, FENG Hua-jun, et al Dual-focal camera continuous digital zoom based on CNN and feature extraction[J]. Journal of Zhejiang University: Engineering Science, 2019, 53 (6): 1182- 1189
|
|
|
| [10] |
ZHANG Z, WANG Z, LIN Z, et al. Image super-resolution by neural texture transfer[C]// 2019 IEEE Conference on Computer Vision and Pattern Recognition (CVPR). Long Bench: IEEE, 2019: 7982-7991.
|
|
|
| [11] |
YANG F, YANG H, FU J, et al. Learning texture transformer network for image super-resolution[C]// 2020 IEEE Conference on Computer Vision and Pattern Recognition (CVPR). Seattle: IEEE, 2020: 5791-5800.
|
|
|
| [12] |
ZENG Y, FU J, CHAO H, et al. Learning pyramid-context encoder network for high-quality image inpainting[C]// 2019 IEEE Conference on Computer Vision and Pattern Recognition (CVPR). Long Bench: IEEE, 2019: 1486-1494.
|
|
|
| [13] |
YI Z, TANG Q, AZIZI S, et al. Contextual residual aggregation for ultra high-resolution image inpainting[C]// 2020 IEEE Conference on Computer Vision and Pattern Recognition (CVPR). Seattle: IEEE, 2020: 7508-7517.
|
|
|
| [14] |
PATHAK D, KRAHENBUHL P, DONAHUE J, et al. Context encoders: feature learning by inpainting[C]// 2016 IEEE Conference on Computer Vision and Pattern Recognition (CVPR). Las Vegas: IEEE, 2016: 2536-2544.
|
|
|
| [15] |
SATOSHI I, EDGAR S, HIROSHI I Globally and locally consistent image completion[J]. ACM Transactions on Graphics, 2017, 36 (4): 107
|
|
|
| [16] |
YU J, LIN Z, YANG J, et al. Generative image inpainting with contextual attention[C]// 2018 IEEE Conference on Computer Vision and Pattern Recognition (CVPR). Salt Lake City: IEEE, 2018: 5505-5514.
|
|
|
| [17] |
GULRAJANI I, AHMED F, ARJOVSKY M, et al. Improved training of wasserstein GANs[C]// Thirty-first Conference on Neural Information Processing Systems. Long Bench: NIPS, 2017: 5767-5677.
|
|
|
|
Viewed |
|
|
|
Full text
|
|
|
|
|
Abstract
|
|
|
|
|
Cited |
|
|
|
|
| |
Shared |
|
|
|
|
| |
Discussed |
|
|
|
|
