Computer and Aut omation Technology |
|
|
|
|
Dual-focal camera continuous digital zoom based onCNN and feature extraction |
Gui-ran HE( ),Qi LI*( ),Hua-jun FENG,Zhi-hai XU,Yue-ting CHEN |
College of Science and Engineering, Zhejiang University, Hangzhou 310058, China |
|
|
Abstract A digital continuous zooming scheme mainly for smartphone dual-focal lens was proposed, which was based on extraction and patch-matching of the convolutional neural network feature map, making full use of the shooting information of two different focal length lenses. The high-resolution information of the long-focal image was migrated to the repairable area of the short-focus image, then the zoomed image was produced accordingly. The simulation and experimental results show that the image produced by the proposed method has higher subjective resolution and better visual clarity, compared with the traditional interpolation and super-resolution results. The method can greatly enhance the imaging quality of the dual-focal lens when the magnification factor is between the two lens. For the texture outside the Short-Focal-Len filed but in the Long-Focal-Len field, the proposed method also performs better than the existing methods. Otherwise, the proposed method has good robustness in dealing with the parallax problem of the dual-focal camera.
|
Received: 24 October 2018
Published: 22 May 2019
|
|
Corresponding Authors:
Qi LI
E-mail: 3130103692@zju.edu.cn;liqi@zju.edu.cn
|
基于CNN特征提取的双焦相机连续数字变焦
设计一种主要针对智能手机双焦镜头的连续数字变焦方案. 该方案基于卷积神经网络特征层提取和特征块匹配,充分利用2个不同焦距镜头的拍摄信息,将长焦镜头图像的高分辨信息迁移到短焦图像的可修复区域,并以修复后的短焦图像为基础进行数字变焦. 仿真和实验结果表明,相比基于传统插值放大的变焦方案和基于单图像超分辨的变焦方案,所提方案的处理结果拥有更高的主观分辨率和视觉清晰度;当用户给定的变焦倍数在长、短焦镜头倍数之间时,所提方案可以显著提升变焦图像的质量;对于处于长焦相机视场外、短焦相机视场内的纹理,修复效果比现有方法更好;该方法的处理结果对长焦、短焦图像的双目视差大小有着很好的鲁棒性.
关键词:
非对称双焦镜头,
连续数字变焦,
卷积神经网络(CNN),
特征提取,
超分辨成像,
图像恢复
|
|
[1] |
YANG J, WRIGHT J, HUANG T S, et al Image super-resolution via sparse representation[J]. IEEE Transactions on Image Processing, 2010, 19 (11): 2861- 73
doi: 10.1109/TIP.2010.2050625
|
|
|
[2] |
TIMOFTE R, SMET V D, GOOL L V. A+: Adjusted anchored neighborhood regression for fast super-resolution [C] // Asian Conference on Computer Vision 2014. Singapore: Springer, 2014: 111-126.
|
|
|
[3] |
TIMOFTE R, DE V, VAN GOOL L. Anchored neighborhod regression for fast example-based super-resolution [C] // Proceedings of the IEEE International Conference on Computer Vision 2013. Sydney: ICCV, 2013: 1920-1927.
|
|
|
[4] |
ZEYDE R, ELAD M, PROTTER M. On single image scale-up using sparse-representations [C] // International Conference on Curves and Surfaces 2010. Avignon: Springer, 2010: 711-730.
|
|
|
[5] |
DONG C, LOY C, HE K, et al Image super-resolution using deep convolutional networks[J]. IEEE Transactions on Pattern Analysis and Machine Intelligence, 2016, 38 (2): 295- 307
doi: 10.1109/TPAMI.2015.2439281
|
|
|
[6] |
KIM J, LEE JK, LEE KM. Accurate image super-resolution using very deep convolutional networks [C] // 2016 IEEE Conference on Computer Vision and Pattern Recognition (CVPR). Las Vegas: [s. n.], 2016: 1646-1654.
|
|
|
[7] |
GOODFELLOW I, POUGET-ABADIE J, MIRZA M, et al. Generative adversarial nets [C] // Advances in Neural Information Processing Systems. Montreal: [s. n.], 2014: 2672-2680.
|
|
|
[8] |
YU S, MOON B, KIM D, et al Continuous digital zooming of asymmetric dual camera images using registration and variational image restoration[J]. Multidimensional Systems and Signal Processing, 2018, 29 (4): 1959- 87
doi: 10.1007/s11045-017-0534-4
|
|
|
[9] |
MOON B, YU S, KO S, et al Continuous digital zooming using local self-similarity-based super-resolution for an asymmetric dual camera system[J]. Journal of the Optical Society of America A-optics Image Science and Vision, 2017, 34 (6): 991- 1003
doi: 10.1364/JOSAA.34.000991
|
|
|
[10] |
MA H, LI Q, XU Z, et al Photo-realistic continuous digital zooming for an asymmetrical dual camera system[J]. Optics and Laser Technology, 2019, 109: 110- 22
doi: 10.1016/j.optlastec.2018.07.056
|
|
|
[11] |
CRIMINISI A, PEREZ P, TOYAMA K Region filling and object removal by exemplar-based image inpainting[J]. IEEE Transactions on Image Processing, 2004, 13 (9): 1200- 12
doi: 10.1109/TIP.2004.833105
|
|
|
[12] |
CHEN H, CHEN C, CHUANG Y. A deep convolutional neural network for continuous zoom with dual cameras [C] // SIGGRAPH Asia 2017 Posters. Bangkok: ACM, 2017: 13.
|
|
|
[13] |
LI C, WAND M. Combining markov random fields and convolutional neural networks for image synthesis [C] // Conference on Computer Vision and Pattern Recognition (CVPR) 2016. Las Vegas: [s. n.], 2016: 2479-2486.
|
|
|
[14] |
EFROS A, LEUNG T K. Texture synthesis by non-parametric sampling [C] // Proceedings of the Seventh IEEE International Conference on Computer Vision 1999. Kerkyra: IEEE, 1999: 1033-1038.
|
|
|
|
Viewed |
|
|
|
Full text
|
|
|
|
|
Abstract
|
|
|
|
|
Cited |
|
|
|
|
|
Shared |
|
|
|
|
|
Discussed |
|
|
|
|