|
|
Image super-resolution reconstruction algorithm based on multi-level continuous encoding and decoding |
Zhao-yang SONG1,2,3( ),Xiao-qiang ZHAO1,2,3,*( ),Yong-yong HUI1,2,3,Hong-mei JIANG1,2,3 |
1. College of Electrical Engineering and Information Engineering, Lanzhou University of Technology, Lanzhou 730050, China 2. Key Laboratory of Gansu Advanced Control for Industrial Processes, Lanzhou 730050, China 3. National Experimental Teaching Center of Electrical and Control Engineering, Lanzhou University of Technology, Lanzhou 730050, China |
|
|
Abstract A new image super-resolution reconstruction algorithm was proposed, aiming at the problem that the image super-resolution reconstruction algorithms by using convolutional neural network as the model framework was difficult to extract multi-level feature information inside a low-resolution image, resulting in the lack of rich details in the reconstructed image. The proposed algorithm extracted shallow features from low-resolution images through initial convolution layer. Image features of different levels in a low-resolution image were obtained through a plurality of end-to-end connected multi-level continuous encoding and decoding attention residual modules, the weights of these features were generated according to different extraction difficulties, and the image features of different levels were recalibrated to obtain rich detailed features in the image. Through the up-sampling module and reconstruction convolution layer, the extracted rich detailed features and shallow features were reconstructed into high-resolution images. The comparative test results on the test sets of Set5, Set14, BSD100 and Urban100 show that the image reconstructed by the proposed algorithm is superior to the image reconstructed by a mainstream algorithm in objective evaluation index and visual effect.
|
Received: 24 May 2022
Published: 16 October 2023
|
|
Fund: 国家重点研发计划资助项目(2020YFB1713600);国家自然科学基金资助项目(61763029);甘肃省科技计划资助项目(21YF5GA072, 21JR7RA206);甘肃省教育厅产业支撑计划资助项目(2021CYZC-02) |
Corresponding Authors:
Xiao-qiang ZHAO
E-mail: szy@lut.edu.cn;xqzhao@lut.edu.cn
|
基于多级连续编码与解码的图像超分辨率重建算法
以卷积神经网络为模型框架的图像超分辨率重建算法难以提取低分辨率图像内部的多层次特征信息, 导致重建图像缺少丰富细节, 为此提出新的图像超分辨率重建算法. 所提算法通过初始卷积层从低分辨率图像提取浅层特征; 通过多个端对端连接的多级连续编码与解码的注意力残差模块获取低分辨率图像内部不同层级的图像特征, 按照不同的提取难度生成这些特征的权重, 重新校准不同层次的图像特征, 获取图像内部丰富的细节特征;通过上采样模块和重建卷积层将提取到的丰富细节特征和浅层特征重建成高分辨率图像. 在Set5、Set14、BSD100和Urban100测试集上进行的对比测试结果表明,使用所提算法重建的图像在客观评价指标、视觉效果上均优于使用主流算法重建的图像.
关键词:
超分辨率重建,
卷积神经网络,
多级连续编码与解码,
注意力,
多层次特征信息
|
|
[20] |
ZHANG Y, TIAN Y, KONG Y, et al. Residual dense network for image super-resolution [C]// Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition. Salt Lake City: IEEE, 2018: 2472-2481.
|
|
|
[21] |
ZHANG Y, LI K, LI K, et al. Image super-resolution using very deep residual channel attention networks [C]// European Conference on Computer Vision. [S.l.]: Springer, 2018: 294-301.
|
|
|
[22] |
CHEN H, GU J, ZHANG Z. Attention in attention network for image super-resolution [EB/OL]. [2022-04-21]. https://arxiv.org/pdf/2104.09497.pdf.
|
|
|
[23] |
AGUSTSSON E, TIMOFTE R. NTIRE 2017 challenge on single image super-resolution: dataset and study [C]// Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition Workshops. Honolulu: IEEE, 2017: 126-135.
|
|
|
[24] |
BEVILACQUA M, ROUMY A, GUILLEMOT C, et al. Low-complexity single-image super-resolution based on nonnegative neighbor embedding [C]// Proceedings British Machine Vision Conference. Surrey: [s.n.], 2012: 1-10.
|
|
|
[25] |
ZEYDE R, ELAD M, PROTTER M, et al. On single image scale-up using sparse-representations [C]// International Conference on Curves and Surfaces. [S.l.]: Springer, 2010: 711-730.
|
|
|
[26] |
MARTIN D, FOWLKES C, TAL D, et al. A database of human segmented natural images and its application to evaluating segmentation algorithms and measuring ecological statistics [C]// Proceedings of the IEEE International Conference on Computer Vision. Vancouver: IEEE, 2001: 416-423.
|
|
|
[27] |
HUANG J, SINGH A, AHUJA N, et al. Single image super-resolution from transformed self-exemplars [C]// Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition. Boston: IEEE, 2015: 5197-5206
|
|
|
[28] |
KINGMA D P, BA J L. ADAM: a method for stochastic optimization [EB/OL]. [2022-04-21]. https://arxiv.org/pdf/1412.6980.pdf.
|
|
|
[29] |
WANG Z, BOVIK A C, SHEIKH H R, et al Image quality assessment: from error visibility to structural similarity[J]. IEEE Transactions on Image Processing, 2004, 13 (4): 600- 612
doi: 10.1109/TIP.2003.819861
|
|
|
[30] |
HARIS M, SHAKHNAROVICH G, UKITA N Deep back-projection networks for super-resolution[J]. IEEE Transactions on Pattern Analysis and Machine Intelligence, 2021, 43 (12): 4323- 4337
doi: 10.1109/TPAMI.2020.3002836
|
|
|
[31] |
LIU J, TANG J, WU G. Residual feature distillation network for lightweight image super-resolution [C]// European Conference on Computer Vision. [S.l.]: Springer, 2020: 41-55.
|
|
|
[1] |
CHEN H, HE X, QING L, et al Real-world single image super-resolution: a brief review[J]. Information Fusion, 2022, 79: 124- 145
doi: 10.1016/j.inffus.2021.09.005
|
|
|
[2] |
REBECQ H, RANFTL R, KOLTUN V, et al. Events-to-video: bringing modern computer vision to event cameras [C]// Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition. Long Beach: IEEE, 2019: 3857-3866.
|
|
|
[3] |
WANG Z, JIANG K, YI P, et al Ultra-dense GAN for satellite imagery super-resolution[J]. Neurocomputing, 2020, 398: 328- 373
doi: 10.1016/j.neucom.2019.03.106
|
|
|
[4] |
LIU H, GU Y, WANG T, et al Satellite video super-resolution based on adaptively spatiotemporal neighbors and nonlocal similarity regularization[J]. IEEE Transactions on Geoscience and Remote Sensing, 2020, 58 (12): 8372- 8383
doi: 10.1109/TGRS.2020.2987400
|
|
|
[5] |
YUE L, SHEN H, LI J, et al Image super-resolution: the techniques, applications, and future[J]. Signal Processing, 2016, 128: 389- 408
doi: 10.1016/j.sigpro.2016.05.002
|
|
|
[6] |
DENG L, YU D. Deep learning: methods and applications [M]. [S.l.]: Now Foundations and Trends, 2014: 197-387.
|
|
|
[7] |
JORDAN M I, MITCHELL T M Machine learning: trends, perspectives, and prospects[J]. Science, 2015, 349 (6245): 255- 260
doi: 10.1126/science.aaa8415
|
|
|
[8] |
LECUN Y, BENGIO Y, HINTON G Deep learning[J]. Nature, 2015, 521: 436- 444
doi: 10.1038/nature14539
|
|
|
[9] |
ANWAR S, KHAN S, BARNES N A deep journey into super-resolution: a survey[J]. ACM Computing Surveys, 2020, 53 (3): 1- 34
|
|
|
[10] |
WANG Z, CHEN J, HOI S C H Deep learning for image super-resolution: a survey[J]. IEEE Transactions on Pattern Analysis and Machine Intelligence, 2021, 43 (10): 3365- 3387
doi: 10.1109/TPAMI.2020.2982166
|
|
|
[11] |
SONG Z, ZHAO X, JIANG H Gradual deep residual network for super-resolution[J]. Multimedia Tools and Applications, 2021, 80: 9765- 9778
doi: 10.1007/s11042-020-10152-9
|
|
|
[12] |
SONG Z, ZHAO X, HUI Y, et al Progressive back-projection network for COVID-CT super-resolution[J]. Computer Methods and Programs in Biomedicine, 2021, 208: 106193
doi: 10.1016/j.cmpb.2021.106193
|
|
|
[13] |
DONG C, LOY C C, HE K, et al Image super-resolution using deep convolutional networks[J]. IEEE Transactions on Pattern Analysis and Machine Intelligence, 2015, 38 (2): 295- 307
|
|
|
[14] |
DONG C, LOY C C, TANG X, et al. Accelerating the super-resolution convolutional neural network [C]// European Conference on Computer Vision. [S.l.]: Springer, 2016: 391-407.
|
|
|
[15] |
SHI W, CABALLERO J, HUSZÁR F, et al. Real-time single image and video super-resolution using an efficient sub-pixel convolutional neural network [C]// Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition. Las Vegas: IEEE, 2016: 1874-1883.
|
|
|
[16] |
KIM J, LEE J K, LEE K M. Accurate image super-resolution using very deep convolutional networks [C]// Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition. Las Vegas: IEEE, 2016: 1646-1654.
|
|
|
[17] |
KIM J, LEE J K, LEE K M. Deeply-recursive convolutional network for image super-resolution [C]// Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition. Las Vegas: IEEE, 2016: 1637-1645.
|
|
|
[18] |
LIM B, SON S, KIM H, et al. Enhanced deep residual networks for single image super-resolution [C]// Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition. Honolulu: IEEE, 2017: 136-144.
|
|
|
|
Viewed |
|
|
|
Full text
|
|
|
|
|
Abstract
|
|
|
|
|
Cited |
|
|
|
|
|
Shared |
|
|
|
|
|
Discussed |
|
|
|
|