|
|
|
| Image super-resolution reconstruction based on dynamic attention network |
Xiao-qiang ZHAO1,2,3( ),Ze WANG1,Zhao-yang SONG1,Hong-mei JIANG1,2,3 |
1. School of Electrical Engineering and Information Engineering, Lanzhou University of Technology, Lanzhou 730050, China
2. Key Laboratory of Gansu Advanced Control for Industrial Processes, Lanzhou 730050, China
3. National Experimental Teaching Center of Electrical and Control Engineering, Lanzhou University of Technology, Lanzhou 730050, China |
|
|
|
Abstract The image super-resolution algorithm adopts the same processing mode in channels and spatial domains with different importance, which leads to the failure of computing resources to concentrate on important features. Aiming at the above problem, an image super-resolution algorithm based on dynamic attention network was proposed. Firstly, the existing way of equalizing attention mechanisms was changed, and dynamic learning weights were assigned to different attention mechanisms by constructed dynamic attention modules, by which high-frequency information more needed by the network was obtained and high-quality pictures were reconstructed. Secondly, the double butterfly structure was constructed through feature reuse , which fully integrated the information from the two branches of attention and compensated for the missing feature information between the different attention mechanisms. Finally, model evaluation was conducted on Set5, Set14, BSD100, Urban100 and Manga109 datasets. Results show that the proposed algorithm has better overall performance than other mainstream super-resolution algorithms. When the amplification factor was 4, compared with the sub-optimal algorithm, the peak signal-to-noise ratio values were improved by 0.06, 0.07, 0.04, 0.15 and 0.15 dB, respectively, on the above five public test sets.
|
|
Received: 10 October 2022
Published: 31 August 2023
|
|
|
| Fund: 国家自然科学基金资助项目（62263021）；国家重点研发计划资助项目（2020YFB1713600）；甘肃省科技计划资助项目（21YF5GA072, 21JR7RA206） |
基于动态注意力网络的图像超分辨率重建
针对图像超分辨率算法在具有不同重要性的通道和空间域上采取相同的处理方式，导致计算资源无法集中利用到重要特征上的问题，提出基于动态注意力网络的图像超分辨率算法. 该算法改变了现有均等处理注意力机制的方式，通过构建的动态注意力模块对不同的注意力机制赋予动态学习的权重，以获取网络更需要的高频信息，重建高质量图片；通过特征重用的方式构建双蝶式结构，充分融合2个注意力分支的信息，弥补不同注意力机制间所缺失的特征信息. 在Set5、Set14、BSD100、Urban100和Manga109数据集上的模型评估结果表明，相较于其他主流超分辨率算法，本研究所提算法整体性能表现更好. 当放大因子为4时，相较于次优算法，所提算法在5个公开测试集上的峰值信噪比分别提升了0.06、0.07、0.04、0.15、0.15 dB.
关键词：
图像处理,
图像超分辨率,
注意力机制,
动态卷积,
双蝶式结构
|
|
| [1] |
SI W, HAN J, YANG Z, et al. Research on key techniques for super-resolution reconstruction of satellite remote sensing images of transmission lines [C]// Journal of Physics: Conference Series. Sanya: ICAACE, 2021: 012092.
|
|
|
| [2] |
DEEBA F, KUN S, DHAREJO F A, et al Sparse representation based computed tomography images reconstruction by coupled dictionary learning algorithm[J]. IET Image Processing, 2020, 14 (11): 2365- 2375
doi: 10.1049/iet-ipr.2019.1312
|
|
|
| [3] |
ZHANG F, LIU N, CHANG L, et al Edge-guided single facial depth map super-resolution using CNN[J]. IET Image Processing, 2020, 14 (17): 4708- 4716
doi: 10.1049/iet-ipr.2019.1623
|
|
|
| [4] |
LI W, LIAO W Stable super-resolution limit and smallest singular value of restricted Fourier matrices[J]. Applied and Computational Harmonic Analysis, 2021, 51: 118- 156
doi: 10.1016/j.acha.2020.10.004
|
|
|
| [5] |
吴世豪, 罗小华, 张建炜, 等 基于FPGA的新边缘指导插值算法硬件实现[J]. 浙江大学学报: 工学版, 2018, 52 (11): 2226- 2232
WU Shi-hao, LUO Xiao-hua, ZHANG Jian-wei, et al FPGA-based hardware implementation of new edge-directed interpolation algorithm[J]. Journal of Zhejiang University: Engineering Science, 2018, 52 (11): 2226- 2232
|
|
|
| [6] |
段然, 周登文, 赵丽娟, 等 基于多尺度特征映射网络的图像超分辨率重建[J]. 浙江大学学报: 工学版, 2019, 53 (7): 1331- 1339
DUAN Ran, ZHOU Deng-wen, ZHAO Li-juan, et al Image super-resolution reconstruction based on multi-scale feature mapping network[J]. Journal of Zhejiang University: Engineering Science, 2019, 53 (7): 1331- 1339
|
|
|
| [7] |
DONG C, LOY C C, HE K, et al. Learning a deep convolutional network for image super-resolution [C]// European Conference on Computer Vision. Zurich: ECCV, 2014: 184-199.
|
|
|
| [8] |
DONG C, LOY C C, HE K, et al Image super-resolution using deep convolutional networks[J]. IEEE Transactions on Pattern Analysis and Machine Intelligence, 2015, 38 (2): 295- 307
|
|
|
| [9] |
LIM B, SON S, KIM H, et al. Enhanced deep residual networks for single image super-resolution [C]// Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition Workshops. Honolulu: CVPRW, 2017: 136-144.
|
|
|
| [10] |
TAI Y, YANG J, LIU X, et al. Memnet: a persistent memory network for image restoration [C]// Proceedings of the IEEE International Conference on Computer Vision. Venice: ICCV, 2017: 4539-4547.
|
|
|
| [11] |
AHN N, KANG B, SOHN K A. Fast, accurate, and lightweight super-resolution with cascading residual network [C]// Proceedings of the European Conference on Computer Vision. Munich: ECCV, 2018: 252-268.
|
|
|
| [12] |
WANG C, LI Z , SHI J. Lightweight image super-resolution with adaptive weighted learning network [EB/OL]. [2019-04-04]. https://arxiv.org/abs/1904.02358.
|
|
|
| [13] |
WOO S, PARK J, LEE J Y, et al. Cbam: convolutional block attention module [C]// Proceedings of the European Conference on Computer Vision. Munich: ECCV, 2018: 3-19.
|
|
|
| [14] |
DAI T, CAI J, ZHANG Y, et al. Second-order attention network for single image super-resolution [C]// Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition. Long Beach: CVPR, 2019: 11065-11074.
|
|
|
| [15] |
ZHANG Y, LI K, LI K, et al. Residual non-local attention networks for image restoration[EB/OL]. [2019-03-24]. https://arxiv.org/abs/1903.10082.
|
|
|
| [16] |
JIA X, BRABANDERE D B, TUYTELAARS T, et al. Dynamic filter networks for predicting unobserved views [C]// Proceedings of the European Conference on Computer Vision 2016 Workshops. Amsterdam: ECCVW, 2016: 1-2.
|
|
|
| [17] |
YANG B, BENDER G, LE Q V, et al. Condconv: conditionally parameterized convolutions for efficient inference [C]// Advances in Neural Information Processing Systems. 2019, 32: 767-779.
|
|
|
| [18] |
CHEN Y, DAI X, LIU M, et al. Dynamic convolution: attention over convolution kernels [C]// Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition. Seattle: CVPR, 2020: 11030-11039.
|
|
|
| [19] |
ZHANG Y, ZHANG J, WANG Q, et al. Dynet: dynamic convolution for accelerating convolutional neural networks [EB/OL]. [2020-04-22]. https://arxiv.org/abs/2004.10694.
|
|
|
| [20] |
ZHANG Y, LI K, LI K, et al. Image super-resolution using very deep residual channel attention networks [C]// Proceedings of the European Conference on Computer Vision. Munich: ECCV, 2018: 286-301.
|
|
|
| [21] |
CHEN H, GU J, ZHANG Z. Attention in attention network for image super-resolution [EB/OL]. [2021-04-19]. https://arxiv.org/abs/2104.09497.
|
|
|
| [22] |
TIMOFTE R, AGUSTSSON E, VAN G L, et al. Ntire 2017 challenge on single image super-resolution: methods and results [C]// Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition Workshops. Hawaii: CVPRW, 2017: 114-125.
|
|
|
| [23] |
BEVILACQUA M, ROUMY A, GUILLEMOT C, et al. Low-complexity single-image super-resolution based on nonnegative neighbor embedding [C]// Proceedings British Machine Vision Conference. Surrey: Springer, 2012: 1-10.
|
|
|
| [24] |
ZEYDE R, ELAD M, PROTTER M. On single image scale-up using sparse-representations [C]// International Conference on Curves and Surfaces. Avignon: ICCS, 2010: 711-730.
|
|
|
| [25] |
MARTIN D, FOWLKES C, TAL D, et al. A database of human segmented natural images and its application to evaluating segmentation algorithms and measuring ecological statistics [C]// Proceedings 18th IEEE International Conference on Computer Vision. Vancouver: ICCV, 2001: 416-423.
|
|
|
| [26] |
HUANG J B, SINGH A, AHUJA N. Single image super-resolution from transformed self-exemplars [C]// Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition. Santiago: IEEE, 2015: 5197-5206.
|
|
|
| [27] |
MATSUI Y, ITO K, ARAMAKI Y, et al Sketch-based manga retrieval using manga109 dataset[J]. Multimedia Tools and Applications, 2017, 76 (20): 21811- 21838
doi: 10.1007/s11042-016-4020-z
|
|
|
| [28] |
FEI Y, LIAN F H, YAN Y. An improved PSNR algorithm for objective video quality evaluation [C]// 2007 Chinese Control Conference. Zhangjiajie: CCC, 2007: 376-380.
|
|
|
| [29] |
WANG Z, BOVIK A C, SHEIKH H R, et al Image quality assessment: from error visibility to structural similarity[J]. IEEE Transactions on Image Processing, 2004, 13 (4): 600- 612
doi: 10.1109/TIP.2003.819861
|
|
|
| [30] |
KINGMA D P, BA J. Adam: a method for stochastic optimization [EB/OL]. [2014-12-22]. https://arxiv.org/abs/1412.6980.
|
|
|
| [31] |
HUI Z, GAO X, YANG Y, et al. Lightweight image super-resolution with information multi-distillation network [C]// Proceedings of the 27th ACM International Conference on Multimedia. Ottawa: ACM, 2019: 2024-2032.
|
|
|
| [32] |
FANG F, LI J, ZENG T Soft-edge assisted network for single image super-resolution[J]. IEEE Transactions on Image Processing, 2020, 29: 4656- 4668
doi: 10.1109/TIP.2020.2973769
|
|
|
| [33] |
LIU Y, JIA Q, FAN X, et al Cross-srn: structure-preserving super-resolution network with cross convolution[J]. IEEE Transactions on Circuits and Systems for Video Technology, 2021, 32 (8): 4927- 4939
|
|
|
|
Viewed |
|
|
|
Full text
|
|
|
|
|
Abstract
|
|
|
|
|
Cited |
|
|
|
|
| |
Shared |
|
|
|
|
| |
Discussed |
|
|
|
|
