|
|
|
| Multi-scale spatial-spectral feature collaborative network for hyperspectral image pansharpening |
Jianxin ZHANG1( ),Xiaojun LI1,2,3,*( ),Xinyu XU4,Yikun LI1,2,3 |
1. Faculty of Geomatics, Lanzhou Jiaotong University, Lanzhou 730070, China 2. National-Local Joint Engineering Research Center of Technologies and Applications for National Geographic State Monitoring, Lanzhou 730070, China 3. Key Laboratory of Science and Technology in Surveying and Mapping, Gansu Province, Lanzhou 730070, China 4. National Cryosphere Desert Data Center, Northwest Institute of Eco-Environment and Resources, Chinese Academy of Sciences, Lanzhou 730000, China |
|
|
|
Abstract A multi-scale spatial-spectral feature collaborative hyperspectral image pansharpening method was constructed in order to enhance the spatial representation capability of hyperspectral images while preserving spectral consistency. A bidirectional pyramid multi-scale alignment structure was adopted to progressively upsample the hyperspectral image and downsample the panchromatic image, generating multi-scale input pairs and achieving spatial-spectral consistency alignment at the input end. A spatial-spectral feature mining and enhancement module was constructed, and multi-scale cross-attention was employed to strengthen spatial-spectral feature extraction. Spatial-spectral feature mining together with sliding-window fusion were combined to realize deep mining and collaborative fusion of heterogeneous features. A spectral attention mechanism was introduced to enhance channel consistency, forming an extraction-mining-fusion-enhancement spatial-spectral collaborative feature flow. A progressive multi-scale spatial-spectral reconstruction module was designed to integrate multi-scale information through stage-wise reconstruction and feature compensation, achieving hierarchical spatial-spectral consistency reconstruction. Experiments were conducted on multiple hyperspectral datasets. Results demonstrate that the proposed method outperforms several existing pansharpening techniques in both spatial detail recovery and spectral fidelity. The quality of hyperspectral image pansharpening can be effectively improved by the proposed method, which exhibits superior spatial detail and spectral fidelity across different scenes.
|
|
Received: 07 December 2025
Published: 21 October 2026
|
|
|
| Fund: 国家重点研发计划资助项目(2022YFB3903604);国家自然科学基金资助项目(41861055);中国博士后基金资助项目(2019M653795). |
|
Corresponding Authors:
Xiaojun LI
E-mail: 11230866@stu.lzjtu.edu.cn;xjli@mail.lzjtu.cn
|
多尺度空谱特征协同的高光谱图像全色锐化网络
为了提高高光谱图像的空间表达能力并保持光谱一致性,构建多尺度空谱特征协同的高光谱图像全色锐化方法. 采用双向金字塔多尺度对齐结构,通过对高光谱图像逐级上采样和对全色图像逐级下采样生成多尺度输入对,实现输入端的空谱一致性对齐. 构建空谱特征挖掘增强模块,通过多尺度交叉注意增强空谱特征提取能力,结合空间-光谱特征挖掘与滑动窗口融合实现异构特征的深层挖掘与协同融合. 引入光谱注意力机制增强通道一致性,形成“提取-挖掘-融合-增强”的空谱协同特征流. 设计渐进式多尺度空谱重建模块,通过逐级重建与特征补偿整合多尺度信息,实现层次化的空谱一致性重建. 实验在多个高光谱数据集上进行验证,结果表明,该方法在空间细节恢复和光谱保真度方面均优于多种现有的全色锐化技术. 利用提出的模型,能够有效提升高光谱图像锐化融合的质量,在不同场景下均表现出更优的空间细节和光谱保真度.
关键词:
全色锐化,
高光谱图像,
多尺度特征,
空谱融合,
特征提取
|
|
| [1] |
陈江浩, 杨军 结合深度可分离卷积的多源遥感融合影像目标检测[J]. 浙江大学学报(工学版), 2025, 59 (12): 2545- 2555 Chen Jianghao, Yang Jun Object detection for multi-source remote sensing fused images based on depthwise separable convolution[J]. Journal of Zhejiang University (Engineering Science), 2025, 59 (12): 2545- 2555
|
|
|
| [2] |
Rajabi R, Zehtabian A, Singh K D, et al Editorial: hyperspectral imaging in environmental monitoring and analysis[J]. Frontiers in Environmental Science, 2024, 11: 1353447
doi: 10.3389/fenvs.2023.1353447
|
|
|
| [3] |
李明婉, 房胜, 李哲 非对称结构的高光谱与激光雷达图像分类模型[J]. 浙江大学学报(工学版), 2025, 59 (12): 2539- 2544, 565 Li Mingwan, Fang Sheng, Li Zhe Asymmetric structure based hyperspectral and LiDAR image classification model[J]. Journal of Zhejiang University (Engineering Science), 2025, 59 (12): 2539- 2544, 565
|
|
|
| [4] |
李强强, 李小军, 李轶鲲, 等 自适应膨胀和结构嵌入的非对称哈希遥感图像检索算法[J]. 地球信息科学学报, 2024, 26 (8): 1926- 1940 Li Qiangqiang, Li Xiaojun, Li Yikun, et al An adaptive dilated and structural embedding asymmetric hashing algorithm for remote sensing image retrieval[J]. Journal of Geo-Information Science, 2024, 26 (8): 1926- 1940
doi: 10.12082/dqxxkx.2024.240168
|
|
|
| [5] |
Tu T M, Su S C, Shyu H C, et al A new look at IHS-like image fusion methods[J]. Information Fusion, 2001, 2 (3): 177- 186
doi: 10.1016/S1566-2535(01)00036-7
|
|
|
| [6] |
Kwarteng P, Chavez A Extracting spectral contrast in Landsat Thematic Mapper image data using selective principal component analysis[J]. Photogrammetric Engineering and Remote Sensing, 1989, 55 (1): 339- 348
|
|
|
| [7] |
Vivone G Robust band-dependent spatial-detail approaches for panchromatic sharpening[J]. IEEE Transactions on Geoscience and Remote Sensing, 2019, 57 (9): 6421- 6433
doi: 10.1109/TGRS.2019.2906073
|
|
|
| [8] |
Vivone G, Restaino R, Dalla Mura M, et al Contrast and error-based fusion schemes for multispectral image pansharpening[J]. IEEE Geoscience and Remote Sensing Letters, 2014, 11 (5): 930- 934
doi: 10.1109/LGRS.2013.2281996
|
|
|
| [9] |
Aiazzi B, Alparone L, Baronti S, et al Context-driven fusion of high spatial and spectral resolution images based on oversampled multiresolution analysis[J]. IEEE Transactions on Geoscience and Remote Sensing, 2002, 40 (10): 2300- 2312
doi: 10.1109/TGRS.2002.803623
|
|
|
| [10] |
Vivone G, Restaino R, Chanussot J Full scale regression-based injection coefficients for panchromatic sharpening[J]. IEEE Transactions on Image Processing, 2018, 27 (7): 3418- 3431
doi: 10.1109/TIP.2018.2819501
|
|
|
| [11] |
Wei Q, Dobigeon N, Tourneret J Y Fast fusion of multi-band images based on solving a sylvester equation[J]. IEEE Transactions on Image Processing, 2015, 24 (11): 4109- 4121
doi: 10.1109/TIP.2015.2458572
|
|
|
| [12] |
Masi G, Cozzolino D, Verdoliva L, et al Pansharpening by convolutional neural networks[J]. Remote Sensing, 2016, 8 (7): 594
doi: 10.3390/rs8070594
|
|
|
| [13] |
He L, Zhu J W, Li J, et al HyperPNN: hyperspectral pansharpening via spectrally predictive convolutional neural networks[J]. IEEE Journal of Selected Topics in Applied Earth Observations and Remote Sensing, 2019, 12 (8): 3092- 3100
doi: 10.1109/JSTARS.2019.2917584
|
|
|
| [14] |
Zhuo Y W, Zhang T J, Hu J F, et al A deep-shallow fusion network with multidetail extractor and spectral attention for hyperspectral pansharpening[J]. IEEE Journal of Selected Topics in Applied Earth Observations and Remote Sensing, 2022, 15: 7539- 7555
doi: 10.1109/JSTARS.2022.3202866
|
|
|
| [15] |
Zheng Y, Li J, Li Y, et al Hyperspectral pansharpening using deep prior and dual attention residual network[J]. IEEE Transactions on Geoscience and Remote Sensing, 2020, 58 (11): 8059- 8076
doi: 10.1109/TGRS.2020.2986313
|
|
|
| [16] |
Bandara W G C, Valanarasu J M J, Patel V M Hyperspectral pansharpening based on improved deep image prior and residual reconstruction[J]. IEEE Transactions on Geoscience and Remote Sensing, 2022, 60: 5520816
doi: 10.36227/techrxiv.14912976
|
|
|
| [17] |
Liu J, Wu Z, Xiao L A spectral diffusion prior for unsupervised hyperspectral image super-resolution[J]. IEEE Transactions on Geoscience and Remote Sensing, 2024, 62: 5528613
|
|
|
| [18] |
Meng Q, Shi W, Li S, et al PanDiff: a novel pansharpening method based on denoising diffusion probabilistic model[J]. IEEE Transactions on Geoscience and Remote Sensing, 2023, 61: 5611317
doi: 10.1109/tgrs.2023.3279864
|
|
|
| [19] |
Li S, Li S, Zhang L Hyperspectral and panchromatic images fusion based on the dual conditional diffusion models[J]. IEEE Transactions on Geoscience and Remote Sensing, 2023, 61: 5526315
doi: 10.1109/tgrs.2023.3321318
|
|
|
| [20] |
Jiang H, Chen Z Transformer-based diffusion and spectral priors model for hyperspectral pansharpening[J]. IEEE Journal of Selected Topics in Applied Earth Observations and Remote Sensing, 2025, 18: 18962- 18977
doi: 10.1109/JSTARS.2025.3590685
|
|
|
| [21] |
Guan P, Lam E Y Multistage dual-attention guided fusion network for hyperspectral pansharpening[J]. IEEE Transactions on Geoscience and Remote Sensing, 2022, 60: 5515214
doi: 10.1109/tgrs.2021.3114552
|
|
|
| [22] |
Yuan Q, Wei Y, Meng X, et al A multiscale and multidepth convolutional neural network for remote sensing imagery pan-sharpening[J]. IEEE Journal of Selected Topics in Applied Earth Observations and Remote Sensing, 2018, 11 (3): 978- 989
doi: 10.1109/JSTARS.2018.2794888
|
|
|
| [23] |
Zhang Y, Yang X, Li H, et al DCPNet: a dual-task collaborative promotion network for pansharpening[J]. IEEE Transactions on Geoscience and Remote Sensing, 2024, 62: 5404016
doi: 10.1109/tgrs.2024.3377635
|
|
|
| [24] |
Liu Z, Lin Y, Cao Y, et al. Swin transformer: hierarchical vision transformer using shifted windows [C]//Proceedings of the IEEE/CVF International Conference on Computer Vision. Montreal: IEEE, 2021: 9992–10002.
|
|
|
| [25] |
Vrabel J Multispectral imagery advanced band sharpening study[J]. Photogrammetric Engineering and Remote Sensing, 2000, 66 (1): 73- 80
|
|
|
| [26] |
Simões M, Bioucas-dias J, Almeida L B, et al A convex formulation for hyperspectral image superresolution via subspace-based regularization[J]. IEEE Transactions on Geoscience and Remote Sensing, 2015, 53 (6): 3373- 3388
doi: 10.1109/TGRS.2014.2375320
|
|
|
| [27] |
Rui X, Cao X, Pang L, et al Unsupervised hyperspectral pansharpening via low-rank diffusion model[J]. Information Fusion, 2024, 107: 102325
doi: 10.1016/j.inffus.2024.102325
|
|
|
| [28] |
Hou S, Xiao S, Qu J, et al Multiscale common-private feature adversarial decoupling network for hyperspectral pansharpening[J]. Knowledge-Based Systems, 2025, 310: 113031
doi: 10.1016/j.knosys.2025.113031
|
|
|
| [29] |
Liu C, Zhang Z, Wang M RAMSF: a novel generic framework for optical remote sensing multimodal spatial-spectral fusion[J]. IEEE Transactions on Geoscience and Remote Sensing, 2025, 63: 5617722
doi: 10.1109/tgrs.2025.3552937
|
|
|
| [30] |
Wald L, Ranchin T, Mangolini M Fusion of satellite images of different spatial resolutions: assessing the quality of resulting images[J]. Photogrammetric Engineering and Remote Sensing, 1997, 63: 691- 699
|
|
|
| [31] |
杨睿哲, 李小军, 张剑鑫, 等 多尺度级联的遥感图像全色锐化融合网络[J]. 激光与光电子学进展, 2025, 62 (22): 306- 314 Yang Ruizhe, Li Xiaojun, Zhang Jianxin, et al Multi-scale cascaded remote sensing image pansharpening fusion network[J]. Laser and Optoelectronics Progress, 2025, 62 (22): 306- 314
doi: 10.3788/LOP250991
|
|
|
| [32] |
Vivone G, Dalla Mura M, Garzelli A, et al A new benchmark based on recent advances in multispectral pansharpening: revisiting pansharpening with classical and emerging pansharpening methods[J]. IEEE Geoscience and Remote Sensing Magazine, 2021, 9 (1): 53- 81
doi: 10.1109/MGRS.2020.3019315
|
|
|
| [33] |
Sara D, Mandava A K, Kumar A, et al Hyperspectral and multispectral image fusion techniques for high resolution applications: a review[J]. Earth Science Informatics, 2021, 14 (4): 1685- 1705
doi: 10.1007/s12145-021-00621-6
|
|
|
| [34] |
Ren K, Sun W, Meng X, et al Fusing China GF-5 hyperspectral data with GF-1, GF-2 and sentinel-2A multispectral data: which methods should be used?[J]. Remote Sensing, 2020, 12 (5): 882
doi: 10.3390/rs12050882
|
|
|
| [35] |
Yokoya N, Grohnfeldt C, Chanussot J Hyperspectral and multispectral data fusion: a comparative review of the recent literature[J]. IEEE Geoscience and Remote Sensing Magazine, 2017, 5 (2): 29- 56
doi: 10.1109/MGRS.2016.2637824
|
|
|
|
Viewed |
|
|
|
Full text
|
|
|
|
|
Abstract
|
|
|
|
|
Cited |
|
|
|
|
| |
Shared |
|
|
|
|
| |
Discussed |
|
|
|
|