|
|
|
| Remote sensing semantic segmentation network with linear attention and cross-scale fusion |
Jinghong ZHANG1( ),Yubin SHAO1,*(),Jingmin TANG1,Qingzhi DU1,Hua LONG1,2,Zetao ZHANG2 |
1. Faculty of Information Engineering and Automation, Kunming University of Science and Technology, Kunming 650500, China
2. Yunnan Provincial Key Laboratory of Media Convergence, Kunming 650228, Kunming, China |
|
|
|
Abstract An ORLA-OCM UNetFormer with linear self-attention skip enhancement and cross-scale mixing was proposed to address insufficient cross-scale alignment and semantic gaps in skip connections for high-resolution remote sensing image segmentation. The UNetFormer encoder–decoder architecture and its training and inference protocols were kept unchanged, while only the skip connections and cross-scale interactions were modified. Orthogonal random-feature linear self-attention blocks (ORLTB) were serially inserted into the skip paths at the 1/4, 1/8 and 1/16 scales. One-step interaction and reinjection of multi-resolution contexts were performed in a unified embedding space through a linear cross-scale mixer (OCM) before decoder input. Experiments were conducted on the ISPRS Potsdam, ISPRS Vaihingen and LoveDA benchmarks under a unified ResNet-18 backbone, single-model single-scale inference and consistent data processing. Overall accuracy (OA), mean F1, and mIoU reached 91.7%, 93.0% and 87.1% on Potsdam, and 91.2%, 90.7% and 83.7% on Vaihingen compared with the UNetFormer baseline. The mIoU on LoveDA increased to 53.4%. Competitive segmentation performance across all three benchmarks was demonstrated by these results.
|
|
Received: 23 October 2025
Published: 21 October 2026
|
|
|
| Fund: 云南省媒体融合重点实验室资助项目（220245203）. |
|
Corresponding Authors:
Yubin SHAO
E-mail: 2783707546@qq.com;shaoyubin999@qq.com
|
线性注意力与跨尺度融合的遥感语义分割网络
针对遥感高分辨率影像语义分割中存在的跨尺度信息对齐不足、跳跃连接语义落差明显的问题，提出线性自注意力跳跃增强与跨尺度混合网络ORLA-OCM UNetFormer. 在保持 UNetFormer 的编码-解码框架及其训练与推理协议不变的前提下，仅对跳跃连接和跨尺度交互部分进行改进：在1/4、1/8、1/16 3个尺度的跳跃侧串行引入基于正交随机特征的线性自注意力块（ORLTB），并在进入解码器前通过线性跨尺度混合器（OCM）对多分辨率上下文在统一嵌入维度内进行一次性交互与回注. 实验在ISPRS Potsdam、ISPRS Vaihingen与LoveDA 3个基准数据集上，在统一的ResNet-18主干、单模型单尺度推理策略与一致的数据处理流程下进行评测. 结果显示，相对UNetFormer基线，Potsdam上总体精度（OA）、平均F1、mIoU分别达到91.7%、93.0%、87.1%，Vaihingen上为91.2%、90.7%、83.7%，LoveDA上mIoU提升至53.4%，表明所提方法在3个基准数据集上均取得了具有竞争力的分割性能.
关键词：
遥感图像,
语义分割,
线性自注意力,
正交随机特征,
跨尺度特征融合
|
|
| [1] |
ISPRS. 2D semantic labeling contest: Potsdam data [DS/OL]. (2024-11-11) [2025-10-02]. https://www.isprs.org/resources/datasets/benchmarks/UrbanSemLab/2d-sem-label-potsdam.aspx.
|
|
|
| [2] |
ISPRS. 2D semantic labeling: Vaihingen data [DS/OL]. (2024-11-11) [2025-10-02]. https://www.isprs.org/resources/datasets/benchmarks/UrbanSemLab/2d-sem-label-vaihingen.aspx.
|
|
|
| [3] |
Wang J, Zheng Z, Ma A, et al. LoveDA: a remote sensing land-cover dataset for domain adaptive semantic segmentation [PP/OL].arXiv (2021-10-17)[2025-10-02]. https://arxiv.org/abs/2110.08733.
|
|
|
| [4] |
Oršić M, Šegvić S Efficient semantic segmentation with pyramidal fusion[J]. Pattern Recognition, 2021, 110: 107611
doi: 10.1016/j.patcog.2020.107611
|
|
|
| [5] |
Zhou X, Wei X. Feature aggregation network for building extraction from high-resolution remote sensing images [C]// Pacific Rim International Conference on Artificial Intelligence. Singapore: Springer, 2023: 105–116.
|
|
|
| [6] |
Yang G, Zhang Q, Zhang G EANet: edge-aware network for the extraction of buildings from aerial images[J]. Remote Sensing, 2020, 12 (13): 2161
doi: 10.3390/rs12132161
|
|
|
| [7] |
Li R, Zheng S, Duan C, et al Multistage attention ResU-net for semantic segmentation of fine-resolution remote sensing images[J]. IEEE Geoscience and Remote Sensing Letters, 2022, 19: 8009205
doi: 10.1109/lgrs.2021.3063381
|
|
|
| [8] |
Zheng Z, Zhong Y, Wang J, et al. Foreground-aware relation network for geospatial object segmentation in high spatial resolution remote sensing imagery [C]//Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition. Seattle: IEEE, 2020: 4095–4104.
|
|
|
| [9] |
Ma A, Wang J, Zhong Y, et al FactSeg: foreground activation-driven small object semantic segmentation in large-scale remote sensing imagery[J]. IEEE Transactions on Geoscience and Remote Sensing, 2022, 60: 5606216
doi: 10.1109/tgrs.2021.3097148
|
|
|
| [10] |
Wang L, Li R, Wang D, et al Transformer meets convolution: a bilateral awareness network for semantic segmentation of very fine resolution urban scene images[J]. Remote Sensing, 2021, 13 (16): 3065
doi: 10.3390/rs13163065
|
|
|
| [11] |
Li R, Zheng S, Zhang C, et al ABCNet: attentive bilateral contextual network for efficient semantic segmentation of Fine-Resolution remotely sensed imagery[J]. ISPRS Journal of Photogrammetry and Remote Sensing, 2021, 181: 84- 98
doi: 10.1016/j.isprsjprs.2021.09.005
|
|
|
| [12] |
Chen J, Lu Y, Yu Q, et al. TransUNet: transformers make strong encoders for medical image segmentation [PP/OL]. arXiv (2021-02-08)[2025-10-02]. https://arxiv.org/abs/2102.04306.
|
|
|
| [13] |
Strudel R, Garcia R, Laptev I, et al. Segmenter: transformer for semantic segmentation [C]//Proceedings of the IEEE/CVF International Conference on Computer Vision. Montreal: IEEE, 2022: 7242–7252.
|
|
|
| [14] |
Liu Z, Lin Y, Cao Y, et al. Swin transformer: hierarchical vision transformer using shifted windows [C]//Proceedings of the IEEE/CVF International Conference on Computer Vision. Montreal: IEEE, 2022: 9992–10002.
|
|
|
| [15] |
Wang L, Li R, Duan C, et al A novel transformer based semantic segmentation scheme for fine-resolution remote sensing images[J]. IEEE Geoscience and Remote Sensing Letters, 2022, 19: 6506105
doi: 10.1109/lgrs.2022.3143368
|
|
|
| [16] |
Wadekar S N, Chaurasia A. MobileViTv3: mobile-friendly vision transformer with simple and effective fusion of local, global and input features [PP/OL]. arXiv (2022-09-30)[2025-10-02]. https://arxiv.org/abs/2209.15159.
|
|
|
| [17] |
Wang L, Li R, Zhang C, et al UNetFormer: a UNet-like transformer for efficient semantic segmentation of remote sensing urban scene imagery[J]. ISPRS Journal of Photogrammetry and Remote Sensing, 2022, 190: 196- 214
doi: 10.1016/j.isprsjprs.2022.06.008
|
|
|
| [18] |
Ma X, Zhang X, Pun M O RS3Mamba: visual state space model for remote sensing image semantic segmentation[J]. IEEE Geoscience and Remote Sensing Letters, 2024, 21: 6011405
|
|
|
| [19] |
Xue R, Li C, Peng W, et al Non-exemplar incremental ISAR target classification via mix-mamba feature adjustment network[J]. Remote Sensing, 2025, 17 (12): 2081
doi: 10.3390/rs17122081
|
|
|
| [20] |
Dong Z, Gao G, Liu T, et al Distilling segmenters from CNNs and transformers for remote sensing images’ semantic segmentation[J]. IEEE Transactions on Geoscience and Remote Sensing, 2023, 61: 5613814
doi: 10.1109/tgrs.2023.3290411
|
|
|
| [21] |
Katharopoulos A, Vyas A, Pappas N, et al. Transformers are RNNs: fast autoregressive transformers with linear attention [PP/OL]. arXiv (2020-06-29)[2025-10-02]. https://arxiv.org/abs/2006.16236.
|
|
|
| [22] |
Yang G, Lei J, Zhu Z, et al. AFPN: asymptotic feature pyramid network for object detection [C]//Proceedings of the IEEE International Conference on Systems, Man, and Cybernetics. Honolulu: IEEE, 2024: 2184–2189.
|
|
|
| [23] |
PyTorch. AdaptiveAvgPool2d: PyTorch documentation [EB/OL]. (2024-11-06)[2025-10-02]. https://pytorch.org/docs/stable/generated/torch.nn.AdaptiveAvgPool2d.html.
|
|
|
| [24] |
魏新雨, 饶蕾, 范光宇, 等 用于无人机遥感图像的高精度实时语义分割网络[J]. 浙江大学学报(工学版), 2025, 59 (7): 1411- 1420
Wei Xinyu, Rao Lei, Fan Guangyu, et al High-precision real-time semantic segmentation network for UAV remote sensing images[J]. Journal of Zhejiang University (Engineering Science), 2025, 59 (7): 1411- 1420
doi: 10.3785/j.issn.1008-973X.2025.07.009
|
|
|
|
Viewed |
|
|
|
Full text
|
|
|
|
|
Abstract
|
|
|
|
|
Cited |
|
|
|
|
| |
Shared |
|
|
|
|
| |
Discussed |
|
|
|
|
