An ORLA-OCM UNetFormer with linear self-attention skip enhancement and cross-scale mixing was proposed to address insufficient cross-scale alignment and semantic gaps in skip connections for high-resolution remote sensing image segmentation. The UNetFormer encoder–decoder architecture and its training and inference protocols were kept unchanged, while only the skip connections and cross-scale interactions were modified. Orthogonal random-feature linear self-attention blocks (ORLTB) were serially inserted into the skip paths at the 1/4, 1/8 and 1/16 scales. One-step interaction and reinjection of multi-resolution contexts were performed in a unified embedding space through a linear cross-scale mixer (OCM) before decoder input. Experiments were conducted on the ISPRS Potsdam, ISPRS Vaihingen and LoveDA benchmarks under a unified ResNet-18 backbone, single-model single-scale inference and consistent data processing. Overall accuracy (OA), mean F1, and mIoU reached 91.7%, 93.0% and 87.1% on Potsdam, and 91.2%, 90.7% and 83.7% on Vaihingen compared with the UNetFormer baseline. The mIoU on LoveDA increased to 53.4%. Competitive segmentation performance across all three benchmarks was demonstrated by these results.