Real-time detection algorithm for railway foreign objects in complex weather conditions based on improved RT-DETR

doi:10.3785/j.issn.1008-973X.2026.10.009

Journal of ZheJiang University (Engineering Science)

2026, Vol. 60

Issue (10): 2165-2175 DOI: 10.3785/j.issn.1008-973X.2026.10.009

Real-time detection algorithm for railway foreign objects in complex weather conditions based on improved RT-DETR

Hongxia NIU1,2(

),Dingchao FENG1,2(

),Tao HOU1

1. School of Automation and Electrical Engineering, Lanzhou Jiaotong University, Lanzhou 730070, China
2. Key Laboratory of Plateau Traffic Information Engineering and Control of Gansu Province, Lanzhou Jiaotong University, Lanzhou 730070, China

Download:

HTML

PDF(7802KB) HTML
Export: BibTeX | EndNote (RIS)

Abstract

A railway foreign object detection algorithm for complex weather conditions based on an improved RT-DETR, named FRP-DETR, was proposed. A feature complementary mapping module (FCM) and Pzconv units were introduced, which built complementary paths between shallow spatial details and deep semantic information to compensate for the limitations of single-scale feature representation, with the two components working collaboratively to enhance the perception capabilities of small and edge targets. A railway perception modulation fusion module (RMFM) was introduced, which was based on adaptive channel attention and spatial modulation mechanisms to enhance the model’s response to key semantic information in railway scenes. The original downsampling module was replaced with pinwheel-shaped convolution (PSConv), which enhanced edge texture extraction through multi-directional asymmetric padding and separable convolution. A railway foreign object detection dataset containing four weather conditions was constructed based on the SaMam style transfer method, by transferring sunny railway foreign object images to rainy, foggy, and snowy scenes. Experimental results showed that compared to the original RT-DETR-R18 model, this method achieved improvements of 1.65 percentage points and 3.4 percentage points in mAP@0.5 and mAP@0.5:0.95, respectively, with a 63.5% reduction in parameter count and an inference speed of 88 frames per second. The results verified that FRP-DETR achieved high accuracy, lightweight design, and real-time performance in railway foreign object detection under complex weather conditions.

Key words： railway foreign object detection complex weather feature fusion real-time detection style transfer

Received: 28 August 2025 Published: 28 July 2026

CLC:

TP 391.4

Fund: 甘肃省重点研发计划-工业类项目(23YFGA0049)；兰州市人才创新创业项目(2023-RC-13)；甘肃省科技专员专项(24CXGA020).

	Service
	E-mail this article
	Add to my bookshelf
	Add to citation manager
	E-mail Alert
	RSS
	Articles by authors
	Hongxia NIU
	Dingchao FENG
	Tao HOU

Cite this article:

Hongxia NIU,Dingchao FENG,Tao HOU. Real-time detection algorithm for railway foreign objects in complex weather conditions based on improved RT-DETR. Journal of ZheJiang University (Engineering Science), 2026, 60(10): 2165-2175.

URL:

https://www.zjujournals.com/eng/10.3785/j.issn.1008-973X.2026.10.009 OR https://www.zjujournals.com/eng/Y2026/V60/I10/2165

基于改进RT-DETR的复杂天气下铁路异物实时检测算法

提出基于改进RT-DETR的复杂天气下铁路异物检测算法FRP-DETR. 引入特征互补映射模块（FCM）与Pzconv单元，通过构建浅层空间细节与深层语义信息弥补单一尺度特征表达的局限性，两者协同增强小目标和边缘目标的感知能力；引入铁路感知调制融合模块（RMFM），基于自适应通道注意力与空间调制机制，增强模型对铁路场景关键语义信息的响应能力；以风车状卷积（PSConv）替代原始下采样模块，通过多方向非对称填充与分离卷积强化边缘纹理提取. 基于SaMam风格迁移方法，将晴天铁路异物图像迁移至雨天、雾天、雪天等场景，构建了包含4种天气条件的铁路异物检测数据集. 实验结果表明，相比原RT-DETR-R18模型，所提方法在mAP@0.5和mAP@0.5:0.95上分别提升了1.65个百分点和3.4个百分点，参数量降低63.5%，推理速度达到88 帧/s，实验结果验证了FRP-DETR在复杂天气下的铁路异物检测任务中兼具高精度、轻量化与实时性优势.

关键词： 铁路异物检测, 复杂天气, 特征融合, 实时检测, 风格迁移

Fig.1 RT-DETR structure diagram

Fig.2 Structure diagram of feature complementary mapping module (FCM)

Fig.3 Structure diagram of Pzconv feature extraction unit

Fig.4 Structure diagram of railway-aware modulation fusion module (RMFM)

Fig.5 Structure diagram of pinwheel-shaped convolutional module (PSConv)

Fig.6 Overall architecture of improved FRP-DETR model

Tab.1 Experimental environment configuration

Tab.2 Experimental parameter configuration

Fig.7 Structure diagram of SaMam style transfer model

Fig.8 Sample quantity distribution statistics for seven target categories in railway foreign object detection dataset

Fig.9 Example of image after style transfer

Tab.3 Results of image quality evaluation based on SaMam style transfer

Tab.4 Comparison of ablation experiment results for each module in improved model

Tab.5 Comparison of results from different object detection algorithms

Tab.6 Analysis results of channel fusion weight α

Tab.7 Effectiveness verification of longitudinal offset improvement for PSConv module

Tab.8 MFM vs RMFM module performance comparison

Fig.10 Comparative analysis of feature response heatmaps between MFM and RMFM modules

Fig.11 Comparison curves of key performance metrics during training between Baseline model and improved model

Fig.12 Detection result comparison between Baseline model and proposed method under clear and snowy weather conditions

Fig.13 Detection result comparison between Baseline model and proposed method under sand and rainy weather conditions

Fig.14 Feature attention heatmap comparison between Baseline model and improved model in night time low-illumination environment

Fig.15 Feature attention heatmap comparison between Baseline model and improved model in dense target occlusion scenarios

Tab.9 Results of generalizability experiments in VisDrone2019 dataset


[1]	DONG W, FANG W, JIANG X, et al Railway safety under increasing speed: train drivers’ hazard perception of foreign object intrusion on railway tracks[J]. International Journal of Industrial Ergonomics, 2025, 105: 103684 doi: 10.1016/j.ergon.2024.103684

[2]	侯涛, 宝才文, 陈燕楠基于自适应高斯混合模型的铁轨异物入侵检测研究[J]. 光电子·激光, 2022, 33 (4): 403- 413 HOU Tao, BAO Caiwen, CHEN Yannan Research on detection of foreign object intrusion in railroad tracks based on AGMM[J]. Journal of OptoElectronics Lasers, 2022, 33 (4): 403- 413

[3]	PAN H, LI Y, WANG H, et al Railway obstacle intrusion detection based on convolution neural network multitask learning[J]. Electronics, 2022, 11 (17): 2697 doi: 10.3390/electronics11172697

[4]	何文玉, 杨杰, 张天露基于深度学习的轨道异物入侵检测算法[J]. 计算机工程与设计, 2020, 41 (12): 3376- 3383 HE Wenyu, YANG Jie, ZHANG Tianlu Orbital foreign object intrusion detection algorithm based on deep learning[J]. Computer Engineering and Design, 2020, 41 (12): 3376- 3383 doi: 10.16208/j.issn1000-7024.2020.12.012

[5]	鞠逸凡. 基于深度学习的铁路周界异物入侵检测方法研究[D]. 青岛: 青岛科技大学, 2024 JU Yifan. Research on railroad perimeter foreign object intrusion detection method based on deep learing [D]. Qingdao: Qingdao University of Science and Technology, 2024.

[6]	ZHANG Z, CHEN P, HUANG Y, et al Railway obstacle intrusion warning mechanism integrating YOLO-based detection and risk assessment[J]. Journal of Industrial Information Integration, 2024, 38: 100571 doi: 10.1016/j.jii.2024.100571

[7]	REN S, HE K, GIRSHICK R, et al Faster R-CNN: towards real-time object detection with region proposal networks[J]. IEEE Transactions on Pattern Analysis and Machine Intelligence, 2017, 39 (6): 1137- 1149 doi: 10.1109/TPAMI.2016.2577031

[8]	SUN P, ZHANG R, JIANG Y, et al Sparse R-CNN: an end-to-end framework for object detection[J]. IEEE Transactions on Pattern Analysis and Machine Intelligence, 2023, 45 (12): 15650- 15664 doi: 10.1109/TPAMI.2023.3292030

[9]	HE K, GKIOXARI G, DOLLAR P, et al Mask R-CNN[J]. IEEE Transactions on Pattern Analysis and Machine Intelligence, 2020, 42 (2): 386- 397 doi: 10.1109/TPAMI.2018.2844175

[10]	REDMON J, DIVVALA S, GIRSHICK R, et al. You only look once: unified, real-time object detection [C]// Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition. Las Vegas: IEEE, 2016: 779–788.

[11]	REDMON J, FARHADI A. YOLOv3: an incremental improvement [EB/OL]. (2018−04−08) [2026−04–16]. https://arxiv.org/abs/1804.02767.

[12]	BOCHKOVSKIY A, WANG C Y, LIAO H Y M. YOLOv4: optimal speed and accuracy of object detection [EB/OL]. (2020−04–23) [2026−04–16]. https://arxiv.org/abs/2004.10934.

[13]	LIU W, ANGUELOV D, ERHAN D, et al. SSD: single shot MultiBox detector [C]// Computer Vision – ECCV 2016. Cham: Springer, 2016: 21–37.

[14]	ZHU X, SU W, LU L, et al. Deformable DETR: deformable transformers for end-to-end object detection [EB/OL]. (2020–10−08) [2026−04–16]. https://arxiv.org/abs/2010.04159.

[15]	DAI X, CHEN Y, YANG J, et al. Dynamic DETR: end-to-end object detection with dynamic attention [C]// Proceedings of the IEEE/CVF International Conference on Computer Vision. Montreal: IEEE, 2021: 2968–2977.

[16]	LI F, ZHANG H, LIU S, et al. DN-DETR: accelerate DETR training by introducing query denoising [EB/OL]. (2022−03−02) [2026−04–16]. https://arxiv.org/abs/2203.01305.

[17]	ZHANG H, LI F, LIU S, et al. DINO: DETR with improved denoising anchor boxes for end-to-end object detection [EB/OL]. (2022−03−07) [2026−04–16]. https://arxiv.org/abs/2203.03605.

[18]	邵磊, 李金钊, 李季, 等. 基于YOLOv8的轻量化高速铁路接触网异物入侵目标检测 [EB/OL]. (2024–12−02) [2026−04–16]. https://link.cnki.net/urlid/12.1374.N.20241130.1122.006.

[19]	高修强, 余星阳, 刘伯鹍, 等. 基于改进Mask R-CNN的轨道交通异物入侵模型研究 [J]. 自动化与仪表, 2025, 40(6): 111–115. GAO Xiuqiang, YU Xingyang, LIU Bokun, et al. Research on foreign object intrusion model for rail transit based on improved mask R-CNN [J]. Automation and Instrumentation. 2025, 40(6): 111–115.

[20]	ZHAO Y, LV W, XU S, et al. DETRs beat YOLOs on real-time object detection [C]// Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition. Seattle: IEEE, 2024: 16965–16974.

[21]	XIAO Y, XU T, XIN Y, et al FBRT-YOLO: faster and better for real-time aerial image detection[J]. Proceedings of the AAAI Conference on Artificial Intelligence, 2025, 39 (8): 8673- 8681 doi: 10.1609/aaai.v39i8.32937

[22]	YANG J, LIU S, WU J, et al Pinwheel-shaped convolution and scale-based dynamic loss for infrared small target detection[J]. Proceedings of the AAAI Conference on Artificial Intelligence, 2025, 39 (9): 9202- 9210 doi: 10.1609/aaai.v39i9.32996

[23]	LIU H, WANG L, ZHANG Y, et al. SaMam: style-aware state space model for arbitrary image style transfer [EB/OL]. (2025−03–20) [2026−04–16]. https://arxiv.org/abs/2503.15934.

[24]	FENG C, ZHONG Y, GAO Y, et al. TOOD: task-aligned one-stage object detection [C]// Proceedings of the IEEE/CVF International Conference on Computer Vision. Montreal: IEEE, 2021: 3490–3499.

[25]	HUANG S, LU Z, CUN X, et al. DEIM: DETR with improved matching for fast convergence [EB/OL]. (2024–12−05) [2026−04–16]. https://arxiv.org/abs/2412.04234.

[26]	杨文, 胡昊, 李凌志, 等基于机器视觉的铁路限界入侵检测方法[J]. 铁道科学与工程学报, 2025, 22 (3): 1328- 1343 YANG Wen, HU Hao, LI Lingzhi, et al Railway boundary foreign object intrusion detection method based on machine vision[J]. Journal of Railway Science and Engineering, 2025, 22 (3): 1328- 1343

[1]	Kaijun WU,Yunqi ZHENG,Ding WEI,Haixiang YUAN. YOLOv8s based lightweight algorithm for small object detection in aerial imagery[J]. Journal of ZheJiang University (Engineering Science), 2026, 60(9): 1912-1923.

[2]	Yijie TANG,Mingen ZHONG,Bingan YUAN,Kang FAN,Jiawei TAN,Zhiqiang LIN. Road segmentation algorithm based on joint orthogonal feature fusion and large kernel separable attention[J]. Journal of ZheJiang University (Engineering Science), 2026, 60(9): 1942-1952.

[3]	Guangqiu CHEN,Tianrong REN,Jin DUAN,Dandan HUANG. Cross-modal semantic segmentation algorithm with edge-assisted and multi-level feature fusion[J]. Journal of ZheJiang University (Engineering Science), 2026, 60(8): 1782-1791.

[4]	Zheng WANG,Mengjun ZHANG,Nan JIANG,Wanliang WANG,Hangyao TU. Daily runoff prediction using LSTM based on multi-feature fusion and Newton-Raphson-based optimizer[J]. Journal of ZheJiang University (Engineering Science), 2026, 60(7): 1567-1576.

[5]	Naizhou ZHANG,Yunchao ZHAO,Wei CAO,Xiaojian ZHANG. Image captioning generation based on multiple-view cross-modal feature fusion[J]. Journal of ZheJiang University (Engineering Science), 2026, 60(6): 1205-1212.

[6]	Bo DONG,Donghao LV,Dahua YU,Xiaowei DU. VAE model combined with multi-domain feature for muscle fatigue analysis[J]. Journal of ZheJiang University (Engineering Science), 2026, 60(6): 1317-1328.

[7]	Guoyan LI,Wei YU,Yupeng MEI,Minghui ZHANG,Xinqiang WANG. Building extraction from remote sensing images with global-local feature fusion[J]. Journal of ZheJiang University (Engineering Science), 2026, 60(5): 1100-1108.

[8]	Tianhe YU,Wenlong WANG,Yong LIU,Zhuangzhuang YANG,Shanchong HOU. Improved algorithm for identifying occluded vehicles and pedestrians in foggy images[J]. Journal of ZheJiang University (Engineering Science), 2026, 60(4): 738-750.

[9]	Long MA,Yongqi HOU,Baijing WU,Li GAO,Jianwei DENG,Guanghui YAN. Water-floating garbage trajectory prediction model based on multi-scale graph convolution[J]. Journal of ZheJiang University (Engineering Science), 2026, 60(4): 751-762.

[10]	Xiaoan BAO,Enlin CHEN,Na ZHANG,Xiaomei TU,Biao WU,Qingqi ZHANG. 3D human pose estimation based on multi-scale encoder fusion[J]. Journal of ZheJiang University (Engineering Science), 2026, 60(3): 565-573.

[11]	Guoyan LI,Penghui LI,Rong LIU,Yupeng MEI,Minghui ZHANG. Remote sensing road extraction by fusing multi-scale resolution and strip feature[J]. Journal of ZheJiang University (Engineering Science), 2026, 60(3): 585-593.

[12]	Jiangang ZHANG,Xiao LI,Dandan FENG. Dynamic kernel perception for pavement distress detection in UAV inspection[J]. Journal of ZheJiang University (Engineering Science), 2026, 60(10): 2141-2152.

[13]	Siyao ZHOU,Nan XIA,Jiahong JIANG. Pose-guided dual-branch network for clothing-changing person re-identification[J]. Journal of ZheJiang University (Engineering Science), 2026, 60(1): 71-80.

[14]	Jian XIAO,Xinze HE,Hongliang CHENG,Xiaoyuan YANG,Xin HU. Aerial small target detection algorithm based on multi-scale feature enhancement[J]. Journal of ZheJiang University (Engineering Science), 2026, 60(1): 19-31.

[15]	Wenxin CHENG,Guanghui YAN,Wenwen CHANG,Baijing WU,Yaning HUANG. Channel-weighted multimodal feature fusion for EEG-based fatigue driving detection[J]. Journal of ZheJiang University (Engineering Science), 2025, 59(9): 1775-1783.

Viewed

Full text

Abstract

Cited

Shared

Discussed