Federated reinforcement learning for resource management in vehicular networks with differentiated interferences

doi:10.3785/j.issn.1008-973X.2026.10.013

Journal of ZheJiang University (Engineering Science)

2026, Vol. 60

Issue (10): 2207-2214 DOI: 10.3785/j.issn.1008-973X.2026.10.013

Federated reinforcement learning for resource management in vehicular networks with differentiated interferences

Xingxing LI(

),Fan YANG,Jie HUANG*(

),Xianzhi LAI,Fenghang YAO,Jieliang CAI,Ni ZHANG

School of Electrical and Electronic Engineering, Chongqing University of Technology, Chongqing 400054, China

Download:

HTML

PDF(2189KB) HTML
Export: BibTeX | EndNote (RIS)

Abstract

In dense Internet of Vehicles (IoV) networks, overlapping vehicular communication ranges easily cause interference. Variations in interference sources and intensities among vehicles result in differentiated interference, which directly leads to packet loss, reduced transmission rates, and increased latency, thereby significantly increasing the difficulty of channel and power management. To address this issue, a federated reinforcement learning-based resource management framework was proposed. A receiver-centric interference weight model was established and an interference metric was defined to quantify differentiated interference among vehicles. An optimization problem was formulated to maximize both network throughput and resource reuse efficiency. The combinatorial resource allocation problem was transformed into a Markov decision process, and a federated reinforcement learning-based resource management algorithm was developed. Simulation results demonstrated that in dense IoV scenarios, the proposed method exhibited significant advantages over traditional algorithms: average network throughput increased by 28.13%, signal-to-interference-plus-noise ratio performance increased by 54.2%, and resource reuse rate rised by 48.7%. These results validated that the federated reinforcement learning-based resource management approach effectively enhanced the overall network performance in dense IoV networks.

Key words： dense internet of vehicles interference weight Markov decision process federated reinforcement learning resource management

Received: 14 April 2025 Published: 28 July 2026

CLC:

TN 929.53

Fund: 国家自然科学基金资助项目（62301094）；重庆市教育委员会科技研究计划资助项目（KJQN202201157，KJQN202301135）；重庆理工大学2025年研究生创新项目资助（gzlcx20253147, gzlcx20253156） .

Corresponding Authors: Jie HUANG E-mail: 2409299367@qq.com;huangjie_cq@cqut.edu.cn

	Service
	E-mail this article
	Add to my bookshelf
	Add to citation manager
	E-mail Alert
	RSS
	Articles by authors
	Xingxing LI
	Fan YANG
	Jie HUANG
	Xianzhi LAI
	Fenghang YAO
	Jieliang CAI
	Ni ZHANG

Cite this article:

Xingxing LI,Fan YANG,Jie HUANG,Xianzhi LAI,Fenghang YAO,Jieliang CAI,Ni ZHANG. Federated reinforcement learning for resource management in vehicular networks with differentiated interferences. Journal of ZheJiang University (Engineering Science), 2026, 60(10): 2207-2214.

URL:

https://www.zjujournals.com/eng/10.3785/j.issn.1008-973X.2026.10.013 OR https://www.zjujournals.com/eng/Y2026/V60/I10/2207

面向车联网差异化干扰的联邦强化学习资源管理

密集车联网（IoV）中车辆通信范围重叠易引发干扰，不同车辆受干扰源及强度差异形成差异化干扰，直接导致通信数据丢失、速率下降、延迟增加，显著加剧信道与功率管理挑战. 为此，提出基于联邦强化学习的资源管理框架. 该框架以接收端为中心建立干扰权值模型，并定义干扰度来量化车辆间的差异化干扰. 提出以最大化网络吞吐量和资源复用率为目标的资源分配优化问题. 将无线资源分配的组合优化问题转化为马尔可夫决策过程模型，并提出基于联邦强化学习的资源管理算法. 仿真实验表明，在密集车联网场景下，所提方法相较于传统算法展现出显著优势：网络吞吐量平均提升28.13%，信干噪比性能提高54.2%，资源复用率增幅达48.7%. 研究成果验证了基于联邦强化学习的资源管理方法能有效提升密集车联网的整体网络性能.

关键词： 密集车联网, 干扰权值, 马尔可夫决策过程, 联邦强化学习, 资源管理

Fig.1 Scenario diagram of Internet of Vehicles

Fig.2 Interference illustration in Internet of Vehicles

Fig.3 FedAvg-AC-based resource management framework

Tab.1 Simulation parameters

Fig.4 Convergence performance comparison of FedAvg-AC and AC

Fig.5 Convergence performance with different learning rates

Fig.6 SINR comparison among different algorithms

Fig.7 Network throughput comparison among different algorithms

Fig.8 Resource reuse rate comparison among different algorithms


[1]	ZHAO J, QUAN H, XIA M, et al Adaptive resource allocation for mobile edge computing in Internet of vehicles: a deep reinforcement learning approach[J]. IEEE Transactions on Vehicular Technology, 2024, 73 (4): 5834- 5848 doi: 10.1109/TVT.2023.3335663

[2]	SUN L, LIU M, GUO J, et al Deep reinforcement learning empowered resource allocation in vehicular fog computing[J]. IEEE Transactions on Vehicular Technology, 2023, 73 (5): 7066- 7076

[3]	LIN D, WU W Resource allocation in a secure Internet of battle vehicles through RF fingerprint recognition[J]. IEEE Transactions on Vehicular Technology, 2023, 72 (5): 6880- 6885 doi: 10.1109/TVT.2023.3235884

[4]	HUANG J, YANG F, CHAKRABORTY C, et al Opportunistic capacity based resource allocation for 6G wireless systems with network slicing[J]. Future Generation Computer Systems, 2023, 140: 390- 401 doi: 10.1016/j.future.2022.10.032

[5]	YANG F, HUANG J, BHARDWAJ A, et al Adaptive modulation based on nondata-aided error vector magnitude for smart systems in smart cities[J]. IEEE Internet of Things Journal, 2023, 10 (21): 18672- 18685 doi: 10.1109/JIOT.2023.3268659

[6]	HUANG J, YU T, ZHU X, et al Energy efficiency maximization in UAV-assisted intelligent autonomous transport system for 6G networks with energy harvesting[J]. IEEE Transactions on Intelligent Transportation Systems, 2025, 26 (10): 17212- 17222 doi: 10.1109/TITS.2024.3445088

[7]	YANG F, ZHAO Z, HUANG J, et al A federated reinforcement learning approach for optimizing wireless communication in UAV-enabled IoT network with dense deployments[J]. IEEE Internet of Things Journal, 2024, 11 (20): 33953- 33966 doi: 10.1109/JIOT.2024.3434713

[8]	PAN T, WU X, ZHANG T, et al Energy-efficient resource allocation in ultra-dense networks with EMBB and URLLC users coexistence[J]. IEEE Transactions on Vehicular Technology, 2023, 73 (2): 2549- 2563

[9]	SHAMAEI S, BAYAT S, HEMMATYAR A M A Interference-aware resource allocation algorithm for D2D-enabled cellular networks using matching theory[J]. IEEE Transactions on Network and Service Management, 2024, 21 (1): 759- 772 doi: 10.1109/TNSM.2023.3283993

[10]	XU Y, ZHENG L, WU X, et al Energy-efficient resource allocation for V2X communications[J]. IEEE Internet of Things Journal, 2024, 11 (18): 30014- 30026 doi: 10.1109/JIOT.2024.3410098

[11]	CHENG N, ZHANG N, LU N, et al Opportunistic spectrum access for CR-VANETs: a game-theoretic approach[J]. IEEE Transactions on Vehicular Technology, 2014, 63 (1): 237- 251 doi: 10.1109/TVT.2013.2274201

[12]	CHIEN W C, LAI C F, CHAO H C Dynamic resource prediction and allocation in C-RAN with edge artificial intelligence[J]. IEEE Transactions on Industrial Informatics, 2019, 15 (7): 4306- 4314 doi: 10.1109/TII.2019.2913169

[13]	YANG C, LOU W, LIU Y, et al Resource allocation for edge computing-based vehicle platoon on freeway: a contract-optimization approach[J]. IEEE Transactions on Vehicular Technology, 2020, 69 (12): 15988- 16000 doi: 10.1109/TVT.2020.3039851

[14]	JIA Y, ZHANG C, HUANG Y, et al Lyapunov optimization based mobile edge computing for Internet of vehicles systems[J]. IEEE Transactions on Communications, 2022, 70 (11): 7418- 7433 doi: 10.1109/TCOMM.2022.3206885

[15]	SUN W, WANG P, XU N, et al Dynamic digital twin and distributed incentives for resource allocation in aerial-assisted Internet of vehicles[J]. IEEE Internet of Things Journal, 2022, 9 (8): 5839- 5852 doi: 10.1109/JIOT.2021.3058213

[16]	WU C, HUANG Z, ZOU Y Delay constrained hybrid task offloading of Internet of vehicle: a deep reinforcement learning method[J]. IEEE Access, 2022, 10: 102778- 102788 doi: 10.1109/ACCESS.2022.3206359

[17]	XU J, AI B, CHEN L, et al Deep reinforcement learning for computation and communication resource allocation in multiaccess MEC assisted railway IoT networks[J]. IEEE Transactions on Intelligent Transportation Systems, 2022, 23 (12): 23797- 23808 doi: 10.1109/TITS.2022.3205175

[18]	HUANG J, WAN J, LV B, et al Joint computation offloading and resource allocation for edge-cloud collaboration in Internet of vehicles via deep reinforcement learning[J]. IEEE Systems Journal, 2023, 17 (2): 2500- 2511 doi: 10.1109/JSYST.2023.3249217

[19]	PARVINI M, JAVAN M R, MOKARI N, et al AoI-aware resource allocation for platoon-based C-V2X networks via multi-agent multi-task reinforcement learning[J]. IEEE Transactions on Vehicular Technology, 2023, 72 (8): 9880- 9896 doi: 10.1109/TVT.2023.3259688

[20]	WU Q, WANG W, FAN P, et al Cooperative edge caching based on elastic federated and multi-agent deep reinforcement learning in next-generation networks[J]. IEEE Transactions on Network and Service Management, 2024, 21 (4): 4179- 4196 doi: 10.1109/TNSM.2024.3403842

[21]	ZHANG P, ZHANG Y, KUMAR N, et al Dynamic SFC embedding algorithm assisted by federated learning in space-air-ground-integrated network resource allocation scenario[J]. IEEE Internet of Things Journal, 2022, 10 (11): 9308- 9318

[22]	LI N, SONG X, LI K, et al Multiagent federated deep-reinforcement-learning-enabled resource allocation for an air–ground-integrated Internet of vehicles network[J]. IEEE Internet Computing, 2023, 27 (5): 15- 23 doi: 10.1109/MIC.2023.3307431

[23]	YANG F, ZHANG S, LIU C, et al A hierarchical network management strategy for distributed CIIoT with imperfect CSI[J]. IEEE Internet of Things Journal, 2023, 11 (8): 13509- 13523

[24]	HUANG J, ZHANG S, YANG F, et al Hypergraph-based interference avoidance resource management in customer-centric communication for intelligent cyber-physical transportation systems[J]. IEEE Transactions on Consumer Electronics, 2024, 70 (1): 1775- 1786 doi: 10.1109/TCE.2023.3324680

[25]	HUANG J, YU T, YANG F, et al AoI-aware resource allocation with interference avoidance for ultradense industrial Internet of Things networks[J]. IEEE Internet of Things Journal, 2024, 11 (17): 28787- 28797 doi: 10.1109/JIOT.2024.3403849

[26]	YANG F, YANG C, HUANG J, et al Mutual-interference-aware throughput enhancement in massive IoT: a graph reinforcement learning framework[J]. IEEE Internet of Things Journal, 2024, 11 (18): 30341- 30353 doi: 10.1109/JIOT.2024.3411653

[27]	HUANG J, YANG C, ZHANG S, et al Reinforcement learning based resource management for 6G-enabled mIoT with hypergraph interference model[J]. IEEE Transactions on Communications, 2024, 72 (7): 4179- 4192 doi: 10.1109/TCOMM.2024.3372892

[1]	JIN Zhuo-jun, QIAN Hui, ZHU Miao-liang. Trajectory evaluation method based on intention analysis[J]. Journal of ZheJiang University (Engineering Science), 2011, 45(10): 1732-1737.

Viewed

Full text

Abstract

Cited

Shared

Discussed