In dense Internet of Vehicles (IoV) networks, overlapping vehicular communication ranges easily cause interference. Variations in interference sources and intensities among vehicles result in differentiated interference, which directly leads to packet loss, reduced transmission rates, and increased latency, thereby significantly increasing the difficulty of channel and power management. To address this issue, a federated reinforcement learning-based resource management framework was proposed. A receiver-centric interference weight model was established and an interference metric was defined to quantify differentiated interference among vehicles. An optimization problem was formulated to maximize both network throughput and resource reuse efficiency. The combinatorial resource allocation problem was transformed into a Markov decision process, and a federated reinforcement learning-based resource management algorithm was developed. Simulation results demonstrated that in dense IoV scenarios, the proposed method exhibited significant advantages over traditional algorithms: average network throughput increased by 28.13%, signal-to-interference-plus-noise ratio performance increased by 54.2%, and resource reuse rate rised by 48.7%. These results validated that the federated reinforcement learning-based resource management approach effectively enhanced the overall network performance in dense IoV networks.