The DWA-LSTM TD3 algorithm was proposed in order to address the challenges of high data demand in deep reinforcement learning (DRL) and insufficient utilization of continuous navigation information. Robot motion was controlled based on the relative position of the target point, the robot’s own velocity, and current LiDAR data, without relying on any prior map. The dynamic window approach (DWA) was employed to guide the twin delayed deep deterministic policy gradient (TD3) algorithm during training, thereby enhancing the quality of collected training data. A long short-term memory (LSTM) neural network was integrated into the policy network in order to improve the agent’s ability to process continuous navigation information. A simulation environment was constructed for training and evaluation, and comparative experiments were conducted with other methods. The experimental results show that the DWA-LSTM TD3 algorithm achieves higher cumulative rewards and improves the success rate of navigation tasks under the same number of training steps. The fluctuation range of navigation orientation angles was reduced, smoother trajectories were produced, and the safety performance of robot motion was enhanced. The algorithm can be used to efficiently accomplish across various scenarios. The algorithm has strong generalization ability.