A small target detection algorithm (DLSRF-Net) for multi-scale complex scenarios from UAV viewpoint was proposed by reconstructing the YOLOv11, to address the insufficient feature extraction and poor detection performance of existing algorithms in small target detection under UAV viewpoint due to small target sizes, complex backgrounds, and multi-scale information in the scenarios. The adaptive depthwise separative receptive field attention convolution module (DWRFAConv) was proposed to improve the model’s ability to extract the receptive field features of small targets and reduce the model load. The multi-branch lightweight multi-scale linear attention mechanism was designed to enhance the model’s attention to small targets. The RSCDI module was designed as the upsampling layer and fully connected layer of the model to solve the problem of feature information loss, suppress the useless information, and improve the model’s detection accuracy. The model sizes were classified into two categories based on parameter count and computational complexity, and experimental validation was carried out on the VisDrone2021 dataset. The results showed that the proposed algorithm achieved the optimal performance under both model size categories, and the generalization ability of the proposed algorithm was verified on the DOTA and the SSDD datasets.