A lightweight defect detection algorithm was proposed to address the issues of insufficient detection accuracy for rebar surface defects and limited computational resources of terminal devices. Based on the YOLOv8n model, the C2f module was redesigned and a backbone network with global information modeling capabilities was constructed by combining the conditional positional encoding characteristics of the convolutional gated linear unit and the dynamic global interaction capability of the self-attention mechanism. A context anchor attention module was used to improve the high-level screening feature pyramid network. A combined strategy of width-wise and height-wise strip convolutions was adopted to effectively focus on the long-distance pixel information, and the information redundancy in multi-scale feature fusion was reduced through feature selection. An adaptive detection head was proposed via shared convolution and separated BN layers to improve the parameter utilization while ensuring the detection accuracy. The Unified-IoU bounding box loss function was employed to enhance the performance of dense defect detection through dynamic weight distribution. Experimental results showed that the improved algorithm achieved a mAP@0.5 of 95.4% on a self-constructed rebar dataset, which was 4.3 percentage points higher than that of YOLOv8n. The model’s parameter count was reduced to 1.4 M, representing a decrease of 53.33%, the computational complexity was reduced by 34.57%, and the FPS reached 156 frames per second, achieving a balance between the detection performance and the computational resource consumption. Additionally, the algorithm’s good generalization performance was validated on the NEU-DET dataset.