The surface defects of steel are diverse in form, complex in structure, and exhibit a high proportion of small targets, while general object detection algorithms have excessive computational complexity and are not suitable for the deployment on edge devices. To address these issues, a lightweight steel defect detection algorithm based on YOLOv8s, called SDB-YOLOv8s, was proposed. Firstly, a redesigned feature interaction module (S-C2f) was introduced to suppress spatial and channel redundant information, enhancing detection accuracy. Secondly, a dilated Transformer module was incorporated to enhance the network’s ability to capture global contextual information and sparse sampling characteristics, reducing fine-grained information loss and improving feature extraction capabilities for small targets. Finally, a lightweight network, BS-ShuffleNetV2, was designed as the backbone network to reduce model complexity while maintaining detection accuracy. Experimental validation on the NEU-DET and Severstal steel defect datasets showed that compared to baseline models, the SDB-YOLOv8 algorithm achieved an improvement in mAP of 6.4 and 7.0 percentage points, and detection frames per second of 146 and 121, with accuracy improvements of 4.6 and 6.5 percentage points respectively. The number of parameters and computational complexity were only 64.8% and 56.2% of the baseline model. The experimental results demonstrated that this algorithm achieved a better balance in terms of detection accuracy, speed, and lightweight characteristics, while providing a reference for high accuracy and real-time capabilities for edge terminal devices.