Random defects are characterized by micro-scale, various forms and rich types, and the defect detection accuracy and generalization performance of traditional machine learning models are ordinary. Based on the improved YOLO-v8, a multi-source pipe fitting surface defect detection method was proposed. In order to obtain global information on pipe fittings, a whole-surface image acquisition system based on the adjustable focal length of pipe fittings was constructed, and high-definition imaging of different types of pipe fittings was conducted quickly and efficiently. For the diverse sample sources, the asymptotic feature pyramid network (AFPN) architecture was added to the backbone feature extraction module of the YOLO-v8 model, and the squeeze-and-excitation (SE) attention mechanism was embedded in the bottleneck convolution feature layer to effectively improve the generalization of defect detection model. A training set and a test set were constructed by combining the video extraction of keyframes and static fixed-focus images, and the improved YOLO-v8 algorithm was used to automatically identify the surface defects of pipe fittings. Experimental results showed that the detection mAP50 of the proposed method in the inference stage was 82.2%, which was improved by 3.1 percentage points compared to the traditional YOLO-v8. The results provide a reference for the universal defect detection of metal targets.