Fabric defects with various shapes and sizes, especially those with extremely large aspect ratio, pose a great challenge to detection. In order to improve detection accuracy with limited computational resources, a lightweight fabric defect detection method with high-precision multiscale integration was proposed. Firstly, coordinate parallel depthwise convolution (CPDConv) was designed to provide the absolute position of each pixel point in the image by adding coordinates to the convolution, as well as extract different-scale feature maps, and splice significant defect features with multi-scale and multi-fields of view through parallel multiscale convolution kernels. New bottleneck block CPDBottleneck was reconstructed based on CPDConv, so that high-quality feature extraction module C3CPD had the ability to obtain more feature information. Secondly, a multiscale stepwise feature pyramid network (MSFPN) was constructed, which reduced the loss of feature information between non-adjacent layers during transmission. Finally, a powerful IoU (Inner-PIoU) based on auxiliary bounding boxes was introduced to avoid the expansion of prediction boxes and accelerate their regression speed. The experimental results showed that, on the self-constructed fabric dataset, compared with the original YOLOv5s, the mAP50 of the YOLOv5s improved by the above modules was increased by 3.2 percentage points, its parameters and computational complexity were reduced by about 24.3 percentage points and 16.9 percentage points, respectively, and its frames per second decreased by only about 1.9. The proposed method can meet the requirements of accuracy and speed in industrial fabric defect detection.