Idler fault has become a common problem in the operation of belt conveyors. If the idler fault cannot be diagnosed in time, it will seriously restrict the safe operation of belt conveyors. To solve the above problems, based on the actual operating conditions of idlers in the middle section of a certain mining belt conveyor, an idler fault diagnosis method fusing short-time Fourier transform (STFT) and convolutional neural network (CNN) is proposed. Firstly, based on distributed optical fiber, the vibration signals of the idler operating under normal, bearing damage and cylinder skin fracture conditions were collected and processed by STFT to obtain corresponding time-frequency image sample set, and the sample set was divided into training set and testing set. Then, the training set was input into the CNN model for diagnostic model training, and the operating state characteristics of idlers under different working conditions were constantly updated during the training process. Finally, the trained CNN model was applied to the testing set, and the recognition result of the idler operating state was output. The results showed that the recognition accuracy of the constructed CNN model was as high as 99.6%. Based on the proposed fault diagnosis method, field experiment were carried out in a certain mine to further verify the recognition accuracy of the CNN model. The experimental results showed that the CNN model had a high recognition accuracy of 96.5% for the operating state of idlers in the middle section of the belt conveyor, which was 3.1 percentage points lower than the recognition accuracy on the testing set, indicating that the proposed fault diagnosis method had a certain reliability. Subsequently, the robustness of the fault diagnosis method can be improved by continuously increasing the operation data of idlers under different working conditions, which can provide a powerful theoretical basis for the effective diagnosis of idler faults in coal mine enterprises.