A new segmentation and localization method of coal and gangue identification was proposed, aiming at the problems of low detection accuracy of coal and gangue visual perception model and low accuracy of centroid localization in the complex similar background. The YOLOv5s model was optimized by the efficient multi-scale and squeeze-and-excitation attention module (ESAM), the content-aware reassembly of features module (CARAFE) and the co-designing and scaling ConvNets with masked autoencoders network (ConvNeXtV2) to construct a coal and gangue target detection model (ECC-YOLO), and to improve the comprehensive detection performance of coal and gangue. A coal and gangue edge detection algorithm integrating adaptive median filtering and gradient enhancement was established to achieve accurate segmentation and centroid localization of gangue images. Based on the coal and gangue homemade dataset to carry out experiments, the experimental results show that: the combined detection performance of ECC-YOLO was optimal when compared to YOLOv5s, YOLOv7-tiny, YOLOv8s, SSD, and DETR, with a mean average precision of 89.0% and an average detection speed of 89.29 FPS. Comparing Canny, Prewitt, KSW and OTSU algorithms, the proposed algorithm has the best degree of completeness for edge extraction, and its pixel area and centroid coordinates have a maximum error rate of 1.491% and 1.796%, which has high positioning accuracy for gangue.