An expandable automatic annotation method for traffic images based on cross-modal contrastive learning was proposed aiming at the problems of non-scalable annotation categories and low accuracy in existing automatic annotation methods for traffic images. Dual-modal data comprising text and images were adopted as the research subjects, and the similarity relationships between modalities were captured through contrastive learning. An inter-modal feature enhancement strategy was employed to optimize the effective alignment of cross-modal data. A text-distance fusion encoding module was proposed in the text feature extraction stage, which enhanced the local feature representation capability of text sequences by constructing a distance-aware feature fusion component. A deformable filtering convolution structure was designed for image feature extraction, which effectively enhanced the recognition of irregular objects while filtering out noise information. The original loss structure was improved by establishing a combined contrastive loss function to enhance the discriminative ability between positive and negative cross-modal samples. The experimental results demonstrate that the proposed model achieves an improvement of 5.3% and 4.8% in mAP0.5 and mAP0.5:0.95 respectively on the BIT vehicle dataset compared with other models of similar scale, exhibiting superior performance in the automatic annotation of traffic images.