A chest disease classification network based on convolution-assisted window self-attention was proposed, called CAWSNet, aiming at the issues of varying lesion sizes, complex textures, and mutual interference in chest X-ray images. The Swin Transformer was utilized as the backbone, employing window self-attention to model long-range visual dependencies. Convolution was introduced to enhance local feature extraction capability while compensating for the deficiencies of window self-attention. Image relative position encoding was used to dynamically calculate directed relative positions, helping the network better model pixel-wise spatial relationships. Class-specific residual attention was employed, and the classifier’s focus area was adjusted based on disease categories in order to highlight effective information and enhance multi-label classification capability. Dynamic difficulty loss function was proposed to alleviate the problem of large differences in disease classification difficulty and the imbalance of positive and negative samples in the dataset. The experimental results on the public datasets ChestX-Ray14, CheXpert and MIMIC-CXR-JPG demonstrate that proposed CAWSNet achieves AUC scores of 0.853, 0.898 and 0.819, respectively, confirming the effectiveness and robustness of the network in diagnosing chest diseases through X-ray images.