A feature filtering and feature decoupling based domain generalization model (FF-FDDG) was proposed, aiming at the problem of poor generalization performance of the deep defect detection model caused by inconsistent image brightness across scenes. A designed luminance filtering-residual module (LFR) was included in the proposed model. The brightness variation features were filtered out through instance normalization, the features with high defect correlation and low brightness correlation were extracted from the filtered features, and these features were fused to enhance the generalization ability of the model under the condition of cross-scenario image brightness transformation. Furthermore, a contrast whitening loss (CWL) function was proposed, by which the model was guided to learn the defect texture feature by decoupling the brightness transform feature and the defect texture feature, so as to improve the model generalization ability. The experimental results on the cross-scenario surface defect data collected in the photovoltaic cell manufacturing environment showed that, compared with the state-of-the-art domain generalization model, the average mean average precision (mAP) of the proposed FF-FDDG model in cross-scenario situations was improved by 5.01%.