A cloud removal network for remote sensing imagery that integrated synthetic aperture radar (SAR) and optical data was proposed to address the issues of unstable performance and uneven color tones in existing deep learning-based cloud removal methods. The true texture information from SAR images and the spatial-spectral feature information from optical images were used to construct feature reconstruction tasks both globally and locally, and these tasks guided the network to rebuild missing information in cloud-covered areas. The dual-activation gated convolutional blocks and the channel attention blocks were utilized to build a spatial-spectral feature inference and reconstruction block which significantly enhanced the network’s ability to extract features from useful information in non-cloud areas. The SEN12MS-CR-TS dataset was divided into four subsets based on different cloud morphologies and cloud contents for training and testing. The experimental results showed that the peak signal-to-noise ratio (PSNR) and structural similarity index measure (SSIM) of the proposed method were 1.038 4 dB and 0.091 5, respectively, which were higher than those of the best cloud removal methods. Thus the remote sensing image thick cloud removal network, which integrates SAR and optical data, can effectively remove clouds from images and reconstruct the details beneath the clouds.