A novel FR-Unet image segmentation algorithm based on multi-layer feature adaptive fusion was proposed in order to solve the problem of low accuracy when segmenting medical images by traditional algorithms. The sampling weighting module was designed to replace the traditional convolutional layer in the encoder stage. The image spatial information was extracted and fused by layer, and the correlation between neighboring pixels and the semantic information at different levels was obtained. A multi-layer adaptive fusion module was designed in the decoder stage in order to extract image channel information layer by layer through nonlinear jump connections and adaptively fuse the contextual information of neighboring connected layers. Then each layer focuses on the extraction of different feature information. FR-Unet was substantially reduced in the number of model parameters, allowing the network to be better supported in scene deployment. The experimental results show that the network excels in numerous tasks such as animal cell segmentation, liver organ segmentation and skin lesion segmentation.