A multi-human-robot collaboration task allocation framework considering both caregiver’s fatigue and elderly satisfaction was proposed in order to balance the subjective feelings of caregivers and elderly people. A mathematical model of caregiver’s fatigue was established by considering factors such as caregiver’s rest duration before task execution, the rapport between caregivers and elderly people, and task difficulty. A multi-objective optimization model for multi-human-robot collaboration task allocation was developed combined with elderly satisfaction. A two-dimensional double-constraint encoding method and its reasonable initialization and updating methods were proposed based on the characteristics of common tasks in elderly care scenarios. A multi-objective evolutionary algorithm was employed to solve the multi-objective optimization model by using this encoding. The final task execution plan was determined from the Pareto optimal solution set according to the min-max and max-min principles in order to prevent situations where individual caregivers experience extreme fatigue or individual elderly people have extremely low satisfaction. The simulation results demonstrate that the multi-task allocation framework for ‘multiple caregivers and multiple robots’ collaboration can achieve task allocation within a multi-caregiver and multi-robot team in the proposed elderly care scenario while balancing caregiver’s fatigue and elderly satisfaction, as well as maintaining a balance between the overall and individual caregivers, and between the overall and individual elderly people.