加速大模型训练的双向交错流水线并行方法
Bidirectional interleaved pipeline parallelism for accelerating large model training
加速大模型训练的双向交错流水线并行方法 |
| 吴厚明,陈岭 |
|
Bidirectional interleaved pipeline parallelism for accelerating large model training |
| Houming WU,Ling CHEN |
| 图 3 几种同步流水线并行方法在模型划分与微批次调度上的对比（流水线深度和微批次数均为4） |
| Fig.3 Comparison of different synchronous pipeline parallelism methods in model partitioning and micro-batch scheduling (both pipeline depth and number of micro-batches are four) |
|
|