加速大模型训练的双向交错流水线并行方法
Bidirectional interleaved pipeline parallelism for accelerating large model training
加速大模型训练的双向交错流水线并行方法 |
| 吴厚明,陈岭 |
|
Bidirectional interleaved pipeline parallelism for accelerating large model training |
| Houming WU,Ling CHEN |
| 图 6 微批次数量扩展（流水线深度为4，微批次数为8） |
| Fig.6 Scaling to larger micro-batch number (pipeline depth is four, number of micro-batches is eight) |
|
|