加速大模型训练的双向交错流水线并行方法
吴厚明,陈岭
Bidirectional interleaved pipeline parallelism for accelerating large model training
Houming WU,Ling CHEN
表 6
模块消融实验结果
Tab.6
Results of module ablation study
$ D $
$ \hat{B} $
T
/（sample·s
−1
）
w/o V
w/o E
w/o M
BitPipe
4
16
19.48
18.82
19.58
19.58
4
64
24.19
24.11
24.28
24.28
8
32
38.78
38.32
39.17
39.17
8
128
46.29
46.05
46.43
46.43
16
64
49.21
49.09
48.56
50.67
16
256
55.96
57.05
55.23
57.44