降低分布式训练通信的梯度稀疏压缩方法

陈世达,刘强,韩亮

Gradient sparsification compression approach to reducing communication in distributed training

Shi-da CHEN,Qiang LIU,Liang HAN

表 3 不同策略下模型达到相同精度的计算加速比

Tab.3 Speedup of computation time for models with same accuracy under different strategies

网络模型 α radixSelect 层级选择 剪枝 LDTE-BS ResNet101 1.00 1.54 1.53 1.62 DenseNet169 1.00 0.98 1.08 1.18 ResNet50 1.00 1.48 1.53 1.55 DenseNet121 1.00 0.84 1.01 1.12