I agree that this site is using cookies. You can find further informations here.
X
Rajbhandari, Samyam
51  results:
Search for persons X
?
1

System Optimizations for Enabling Training of Extreme Long ..:

, In: Proceedings of the 43rd ACM Symposium on Principles of Distributed Computing,
Jacobs, Sam Ade ; Tanaka, Masahiro ; Zhang, Chengming... - p. 121-130 , 2024
Link: https://dl.acm.org/doi/1..
 
?
2

System Optimizations for Enabling Training of Extreme Long ..:

, In: 2024 IEEE International Parallel and Distributed Processing Symposium Workshops (IPDPSW),
Jacobs, Sam Ade ; Tanaka, Masahiro ; Zhang, Chengming... - p. 1206-1208 , 2024
Link: https://doi.org/10.1109/..
 
?
3

DeepSpeed-FastGen: High-throughput Text Generation for LLMs..:

Holmes, Connor ; Tanaka, Masahiro ; Wyatt, Michael...
http://arxiv.org/abs/2401.08671.  , 2024
Link: http://arxiv.org/abs/240..
 
?
4

A Hybrid Tensor-Expert-Data Parallelism Approach to Optimiz..:

, In: Proceedings of the 37th ACM International Conference on Supercomputing,
Singh, Siddharth ; Ruwase, Olatunji ; Awan, Ammar Ahmad... - p. 203-214 , 2023
Link: https://dl.acm.org/doi/1..
 
?
5

BLOOM: A 176B-Parameter Open-Access Multilingual Language M..:

Scao, Teven Le ; Fan, Angela ; Akiki, Christopher...
info:eu-repo/semantics/altIdentifier/arxiv/2211.05100.  , 2023
Link: https://inria.hal.scienc..
 
?
6

BLOOM: A 176B-Parameter Open-Access Multilingual Language M..:

Scao, Teven Le ; Fan, Angela ; Akiki, Christopher...
info:eu-repo/semantics/altIdentifier/arxiv/2211.05100.  , 2023
Link: https://inria.hal.scienc..
 
?
7

ZeRO++: Extremely Efficient Collective Communication for Gi..:

Wang, Guanhua ; Qin, Heyang ; Jacobs, Sam Ade...
http://arxiv.org/abs/2306.10209.  , 2023
Link: http://arxiv.org/abs/230..
 
?
8

BLOOM: A 176B-Parameter Open-Access Multilingual Language M..:

Scao, Teven Le ; Fan, Angela ; Akiki, Christopher...
info:eu-repo/semantics/altIdentifier/arxiv/2211.05100.  , 2023
Link: https://inria.hal.scienc..
 
?
9

A Hybrid Tensor-Expert-Data Parallelism Approach to Optimiz..:

Singh, Siddharth ; Ruwase, Olatunji ; Awan, Ammar Ahmad...
http://arxiv.org/abs/2303.06318.  , 2023
Link: http://arxiv.org/abs/230..
 
?
10

BLOOM: A 176B-Parameter Open-Access Multilingual Language M..:

Scao, Teven Le ; Fan, Angela ; Akiki, Christopher...
info:eu-repo/semantics/altIdentifier/arxiv/2211.05100.  , 2023
Link: https://inria.hal.scienc..
 
?
11

DeepSpeed-Chat: Easy, Fast and Affordable RLHF Training of ..:

Yao, Zhewei ; Aminabadi, Reza Yazdani ; Ruwase, Olatunji...
http://arxiv.org/abs/2308.01320.  , 2023
Link: http://arxiv.org/abs/230..
 
?
12

DeepSpeed Ulysses: System Optimizations for Enabling Traini..:

Jacobs, Sam Ade ; Tanaka, Masahiro ; Zhang, Chengming...
http://arxiv.org/abs/2309.14509.  , 2023
Link: http://arxiv.org/abs/230..
 
?
13

DeepSpeed-VisualChat: Multi-Round Multi-Image Interleave Ch..:

Yao, Zhewei ; Wu, Xiaoxia ; Li, Conglong...
http://arxiv.org/abs/2309.14327.  , 2023
Link: http://arxiv.org/abs/230..
 
?
14

DeepSpeed- Inference: Enabling Efficient Inference of Trans..:

, In: SC22: International Conference for High Performance Computing, Networking, Storage and Analysis,
Aminabadi, Reza Yazdani ; Rajbhandari, Samyam ; Awan, Ammar Ahmad... - p. 1-15 , 2022
Link: https://doi.org/10.1109/..
 
?
15

1-bit LAMB: Communication Efficient Large-Scale Large-Batch..:

, In: 2022 IEEE 29th International Conference on High Performance Computing, Data, and Analytics (HiPC),
Li, Conglong ; Awan, Ammar Ahmad ; Tang, Hanlin.. - p. 272-281 , 2022
Link: https://doi.org/10.1109/..
 
1-15