GPU

Quantifying Performance Variability in GPU Clusters
Modern supercomputers are equipped with massive amounts of Graphics Processing Units (GPUs) to meet the growing demands of scientific …
Michael Mogilevsky, Hazem Zaky, Yu Sun, Mingkai Zheng, Lishan Yang, Zhao Zhang
Cite DOI
Efficient Fine-Grained GPU Performance Modeling for Distributed Deep Learning of LLM
Training Large Language Models (LLMs) is one of the most compute-intensive tasks in high-performance computing. Predicting end-to-end …
Biyao Zhang, Mingkai Zheng, Debargha Ganguly, Xuecen Zhang, Vikash Singh, Vipin Chaudhary, Zhao Zhang
PDF Cite DOI arXiv