Performance Modeling

Efficient Fine-Grained GPU Performance Modeling for Distributed Deep Learning of LLM
Training Large Language Models (LLMs) is one of the most compute-intensive tasks in high-performance computing. Predicting end-to-end …
Biyao Zhang, Mingkai Zheng, Debargha Ganguly, Xuecen Zhang, Vikash Singh, Vipin Chaudhary, Zhao Zhang
PDF Cite DOI arXiv