Numerical Stability | Mingkai Zheng's Homepagehttps://zhengmk.github.io/tag/numerical-stability/Numerical StabilityWowchemy (https://wowchemy.com)en-usMon, 28 Sep 2026 00:00:00 +0000https://zhengmk.github.io/media/icon_hu0158c3d4eaa2a6b730ca31a67c18838d_142122_512x512_fill_lanczos_center_2.pngNumerical Stabilityhttps://zhengmk.github.io/tag/numerical-stability/Broken Symmetry in BF16 Attention: Why FlashAttention Gradients Blow Up Late in Traininghttps://zhengmk.github.io/publication/broken-symmetry-in-bf16-attention-why-flashattention-gradients-blow-up-late-in-training/Mon, 28 Sep 2026 00:00:00 +0000https://zhengmk.github.io/publication/broken-symmetry-in-bf16-attention-why-flashattention-gradients-blow-up-late-in-training/