Megadose The AI curve, compressed.

CertiSum: Bitwise-Identical MXFP4 Training Across Platforms and Parallelism Layouts

· ArXiv · AI/CL/LG ·
CertiSum kept Qwen3-8B-Base training states identical bit for bit on AMD and NVIDIA GPUs over 200 steps.

The experiments also preserved identical states when data, tensor and context parallelism changed on AMD MI350X. CertiSum certifies when native FP32 accumulation is exact, recomputes uncertified outputs exactly and merges partial sums without rounding. Across three 8B–14B models, it added 11.8–60.5% to training-step latency versus non-exact MXFP4 at the larger input shapes tested. It still achieved speedups of up to 1.24× over BF16 and 1.70× over deterministic BF16.

ArXiv · AI/CL/LG's note

score 5

Categories: Research