OPEN-1B: A Fully Auditable Training Run
The paper claims a 1B-parameter model whose training steps can be independently replayed and checked bit-for-bit on commodity hardware.
The authors argue that released weights, data, and recipes still do not prove a checkpoint came from the stated run because floating-point nondeterminism breaks cross-hardware reproducibility. Their method fixes ordering around GPU reductions, batch ordering, and collective communication so individual distributed-training steps can be audited on a single machine. Since replaying the full run on one machine is impractical, they propose many auditors each verifying portions of the trajectory. They say Open-1B ships with the pretraining dataset, every intermediate checkpoint, training code, and an audit harness. Source: ArXiv · AI/CL/LG's note.
The authors argue that released weights, data, and recipes still do not prove a checkpoint came from the stated run because floating-point nondeterminism breaks cross-hardware reproducibility. Their method fixes ordering around GPU reductions, batch ordering, and collective communication so individual distributed-training steps can be audited on a single machine. Since replaying the full run on one machine is impractical, they propose many auditors each verifying portions of the trajectory. They say Open-1B ships with the pretraining dataset, every intermediate checkpoint, training code, and an audit harness. Source: ArXiv · AI/CL/LG's note.
score 6