SciExam for ENSO: Can AI Agents Build Climate Models?
Twelve agent systems were tested on building ENSO climate models; six beat the published baseline.
SciExam gives agents six hours to process real observations, lock in their diagnostics, and build low-order stochastic ENSO models from that feedback. Hidden graders then test statistics, unobserved-variable recovery, and held-out-year forecasts. The stronger agents mainly won on reconstruction and forecasting, and their simplified model forms lined up with competing explanations of ENSO warm-cold asymmetry. Controlled runs suggest the top scores were shaped by the information provided, not by memorizing the observational record. ArXiv · AI/CL/LG's note
SciExam gives agents six hours to process real observations, lock in their diagnostics, and build low-order stochastic ENSO models from that feedback. Hidden graders then test statistics, unobserved-variable recovery, and held-out-year forecasts. The stronger agents mainly won on reconstruction and forecasting, and their simplified model forms lined up with competing explanations of ENSO warm-cold asymmetry. Controlled runs suggest the top scores were shaped by the information provided, not by memorizing the observational record. ArXiv · AI/CL/LG's note
score 5