A Theory of Cross-Validation Variance
K-fold cross-validation is variance-optimal at first order because it gives every observation equal weight.
The authors prove an exact finite-sample variance identity covering metrics such as mean squared error and AUC. It separates sampling variance from variance introduced by random splitting; Monte Carlo and bootstrap cross-validation lack K-fold’s first-order optimality. Under stated model conditions, leave-one-out has the smallest variance among K-fold schemes, with an analogous result for leave-pair-out AUC. The paper also establishes asymptotic normality using the full variance, providing a basis for inference on cross-validated performance.
ArXiv · AI/CL/LG's note
The authors prove an exact finite-sample variance identity covering metrics such as mean squared error and AUC. It separates sampling variance from variance introduced by random splitting; Monte Carlo and bootstrap cross-validation lack K-fold’s first-order optimality. Under stated model conditions, leave-one-out has the smallest variance among K-fold schemes, with an analogous result for leave-pair-out AUC. The paper also establishes asymptotic normality using the full variance, providing a basis for inference on cross-validated performance.
ArXiv · AI/CL/LG's note
score 4