Available Guardrails: Certifying Selective Prediction across ML Systems
The paper treats “certified availability” as the deployability limit for selective predictors.
Its core claim is that safety gates can be statistically valid yet impossible to certify for some reporting units when calibration data is thin. The authors use exact-binomial inversion and dynamic programming to map the trade-off between safety, granularity, and covered traffic. In their experiments, finite-sample estimation nearly wipes out a large coverage gain, while split-based partition planning and error-budget reallocation recover part of it. The same pattern appears across tool-calling, moderation, lesion classification, and recommendation settings. ArXiv · AI/CL/LG's note
Its core claim is that safety gates can be statistically valid yet impossible to certify for some reporting units when calibration data is thin. The authors use exact-binomial inversion and dynamic programming to map the trade-off between safety, granularity, and covered traffic. In their experiments, finite-sample estimation nearly wipes out a large coverage gain, while split-based partition planning and error-budget reallocation recover part of it. The same pattern appears across tool-calling, moderation, lesion classification, and recommendation settings. ArXiv · AI/CL/LG's note
score 5