MasterControl Seventeen Every Time
A locked-down analytics setup beat runtime SQL/tool planning 110–0 in this test.
The paper describes an enterprise analytics system where the model reads intent, but policy chooses and runs pre-approved analytical programs. In 330 runtime-planning episodes across three 8B models, none met the full answer-and-evidence contract on all test datasets. The policy-executed analyzer did so in all 110 runs. The authors frame it as configuration-specific, not a general proof that runtime agents cannot work. HF Daily Papers' note
The paper describes an enterprise analytics system where the model reads intent, but policy chooses and runs pre-approved analytical programs. In 330 runtime-planning episodes across three 8B models, none met the full answer-and-evidence contract on all test datasets. The policy-executed analyzer did so in all 110 runs. The authors frame it as configuration-specific, not a general proof that runtime agents cannot work. HF Daily Papers' note
score 4