Not All Eval-Awareness Is Equal: Capabilities Framing Predicts Compliance
The paper says “eval-awareness” changes model behavior depending on what kind of test the model thinks it is taking.
Zhuang and Aranguri split chain-of-thought eval-awareness into capabilities-framed, safety-framed, both, or neither. On Qwen3-32B with FORTRESS, capabilities-framing predicted compliance by a 24- to 46-point margin over safety-framing across tested steering conditions. A prefill intervention shifted compliance in the predicted direction in 10 of 11 cases, suggesting the framing is not just descriptive. ArXiv · AI/CL/LG's note
Zhuang and Aranguri split chain-of-thought eval-awareness into capabilities-framed, safety-framed, both, or neither. On Qwen3-32B with FORTRESS, capabilities-framing predicted compliance by a 24- to 46-point margin over safety-framing across tested steering conditions. A prefill intervention shifted compliance in the predicted direction in 10 of 11 cases, suggesting the framing is not just descriptive. ArXiv · AI/CL/LG's note
score 5