Would this change your answer? Evaluating Explanations of LLM Behavior In The Wild with Counterfactual Experiments
CHIVE tests explanations by asking whether they help predict how an LLM would behave under edited prompts.
The paper introduces an agentic pipeline that finds unexpected model behavior and probes it with counterfactual prompt changes. In its evaluation, the interpretability techniques studied did not improve an agent’s ability to predict those counterfactual behaviors. The authors also report that training on CHIVE-generated counterfactual data generalized to out-of-distribution settings. Source: ArXiv · AI/CL/LG's note
The paper introduces an agentic pipeline that finds unexpected model behavior and probes it with counterfactual prompt changes. In its evaluation, the interpretability techniques studied did not improve an agent’s ability to predict those counterfactual behaviors. The authors also report that training on CHIVE-generated counterfactual data generalized to out-of-distribution settings. Source: ArXiv · AI/CL/LG's note
score 5