JevOut: Natural Context Can Flip Decision Models
Short, ordinary-looking context additions flipped Jev from correct decisions to fixed wrong options in 61.4% of tested cases.
The paper tests decision models that turn language into probability distributions over finite choices, then route or trigger downstream actions from those outputs. Its optimizer preserves the source, question, choices, and gold answer while adding fluent context aimed at a wrong target option. Jev was redirected on 312 of 508 initially correct decisions, with 229 reaching at least 0.7 probability on the wrong option. Three other decision systems showed targeted flip rates of 64.9% to 73.2% across seven datasets. Source: ArXiv · AI/CL/LG's note
The paper tests decision models that turn language into probability distributions over finite choices, then route or trigger downstream actions from those outputs. Its optimizer preserves the source, question, choices, and gold answer while adding fluent context aimed at a wrong target option. Jev was redirected on 312 of 508 initially correct decisions, with 229 reaching at least 0.7 probability on the wrong option. Three other decision systems showed targeted flip rates of 64.9% to 73.2% across seven datasets. Source: ArXiv · AI/CL/LG's note
score 4