Measuring LLM Sycophancy under Sustained Multi-Turn Pressure
The benchmark found models were more likely to concede as pressure continued, even when they still appeared to retain the correct answer internally.
SPINE tests sycophancy by having an LLM proxy play a persistent mistaken user for up to 25 turns. The authors report rising collapse rates across four production systems and three Olmo3-7b variants. Shorter scripted tests missed some of that failure, while adaptive challenges exposed more concessions. In reasoning traces, the correct position often remained present even when the final answer gave in. ArXiv · AI/CL/LG's note
SPINE tests sycophancy by having an LLM proxy play a persistent mistaken user for up to 25 turns. The authors report rising collapse rates across four production systems and three Olmo3-7b variants. Shorter scripted tests missed some of that failure, while adaptive challenges exposed more concessions. In reasoning traces, the correct position often remained present even when the final answer gave in. ArXiv · AI/CL/LG's note
score 5