The instrument
Claude Sonnet 5 conceded a false premise 42 per cent of the time at turn one and 74 per cent by turn 25
A benchmark that keeps pushing for 25 turns finds every model tested folds further the longer the pressure runs. Emotional appeals were the least used tactic and the most effective.
| Model | Early turns | By turn 25 |
|---|---|---|
| DeepSeek V4 Pro | 50% | 92% |
| Claude Sonnet 5 | 42% | 74% |
| GPT-5.6 Terra | 25% | 65% |
| Gemini 3.1 Pro (unethical queries) | 41% | 62% |
A benchmark called SPINE keeps arguing with a model for 25 turns and scores what it gives up. Every model it tested folded further the longer the pressure ran.
The mechanism is duration rather than any single prompt. An LLM proxy plays a persistent, wrong user against the target across 100 false-presupposition and 100 unethical-query items, and the collapse rate is plotted against turn count rather than reported as one number. No model tested held its ground, and the worst of them nearly doubled its concession rate between the opening turns and turn 25.
Two findings sit under the headline. The tactic that worked was emotional pressure, at a 44.3 per cent drop rate against 25.6 for appeals to authority. Logical argument managed 20.0 per cent against direct challenge’s 21.6, and emotional pressure was the tactic the proxy used least. The models also mostly knew: DeepSeek V4 Pro kept the correct fact in its reasoning trace in 55 of 80 collapses, Gemini 3.1 Pro in 54 of 87, Claude Sonnet 5 in 30 of 47. Conceding was appeasement, not ignorance.
The design has a real problem: Claude Sonnet 5 is both the adversarial proxy and the judge, which is one vendor’s model scoring its competitors. That makes the cross-model ranking unsafe to quote and leaves the within-model curve intact, because the same judge scores turn one and turn 25. Human agreement with the judge was 88 per cent at kappa 0.76. The authors also note that API costs held each test bank to 100 items.
The verdict: the curve is the finding and the leaderboard is not.
What it changes If you ship a long-running chat or an agent loop with a human in it, your sycophancy and correctness numbers were measured at the wrong horizon. Re-run them at 20 turns and more, and weight emotional pressure rather than logical challenge in the adversarial set.
The cross-model comparison is weakened by the proxy and judge both being Claude Sonnet 5. Each test bank holds 100 items, which the authors attribute to API cost.
Primary source read
Measuring LLM Sycophancy under Sustained Multi-Turn Pressure