Who Flips? Self- and Cross-Model Counterarguments Reveal Answer Instability in LLMs

arXiv:2606.16011 · cs.CL · Submitted 2026-06-14 · Read on arXiv

cs.CL

Submitted: 2026-06-14

Updated: 2026-08-30

Code: https://github.com/nafisenik/WhoFlips

Terminology

• Sycophancy — 29× in this paper · explained in 4 episodes
• Accuracy — 16× in this paper · explained in 1 episode
• Benchmark — 11× in this paper · explained in 2 episodes
• Robustness — 7× in this paper · explained in 5 episodes
• Verification — 5× in this paper · explained in 2 episodes
• Adam — 4× in this paper · explained in 2 episodes
• Benchmarking — 4× in this paper · explained in 3 episodes
• Consistency — 3× in this paper · explained in 3 episodes
• Alignment — 2× in this paper · explained in 1 episode
• Data Augmentation — 2× in this paper · explained in 7 episodes
• Epistemic Authority — 2× in this paper
• Epistemic Porosity — 2× in this paper
• Multi-Agent System — 2× in this paper · explained in 5 episodes
• Regularization — 2× in this paper · explained in 3 episodes
• Reward Hacking — 2× in this paper · explained in 8 episodes
• Calibration — 1× in this paper · explained in 7 episodes
• Clustering — 1× in this paper · explained in 2 episodes
• Failure Modes — 1× in this paper · explained in 3 episodes
• Fine-Tuning — 1× in this paper · explained in 5 episodes
• Large Language Models (LLMs) — 1× in this paper · explained in 6 episodes
• Metacognition — 1× in this paper · explained in 5 episodes
• Refusal Selectivity Score — 1× in this paper
• Scaffolding — 1× in this paper · explained in 3 episodes
• Self-consistency — 1× in this paper · explained in 2 episodes
• Stage I Coercion Refusal Rates — 1× in this paper
• Uncertainty Quantification — 1× in this paper · explained in 8 episodes

Sources

• When Persuasion Overrides Truth in Multi-Agent LLM Debates: Introducing a Confidence-Weighted Persuasion Override Rate (CW-POR)
• BASIL: Bayesian Assessment of Sycophancy in LLMs
• Vulnerability of LLMs' Stated Beliefs? LLMs Belief Resistance Check Through Strategic Persuasive Conversation Interventions
• Consistency Training Helps Stop Sycophancy and Jailbreaks
• Sycophancy to Subterfuge: Investigating Reward-Tampering in Large Language Models
• Ask don't tell: Reducing sycophancy in large language models
• The Llama 3 Herd of Models
• Are You Sure? Challenging LLMs Leads to Performance Drops in The FlipFlop Experiment
• Consistency of Large Reasoning Models Under Multi-Turn Attacks
• Qwen3.5-Omni Technical Report
• Existing LLMs Are Not Self-Consistent For Simple Tasks
• Large Language Models have Intrinsic Self-Correction Ability
• Certainty robustness: Evaluating LLM stability under self-challenging prompts
• TRUTH DECAY: Quantifying Multi-Turn Sycophancy in Language Models
• Calibration Collapse Under Sycophancy Fine-Tuning: How Reward Hacking Breaks Uncertainty Quantification in LLMs
• Artificial Intelligence Index Report 2025
• How RLHF Amplifies Sycophancy
• Pressure, What Pressure? Sycophancy Disentanglement in Language Models via Reward Decomposition
• OpenAI GPT-5 System Card
• Sycophancy Is Not One Thing: Causal Separation of Sycophantic Behaviors in LLMs

Related papers

• Exploring Solution Divergence and Its Effect on Large Language Model Problem Solving
• Ishigaki-IDS-Bench: A Benchmark for Generating Information Delivery Specification from BIM Information Requirements
• Subliminal Steering: Stronger Encoding of Hidden Signals
• MedStruct-S: A Benchmark for Key Discovery, Key-Conditioned QA and Semi-Structured Extraction from OCR Clinical Reports
• The End of Transformers? On Challenging Attention and the Rise of Sub-Quadratic Architectures
• Untangling the Mechanisms of Misleading Context in Medical Question Answering