Robust Multi-Agent LLMs under Byzantine Faults
cs.MA, cs.AI, cs.LG
Submitted: 2026-05-09
Updated: 2026-08-30
Comments: EMNLP 2026 Main Accepted
License: http://creativecommons.org/publicdomain/zero/1.0/
The gist: Large language model (LLM) agents increasingly collaborate over peer-to-peer networks to improve their reliability.
Terminology
Abstract
Large language model (LLM) agents increasingly collaborate over peer-to-peer networks to improve their reliability. However, these same interactions can also introduce vulnerability to unreliable or Byzantine agents that can propagate incorrect information and degrade overall system performance. To address this, we propose Self-Anchored Consensus (SAC), a fully decentralized filter-and-refine protocol in which agents iteratively exchange responses, locally evaluate and filter unreliable messages, and refine their own outputs. We present (F + 1) -robustness conditions on the communication graph that ensure honest agents preserve and propagate reliable information despite Byzantine influence. Experiments across diverse open- and closed-weight LLMs on mathematical and commonsense reasoning benchmarks show that SAC effectively suppresses Byzantine influence and consistently improves performance across diverse communication topologies, whereas prior methods degrade significantly under Byzantine attacks.
Sources
- Rethinking the Reliability of Multi-agent System: A Perspective from Byzantine Fault Tolerance
- Encouraging Divergent Thinking in Large Language Models through Multi-Agent Debate
- Evil Geniuses: Delving into the Safety of LLM-based Agents
- NetSafe: Exploring the Topological Safety of Multi-agent Networks
- Cut the Crap: An Economical Communication Pipeline for LLM-based Multi-Agent Systems
- A Weighted Byzantine Fault Tolerance Consensus Driven Trusted Multiple Large Language Models Network
- Byzantine-Robust Decentralized Coordination of LLM Agents
- Think you have Solved Question Answering? Try ARC, the AI2 Reasoning Challenge
- Minimal Construction of Graphs with Maximum Robustness
Related papers
- Highway Congestion Reduction through Reinforcement Learning Based Eulerian Headway Control
- You Only Align Once: Propagating Cooperative Behaviors in Multi-Agent Systems through Seed Agents
- Deny Without Disabling: Authorization-Paired Evaluation and Control for Multi-Agent Systems
- MA-SAPO: Multi-Agent Reasoning for Score-Aware Prompt Optimization
- PeroMAS: A Multi-agent System of Perovskite Material Discovery
- StitchCUDA: An Automated Multi-Agents End-to-End GPU Programing Framework with Rubric-based Agentic Reinforcement Learning