RAFT: A Stateful Retrieval-Augmented Framework for Troubleshooting Agents
cs.AI
Submitted: 2026-09-17
Updated: 2026-09-17
Comments: Accepted to the EMNLP 2026 Industry Track
Code: https://github.com/microsoft/RAFT
License: http://creativecommons.org/licenses/by/4.0/
The gist: Effective troubleshooting agents in enterprise customer support depend on retrieving actionable guidance from similar historical cases, yet existing retrieval-augmented generation (RAG) systems treat
Terminology
Abstract
Effective troubleshooting agents in enterprise customer support depend on retrieving actionable guidance from similar historical cases, yet existing retrieval-augmented generation (RAG) systems treat support cases as static documents and overlook their multi-stage, stateful nature. We introduce RAFT (Retrieval-Augmented Framework for Troubleshooting Agents), a stateful RAG framework that abstracts each closed historical case into a directed chain of timeline entries and retrieves at the entry level, surfacing cases whose intermediate states match the active case and returning the parent-case trajectory anchored at the matched state; an optional case-level graph links cases through a configurable similarity representation. We evaluate this retrieval layer directly, which, unlike evaluating a full agent system, requires no production deployment. Because public multi-stage troubleshooting data is extremely rare, we pair a synthetic benchmark built from Microsoft Learn Windows Server documentation with real Apache Jira issues carrying human-created duplicate labels. RAFT improves Case Hit over vanilla RAG and GraphRAG baselines at every stage of case progress, with statistically significant gains over the strongest baseline; the Jira results provide directional evidence that the advantage transfers to real case histories. We release our benchmark, implementation, and the Apache Jira evaluation set.
Sources
- You Don't Need Pre-built Graphs for RAG: Retrieval Augmented Generation with Adaptive Reasoning Structures
- From Local to Global: A Graph RAG Approach to Query-Focused Summarization
- Retrieval-Augmented Generation for Large Language Models: A Survey
- From RAG to Memory: Non-Parametric Continual Learning for Large Language Models
- RAG vs. GraphRAG: A Systematic Evaluation and Key Insights
- Retrieval-Augmented Generation with Graphs (GraphRAG)
- SWE-bench: Can Language Models Resolve Real-World GitHub Issues?
- An Empirical Study of Catastrophic Forgetting in Large Language Models During Continual Fine-tuning
- BM25S: Orders of magnitude faster lexical search via eager sparse scoring
- Graph-Enhanced Retrieval-Augmented Question Answering for E-Commerce Customer Support
- Agentic Retrieval-Augmented Generation: A Survey on Agentic RAG
- When to use Graphs in RAG: A Comprehensive Analysis for Graph Retrieval-Augmented Generation
- Graph-based Agent Memory: Taxonomy, Techniques, and Applications
- G-Memory: Tracing Hierarchical Memory for Multi-Agent Systems
- A Survey of Graph Retrieval-Augmented Generation for Customized Large Language Models
- In-depth Analysis of Graph-based RAG in a Unified Framework
- LinearRAG: Linear Graph Retrieval Augmented Generation on Large-scale Corpora
Related papers
- MAVEN-T: Reinforced Heterogeneous Distillation for Real-Time Multi-Agent Trajectory Prediction
- Model Discovery Agent: LLM-assisted Bayesian experiment design for data-efficient discovery of mechanistic world models
- The Clinician's Veto: Navigating Trust, Liability, and Uncertainty in Autonomous AI Prescribing
- MindHelper: Closed-Loop Embodied Mental-State Reasoning for Precision Intervention
- Incumbent Advantage: Brand Bias and Cognitive Manipulation Dynamics in LLM Recommendation Systems
- VSAL: A Vision Solver with Adaptive Layouts for Graph Property Detection