FLARE: Fine-Grained Diagnostic Feedback for LLM Code Refinement
cs.SE, cs.AI
Submitted: 2026-06-02
Updated: 2026-09-16
License: http://creativecommons.org/licenses/by/4.0/
The gist: Large language models often generate code with bugs.
Terminology
Abstract
Large language models often generate code with bugs. Existing methods rely on feedback signals such as test failures and self-critiques to iteratively refine the generated code. Such signals are either too coarse-grained or too high-level, which is not sufficient to inform the model where to fix the bug. In this work, we present Flare, an iterative framework with a lightweight diagnostic model that predicts line-level suspiciousness signals for bug localization and code refinement. Given the inherent uncertainty of diagnostic predictions, Flare searches over the top-k suspicious regions and selects the best candidate according to execution outcomes. Experiments on LiveCodeBench and BigCodeBench with five base LLMs show that, even without candidate search (k=1), Flare outperforms the strongest baseline with an absolute improvement from 1.72% to 7.42%. Furthermore, searching over 10 candidates yields an average improvement of 8.50% compared with no candidate search. When evaluated in isolation, our lightweight diagnostic model achieves the best performance compared with recent fault localization methods, demonstrating that it can provide reliable fine-grained guidance for code refinement.
Sources
- GPT-4o System Card
- Collu-Bench: A Benchmark for Predicting Language Model Hallucinations in Code
- GPT-4 Technical Report
- The Llama 3 Herd of Models
- Where's the Bug? Attention Probing for Scalable Fault Localization
- DeepSeek-Coder-V2: Breaking the Barrier of Closed-Source Models in Code Intelligence
- Qwen3 Technical Report
Related papers
- Falsification-Based Verification of LLM-Generated Optimization Models: Sound Test Batteries and Their Detection Limits
- GitSkills: A Dataset of Agent Skills on GitHub
- SABER: Benchmarking Operational Safety of LLM Coding Agents in Stateful Project Workspaces
- PackMonitor: Enabling Zero Package Hallucinations Through Decoding-Time Monitoring
- IntentCoding: Amplifying User Intent in Code Generation
- Incentives and Outcomes in Bug Bounties