Opera: A Verbal Critic Framework for Long-horizon Coding Agents

arXiv:2609.33987 · cs.CL · Submitted 2026-09-27 · Read on arXiv

cs.CL

Submitted: 2026-09-27

Updated: 2026-10-08

Code: https://github.com/datacurve-ai/pier

Terminology

• Benchmark — 19× in this paper · explained in 2 episodes
• Verifier — 17× in this paper · explained in 2 episodes
• Fine-Tuning — 9× in this paper · explained in 5 episodes
• Verification — 8× in this paper · explained in 2 episodes
• Distillation — 7× in this paper · explained in 2 episodes
• Ablation — 4× in this paper · explained in 1 episode
• Core — 3× in this paper · explained in 2 episodes
• Reinforcement Learning — 3× in this paper · explained in 4 episodes
• Test-time Scaling — 3× in this paper · explained in 2 episodes
• Trace — 3× in this paper · explained in 2 episodes
• Adaptation — 2× in this paper · explained in 1 episode
• Iterative Refinement — 2× in this paper · explained in 3 episodes
• On-Policy Distillation — 2× in this paper · explained in 3 episodes
• AI Agents — 1× in this paper · explained in 2 episodes
• Agent Harness — 1× in this paper · explained in 2 episodes
• Agentic Coding — 1× in this paper · explained in 1 episode
• Generalization — 1× in this paper · explained in 13 episodes
• International Conference on Learning Representations — 1× in this paper
• LLM Orchestrator — 1× in this paper · explained in 2 episodes
• LLM judge — 1× in this paper · explained in 2 episodes
• LLM-as-a-Judge — 1× in this paper · explained in 3 episodes
• Online Reinforcement Learning — 1× in this paper · explained in 2 episodes
• Regret — 1× in this paper · explained in 3 episodes
• Robustness — 1× in this paper · explained in 5 episodes
• Self-Distillation — 1× in this paper · explained in 7 episodes

Sources

• SWE-Search: Enhancing Software Agents with Monte Carlo Tree Search and Iterative Refinement
• RepairAgent: An Autonomous, LLM-Based Agent for Program Repair
• Retaining by Doing: The Role of On-Policy Data in Mitigating Forgetting
• SWE-Bench Pro: Can AI Agents Solve Long-Horizon Software Engineering Tasks?
• SWE-Shepherd: Advancing PRMs for Reinforcing Code Agents
• When Agents go Astray: Course-Correcting SWE Agents with PRMs
• CRITIC: Large Language Models Can Self-Correct with Tool-Interactive Critiquing
• Code Generation by Differential Test Time Scaling
• DeepSWE: Measuring Frontier Coding Agents on Original, Long-Horizon Engineering Tasks
• Scaling Test-Time Compute for Agentic Coding
• LLM-as-a-Verifier: A General-Purpose Verification Framework
• S*: Test Time Scaling for Code Generation
• Self-Refine: Iterative Refinement with Self-Feedback
• Can LLM-as-a-Judge Reliably Verify Rubrics in Agentic Scenarios?
• ATLAS: Agentic Test-time Learning-to-Allocate Scaling
• Agentic Rubrics as Contextual Verifiers for SWE Agents
• Reflexion: Language Agents with Verbal Reinforcement Learning
• How Coding Agents Fail Their Users: A Large-Scale Analysis of Developer-Agent Misalignment in 20,574 Real-World Sessions
• Accurate Failure Prediction in Agents Does Not Imply Effective Failure Prevention
• OpenHands: An Open Platform for AI Software Developers as Generalist Agents

Related papers

• Exploring Solution Divergence and Its Effect on Large Language Model Problem Solving
• Ishigaki-IDS-Bench: A Benchmark for Generating Information Delivery Specification from BIM Information Requirements
• Subliminal Steering: Stronger Encoding of Hidden Signals
• MedStruct-S: A Benchmark for Key Discovery, Key-Conditioned QA and Semi-Structured Extraction from OCR Clinical Reports
• The End of Transformers? On Challenging Attention and the Rise of Sub-Quadratic Architectures
• Untangling the Mechanisms of Misleading Context in Medical Question Answering