QBugLM: An Agentic Benchmarking Framework for LLM-based Quantum Software Debugging

arXiv:2606.07314 · cs.SE, cs.ET, quant-ph · Submitted 2026-06-05 · Read on arXiv

cs.SE, cs.ET, quant-ph

Submitted: 2026-06-05

Updated: 2026-06-05

Code: https://github.com/qachub/qbuglm

Terminology

• Benchmarking — 14× in this paper · explained in 3 episodes
• Accuracy — 9× in this paper · explained in 1 episode
• Benchmark — 8× in this paper · explained in 2 episodes
• Quantum Circuits — 6× in this paper · explained in 2 episodes
• Structured Prompting — 5× in this paper · explained in 2 episodes
• Resource Constraints — 4× in this paper · explained in 3 episodes
• Large Language Models (LLMs) — 3× in this paper · explained in 6 episodes
• ACM International Conference on Automated Software Engineering — 2× in this paper
• Formal Verification — 2× in this paper · explained in 2 episodes
• Multi-Agent Framework — 2× in this paper · explained in 3 episodes
• Verification — 2× in this paper · explained in 2 episodes
• Adaptation — 1× in this paper · explained in 1 episode
• Agentic Framework — 1× in this paper · explained in 6 episodes
• Chain-of-Thought (CoT) — 1× in this paper · explained in 4 episodes
• Chain-of-Thought Prompting — 1× in this paper · explained in 2 episodes
• Entanglement — 1× in this paper · explained in 3 episodes
• Failure Modes — 1× in this paper · explained in 3 episodes
• Fine-Tuning — 1× in this paper · explained in 5 episodes
• International Conference on Quantum Computing and Engineering — 1× in this paper
• Iterative Refinement — 1× in this paper · explained in 3 episodes
• Multi-Agent System — 1× in this paper · explained in 6 episodes
• Provenance — 1× in this paper · explained in 3 episodes
• Redundancy — 1× in this paper · explained in 2 episodes
• Reinforcement Learning — 1× in this paper · explained in 4 episodes
• Trace — 1× in this paper · explained in 2 episodes

Sources

• Qiskit Code Assistant: Training LLMs for generating Quantum Computing Code
• Qiskit HumanEval: An Evaluation Benchmark For Quantum Code Generative Models
• Leveraging Mutation Analysis for LLM-based Repair of Quantum Programs
• Chain-of-Thought Prompting Elicits Reasoning in Large Language Models
• ReAct: Synergizing Reasoning and Acting in Language Models
• Evaluating Large Language Models Trained on Code
• Asking the Right Questions: Improving Reasoning with Generated Stepping Stones

Related papers

• Falsification-Based Verification of LLM-Generated Optimization Models: Sound Test Batteries and Their Detection Limits
• GitSkills: A Dataset of Agent Skills on GitHub
• SABER: Benchmarking Operational Safety of LLM Coding Agents in Stateful Project Workspaces
• PackMonitor: Enabling Zero Package Hallucinations Through Decoding-Time Monitoring
• IntentCoding: Amplifying User Intent in Code Generation
• Incentives and Outcomes in Bug Bounties