TomasuLLM: Out-of-Order Speculative Execution for LLM Agents

arXiv:2609.38201 · cs.CL, cs.OS, cs.SE · Submitted 2026-09-22 · Read on arXiv

cs.CL, cs.OS, cs.SE

Submitted: 2026-09-22

Updated: 2026-10-01

Code: https://github.com/abundant-ai/swe-marathon

Terminology

• Trace — 41× in this paper · explained in 2 episodes
• Benchmark — 12× in this paper · explained in 2 episodes
• Decode — 9× in this paper · explained in 2 episodes
• Ablation — 6× in this paper · explained in 1 episode
• Accuracy — 4× in this paper · explained in 1 episode
• Agent Harness — 4× in this paper · explained in 2 episodes
• Alignment — 3× in this paper · explained in 1 episode
• Core — 3× in this paper · explained in 2 episodes
• Speculative Decoding — 3× in this paper · explained in 2 episodes
• Adaptation — 2× in this paper · explained in 1 episode
• Verification — 2× in this paper · explained in 2 episodes
• World Model — 2× in this paper · explained in 7 episodes
• Agentic System — 1× in this paper · explained in 2 episodes
• Agentic Workflows — 1× in this paper · explained in 1 episode
• Annual International Symposium on Computer Architecture — 1× in this paper
• Consistency — 1× in this paper · explained in 3 episodes
• Fine-Tuning — 1× in this paper · explained in 5 episodes
• Transformer — 1× in this paper · explained in 2 episodes
• Verifier — 1× in this paper · explained in 2 episodes

Sources

• SPORK: Self-Speculative Forking to Accelerate Agentic LLM Inference
• Accelerating Large Language Model Decoding with Speculative Sampling
• AOSpec: Action and Observation Co-Speculation for Low-Latency Agent Serving
• SWE-Marathon: Can Agents Autonomously Complete Ultra-Long-Horizon Software Work?
• Speculate While You Reason: Teaching Agents to Predict Their Next Tool Call via Joint Agent-Speculator RL
• SWE-bench: Can Language Models Resolve Real-World GitHub Issues?
• An LLM Compiler for Parallel Function Calling
• GAIA: a benchmark for General AI Assistants
• Atomix: Timely, Transactional Tool Use for Reliable Agentic Workflows
• Toolformer: Language Models Can Teach Themselves to Use Tools
• Parallelizing Tool Execution and LLM Generation for Low-Latency Agent Serving
• Speculative Actions: A Lossless Framework for Faster Agentic Systems
• DualSpec: Accelerating Deep Research Agents via Dual-Process Action Speculation
• Qwen-AgentWorld: Language World Models for General Agents

Related papers

• Exploring Solution Divergence and Its Effect on Large Language Model Problem Solving
• Ishigaki-IDS-Bench: A Benchmark for Generating Information Delivery Specification from BIM Information Requirements
• Subliminal Steering: Stronger Encoding of Hidden Signals
• MedStruct-S: A Benchmark for Key Discovery, Key-Conditioned QA and Semi-Structured Extraction from OCR Clinical Reports
• The End of Transformers? On Challenging Attention and the Rise of Sub-Quadratic Architectures
• Untangling the Mechanisms of Misleading Context in Medical Question Answering