Harness Engineering in LLM Tool Use via Agent-Native Reusable Tool Primitives

arXiv:2609.01736 · cs.SE, cs.AI, cs.CL, cs.LG, cs.MA · Submitted 2026-09-01 · Read on arXiv

cs.SE, cs.AI, cs.CL, cs.LG, cs.MA

Submitted: 2026-09-01

Updated: 2026-09-01

Code: https://github.com/tatsu-lab/stanford_alpaca

Terminology

• Verifier — 29× in this paper · explained in 2 episodes
• Benchmark — 22× in this paper · explained in 2 episodes
• Verification — 12× in this paper · explained in 2 episodes
• Consistency — 9× in this paper · explained in 3 episodes
• Robustness — 9× in this paper · explained in 5 episodes
• Accuracy — 8× in this paper · explained in 1 episode
• Ablation — 5× in this paper · explained in 1 episode
• Fine-Tuning — 4× in this paper · explained in 5 episodes
• Multi-Agent System — 4× in this paper · explained in 6 episodes
• Ablation study — 3× in this paper · explained in 1 episode
• Abstraction — 3× in this paper · explained in 1 episode
• Adam — 3× in this paper · explained in 2 episodes
• Benchmarking — 3× in this paper · explained in 3 episodes
• Agent Skills — 2× in this paper · explained in 1 episode
• Core — 2× in this paper · explained in 2 episodes
• Large Language Models (LLMs) — 2× in this paper · explained in 6 episodes
• Ablation Studies — 1× in this paper · explained in 1 episode
• Agentic Reasoning — 1× in this paper · explained in 1 episode
• Distillation — 1× in this paper · explained in 2 episodes
• F1 Score — 1× in this paper · explained in 3 episodes
• Failure Modes — 1× in this paper · explained in 3 episodes
• Foundation Model — 1× in this paper · explained in 3 episodes
• Foundation Models — 1× in this paper · explained in 7 episodes
• Generalization — 1× in this paper · explained in 13 episodes
• In-Context Learning — 1× in this paper · explained in 4 episodes
• In-Context Learning (ICL) — 1× in this paper · explained in 5 episodes
• LLM-as-a-Judge — 1× in this paper · explained in 3 episodes
• Standardized Benchmarks — 1× in this paper · explained in 2 episodes
• Supervised Fine-Tuning (SFT) — 1× in this paper · explained in 8 episodes

Sources

• ToolLLM: Facilitating Large Language Models to Master 16000+ Real-world APIs
• Hammer: Robust Function-Calling for On-Device Language Models via Function Masking
• ToolACE: Winning the Points of LLM Function Calling
• APIGen-MT: Agentic Pipeline for Multi-Turn Data Generation via Simulated Agent-Human Interplay
• $\tau^2$-Bench: Evaluating Conversational Agents in a Dual-Control Environment
• LongFuncEval: Measuring the effectiveness of long context models for function calling
• Agent Primitives: Reusable Latent Building Blocks for Multi-Agent Systems
• $\tau$-bench: A Benchmark for Tool-Agent-User Interaction in Real-World Domains
• GPT-4 Technical Report
• The Llama 3 Herd of Models
• Mixtral of Experts
• DeepSeek-V3 Technical Report
• GPT-4o System Card
• Qwen2.5 Technical Report
• Phi-3 Technical Report: A Highly Capable Language Model Locally on Your Phone
• Qwen3 Technical Report
• Gemini 1.5: Unlocking multimodal understanding across millions of tokens of context
• Prompt Injection Attack to Tool Selection in LLM Agents

Related papers

• Falsification-Based Verification of LLM-Generated Optimization Models: Sound Test Batteries and Their Detection Limits
• GitSkills: A Dataset of Agent Skills on GitHub
• SABER: Benchmarking Operational Safety of LLM Coding Agents in Stateful Project Workspaces
• PackMonitor: Enabling Zero Package Hallucinations Through Decoding-Time Monitoring
• IntentCoding: Amplifying User Intent in Code Generation
• Incentives and Outcomes in Bug Bounties