Agent Behavior Maps: Locating Agents by What They Would Do, Without Running Them

arXiv:2610.30014 · cs.LG, cs.AI, cs.CL · Submitted 2026-10-21 · Read on arXiv

cs.LG, cs.AI, cs.CL

Submitted: 2026-10-21

Updated: 2026-10-21

Comments: 23 pages, 5 figures, 12 tables. Under review

License: http://creativecommons.org/licenses/by/4.0/

The gist: LLM agents with similar success rates can follow different action sequences.

Abstract

LLM agents with similar success rates can follow different action sequences. An Agent Behavior Map represents LLM-harness combinations using centered action log-likelihoods on shared reference trajectories, without requiring new environment rollouts from the target agent. We derive error bounds relating squared coordinate distances to trajectory KL under explicit assumptions, and across eight cross-scored models the distances track the directly measured trajectory KL in rank order. Across 100 LLMs and four environments, ALFWorld-based coordinates support performance prediction and identify behavioral similarities. In ALFWorld, these similarities persist after controlling for success rates and model size, including across model families. As an extension, harness-component displacements approximately compose, with accuracy varying across components and environments.

Related papers