The Tasteful Agent: Measuring and Improving Taste in Long-Horizon Tasks

arXiv:2609.25804 · cs.AI · Submitted 2026-09-22 · Read on arXiv

cs.AI

Submitted: 2026-09-22

Updated: 2026-09-23

Comments: 33 pages, 6 figures. Code: https://github.com/wbopan/tastebench. Dataset: https://huggingface.co/datasets/wenbopan/taste-bench

Code: https://github.com/wbopan/tastebench

License: http://arxiv.org/licenses/nonexclusive-distrib/1.0/

Terminology

• Accuracy — 44× in this paper · explained in 1 episode
• Benchmark — 25× in this paper · explained in 2 episodes
• Distillation — 18× in this paper · explained in 2 episodes
• Trace — 9× in this paper · explained in 2 episodes
• Embeddings — 6× in this paper · explained in 2 episodes
• Verification — 6× in this paper · explained in 2 episodes
• Alignment — 3× in this paper · explained in 1 episode
• Calibration — 2× in this paper · explained in 7 episodes
• Core — 2× in this paper · explained in 2 episodes
• Transformer — 2× in this paper · explained in 2 episodes
• AI Agents — 1× in this paper · explained in 2 episodes
• Adam — 1× in this paper · explained in 2 episodes
• Adaptation — 1× in this paper · explained in 1 episode
• Agentic System — 1× in this paper · explained in 2 episodes
• Benchmarking — 1× in this paper · explained in 3 episodes
• Fine-Tuning — 1× in this paper · explained in 5 episodes
• Generative AI — 1× in this paper · explained in 2 episodes
• Knowledge Distillation — 1× in this paper · explained in 11 episodes
• LLM-as-a-Judge — 1× in this paper · explained in 3 episodes
• LoRA Adapters — 1× in this paper · explained in 3 episodes
• Multi-Agent System — 1× in this paper · explained in 6 episodes
• On-Policy Distillation — 1× in this paper · explained in 3 episodes
• Reasoning Traces — 1× in this paper · explained in 3 episodes
• Reinforcement Learning — 1× in this paper · explained in 4 episodes
• Self-distillation — 1× in this paper · explained in 6 episodes
• Steering — 1× in this paper · explained in 2 episodes
• Synthetic Data Generation — 1× in this paper · explained in 5 episodes

Sources

• Kosmos: An AI Scientist for Autonomous Discovery
• SWE-EVO: Benchmarking Coding Agents in Long-Horizon Software Evolution Scenarios
• Live-SWE-agent: Can Software Engineering Agents Self-Evolve on the Fly?
• AlphaEvolve: A coding agent for scientific and algorithmic discovery
• HCAST: Human-Calibrated Autonomy Software Tasks
• DeepSeek-V4: Towards Highly Efficient Million-Token Context Intelligence
• GLM-5: from Vibe Coding to Agentic Engineering
• Distilling the Knowledge in a Neural Network
• A General Language Assistant as a Laboratory for Alignment
• Learning by Distilling Context

Related papers

• MAVEN-T: Reinforced Heterogeneous Distillation for Real-Time Multi-Agent Trajectory Prediction
• Model Discovery Agent: LLM-assisted Bayesian experiment design for data-efficient discovery of mechanistic world models
• The Clinician's Veto: Navigating Trust, Liability, and Uncertainty in Autonomous AI Prescribing
• MindHelper: Closed-Loop Embodied Mental-State Reasoning for Precision Intervention
• Incumbent Advantage: Brand Bias and Cognitive Manipulation Dynamics in LLM Recommendation Systems
• VSAL: A Vision Solver with Adaptive Layouts for Graph Property Detection