When Guardrails Look Effective: Construct Validity Failures in LLM Agent Commerce Evaluation
cs.AI
Submitted: 2026-09-01
Updated: 2026-09-01
Comments: Submitted to the NeurIPS 2026 Trust-AI-Eval Workshop. 7 pages, 2 figures, 3 tables. The accompanying artifact is available at https://anonymous.4open.science/r/a2a-evaluation-artifact-staging-2F25/
License: http://arxiv.org/licenses/nonexclusive-distrib/1.0/
Terminology
Sources
- Measuring Bargaining Abilities of LLMs: A Benchmark and A Buyer-Enhancement Method
- Magentic Marketplace: An Open-Source Environment for Studying Agentic Markets
- Qwen2.5 Technical Report
- When Aggregate Alignment Misleads: Auditing Policy Repair Without Per-State Expert Actions
- Measuring what Matters: Construct Validity in Large Language Model Benchmarks
- When Outcome Looks Right But Discipline Fails: Trace-Based Evaluation Under Hidden Competitor State
- Algorithmic Collusion by Large Language Models
- When LLM Agents Negotiate: Private Information and Dynamic Bargaining in Supply Chains
- Holistic Evaluation of Language Models
Related papers
- MAVEN-T: Reinforced Heterogeneous Distillation for Real-Time Multi-Agent Trajectory Prediction
- Model Discovery Agent: LLM-assisted Bayesian experiment design for data-efficient discovery of mechanistic world models
- The Clinician's Veto: Navigating Trust, Liability, and Uncertainty in Autonomous AI Prescribing
- MindHelper: Closed-Loop Embodied Mental-State Reasoning for Precision Intervention
- Incumbent Advantage: Brand Bias and Cognitive Manipulation Dynamics in LLM Recommendation Systems
- VSAL: A Vision Solver with Adaptive Layouts for Graph Property Detection