Autonomous Event-Driven Multi-Agent Orchestration for Enterprise AI at Scale

Enterprise AI aims to move toward continuous event monitoring, detection, and action across specialist agents, yet existing multi-agent systems largely assume discrete request-response workflows and remain underexplored at enterprise scale. The gist: scale, not task complexity, dominates orchestration performance as agent discovery noise becomes the primary bottleneck in enterprise settings.

Orchestration Architectures Compared

The paper evaluates two primary orchestration architectures across 208 production-derived scenarios spanning Persona (<10 agents), Department (20–80 agents), and Enterprise (200) scales: DAG Plan & Execute and ReAct. The core difference lies in their approach to planning: DAG Plan & Execute uses a structured approach where a Planner generates an execution graph upfront, separating planning from execution, while ReAct is the default architecture where a single model interleaves reasoning and action in a continuous loop without upfront planning.

The Role of the Task Manager

A critical component introduced to support continuous operation is the Task Manager, which functions analogously to a process scheduler in an operating system kernel. This manager handles incoming events by leveraging LLM-powered decision making to determine whether to create a new task or merge it into an existing one, and assigns the appropriate priority level based on event content and context. It manages tasks through states like QUEUED, ACTIVE, and COMPLETED. The Task Manager integrates with both architectures at defined stoppage points—checkpoints in DAG Plan & Execute or after an agent call returns in ReAct—to enable preemption or task updates (from merged events).

Benchmark Design and Scenario Taxonomy

The evaluation dataset comprises 208 scenarios derived from actual user stories, structured into five types to test specific orchestration capabilities:

1. Simple scenarios (1–3 agents) establish baseline competency.

2. Parallel scenarios (4–8 agents) test concurrent execution of independent subproblems.

3. Complex scenarios (5–10 agents) combine dependency handling with parallel work, creating multi-stage execution paths where upstream agents provide entity identifiers required by downstream agents while independent branches can proceed concurrently.

4. Failure scenarios introduce incomplete or ambiguous responses, requiring the orchestrator to detect missing information, retry with refined requests, or proceed with partial data.

5. Priority and related-event scenarios test the Task Manager’s continuous operation capabilities, verifying correct triage when critical events (equipment failures, safety incidents) arrive during lower-priority work and requiring systems to merge related events into a unified task.

Performance Findings Across Scales

The results show that scale, not task complexity, dominates orchestration performance, as both architectures degrade at enterprise scale due to agent discovery noise. ReAct is noted as the stronger default when discovery is noisy, while DAG Plan & Execute shows advantages on structured workflows at small-to-medium scales through better parallelization and precision. The Task Manager demonstrated significant value, reducing high-priority queue latency by 14–75% and improving related-event correctness by over 20 percentage points at enterprise scale.

Enterprise Deployment Implications

The choice between architectures should be driven less by task complexity than by registry scale and recovery cost. DAG Plan & Execute is attractive when the relevant agent set is small or moderately sized, but its plan-interrupt-replan overhead compounds errors at enterprise scale. ReAct is the stronger default when discovery is noisy, as its stepwise interaction gives it fewer opportunities to overcommit to an incorrect global plan, making it more robust at 200-agent scale. The Task Manager's value is most pronounced when the workload is continuous, enabling a scheduler for event streams through priority inference and preemption. Furthermore, the paper emphasizes that safety decisions must remain with workflow-specific specialist agents rather than the orchestrator itself.

Key Metrics and Analysis

The evaluation used LLM-as-a-Judge evaluation to score correctness by comparing the orchestrator’s final synthesized response against expected answer fragments. Precision and recall analysis revealed that DAG achieves higher precision at every scale as upfront planning selects agents more carefully, but ReAct’s recall holds up better because it lacks the machinery that compounds errors. Cost analysis showed that all systems converge toward higher consumption (20–55K per event) at Enterprise scale as discovery retries and replanning dominate. The findings confirm that the bottleneck at scale is agent selection rather than the orchestration architecture itself.

Limitations of the Study

The study notes several limitations, including mock agent latency, which compresses queue dynamics, fixed orchestration parameters, and the exclusion of persistent memory to establish a controlled baseline independent of learned optimizations. Additionally, all experiments assume a single orchestration worker; future work is needed to address distributed scheduling challenges. The paper concludes that enterprise systems should monitor signals like backlog age, preemption frequency, merge accuracy, repeated agent calls, and discovery recall to diagnose the primary bottleneck.

Here are specific improvements for AI systems based on the scientific paper, detailing what the improved system can achieve:

1. The introduction of a dedicated, LLM-powered Task Manager layer fundamentally shifts enterprise AI from reactive chatbots to autonomous digital workers capable of continuous operation.

2. This Task Manager enables high-priority event preemption and related-event merging, allowing the system to handle asynchronous event streams (alerts, user requests) rather than discrete request-response cycles.

3. The improved system can maintain a deterministic priority queue (LOW, MEDIUM, HIGH) that prevents task starvation by incorporating task age and progress factors into its scoring function.

4. The AI can perform related-event merging, synthesizing fragmented events pertaining to the same underlying situation into a single unified task, significantly improving context coherence for long-horizon operations.

5. For enterprise scales (200 agents), the system utilizes a flexible orchestration choice:

6. If the agent discovery space is noisy (high scale), it defaults to the ReAct architecture, which handles failures incrementally and avoids costly full replanning cycles when faced with unreliable agent returns.

7. If tasks involve structured workflows with known dependencies at small-to-medium scales, it utilizes the DAG Plan & Execute architecture, leveraging upfront planning to ensure precision and structured parallelization.

8. The system can dynamically adjust execution strategy based on scale: ReAct is preferred for robustness against discovery noise at enterprise scale, while DAG excels where explicit parallelization and dependency mapping are beneficial at smaller scales.

9. The improved system incorporates two-tier agent discovery (lightweight summaries first, full cards on demand), ensuring the planning phase remains computationally bounded even with hundreds of registered agents.

10. The AI can perform deterministic guardrails by validating LLM decisions against configuration constraints before mutating the Task Backlog, ensuring no event is silently dropped and actions remain within operational/business boundaries.

11. The system separates orchestration from safety: the orchestrator schedules work and routes context, while specialized domain agents (e.g., a Finance agent) retain ultimate authority for enforcing workflow-specific business rules (e.g., segregation of duties checks).

12. The AI can generate precise execution plans using the DAG Planner's two-phase approach: first, determining necessary information via a directed reasoning step, and second, constructing the execution graph by strictly enforcing dependency minimization and requiring explicit entity identifiers in node descriptions.

13. The system can dynamically adapt its plan through checkpoint mechanisms—pausing execution before replanning to inject merged event context or preempt the current task for higher-priority work—ensuring continuous responsiveness.

14. The AI can use LLM-as-a-Judge evaluation, comparing the orchestrator’s final synthesized response against expected answer fragments derived from agent guidelines, providing a quantifiable correctness score based on semantic presence of required information.

15. The improved system can translate complex process mining data into prescriptive actions using the Dashboard Analyzer Agent, allowing it to autonomously interpret event logs and recommend next best actions for continuous process optimization.