Partially Observable Learning for Multi-Platform Dispatch Optimization

This paper proposes POLO, a Partially Observable multi-agent reinforcement Learning framework for multi-platform dispatch Optimization in instant delivery systems. The problem is motivated by real-world scenarios where crowdsourced couriers may concurrently serve multiple platforms, and each platform can only observe its own orders and courier interactions due to privacy and operational constraints. This results in a multi-platform dispatch environment with inherent partial observability, which degrades the performance of existing methods that assume full courier observability and mandatory assignment acceptance.

The paper formalizes the dispatch problem under partial observability, where each platform-grid pair is modeled as an independent agent that learns dispatch policies solely from platform-local observations. The observation of each agent consists of four components: courier states (filtered by platform-local projection), unassigned orders, order-courier matching features, and demand-supply information. The policy network uses an attention-based inter-courier aggregation encoder to selectively aggregate information across the candidate courier set, capturing implicit interactions and competition among couriers. To mitigate non-stationarity induced by joint actions across grids, the paper designs a counterfactual reward shaping mechanism that isolates individual agent contributions by comparing the actual updated route with a counterfactual route obtained by removing the agent's dispatch action.

The framework is trained using an actor-critic architecture with a shared policy network across all agents. The reward function combines the value of on-time delivered orders, a counterfactual term, and a distance penalty term. The paper develops a high-fidelity simulator based on real-world data from Meituan, a major food delivery platform in China, and evaluates performance across three instance scales (Base, Median, Large) and varying numbers of platforms (1, 2, 3).

Extensive experiments demonstrate that POLO consistently outperforms strong baselines in terms of platform revenue (GMV) and courier travel efficiency (ACTD). Key findings include: (1) all learning-based methods degrade as the number of platforms increases, with degradation becoming more significant in large-scale scenarios; (2) POLO achieves superior GMV performance in most settings, with particularly clear advantages in large-scale scenarios (e.g., approximately 9.47% GMV improvement over TCAC with two platforms in the large-scale setting); (3) POLO consistently attains higher GMV and lower ACTD than other learning-based baselines, occupying a more favorable position in the GMV-ACTD trade-off analysis. The paper also demonstrates robustness under spatial and temporal distribution shifts, and ablation studies confirm the effectiveness of both the attention mechanism and the counterfactual reward design. POLO also achieves the lowest inference time among learning-based methods due to parallel order assignment across grids.

Improvements to AI Systems:

1. Partial-Observability-Aware Multi-Agent Coordination: Replace the common assumption of full observability in multi-agent dispatch systems with a framework that explicitly models each agent’s local observation space (e.g., platform-local courier states, filtered order data). This allows AI systems to operate effectively in federated or privacy-constrained environments where data sharing is prohibited, without performance collapse.

2. Attention-Based Selective Aggregation for Implicit Competition: Integrate an attention encoder that selectively aggregates information across candidate couriers, learning which courier interactions (e.g., shared order pools, route overlaps) matter most for dispatch decisions. This improves the system’s ability to infer unobserved courier competition and avoid redundant assignments, even when full courier states are hidden.

3. Counterfactual Reward Shaping for Non-Stationary Joint Actions: Design a reward mechanism that isolates an agent’s contribution by comparing the actual route update against a counterfactual route (where the agent’s action is removed). This stabilizes training in environments where multiple agents’ actions jointly affect the global state, reducing variance and improving credit assignment in multi-agent reinforcement learning.

4. Shared Policy with Parallel Grid-Level Dispatch: Use a single, shared policy network across all agent-grid pairs, enabling parallel order assignment across grids. This reduces inference time significantly (lowest among learning-based methods) while maintaining scalability, making real-time dispatch feasible for large-scale, multi-platform operations.

5. Robustness to Distribution Shifts via Localized Training: Train the system on platform-local observations and counterfactual rewards, which inherently makes it more resilient to spatial and temporal shifts (e.g., demand spikes, courier availability changes) because it does not rely on global, time-varying coordination signals that may become stale or inaccurate.

---

What the Improved AI System Can Do:

• Operate in Multi-Platform Ecosystems: Manage dispatch for multiple competing platforms simultaneously, where each platform only sees its own orders and couriers, without needing cross-platform data sharing—preserving privacy and business confidentiality.

• Adapt to Large-Scale, Real-Time Scenarios: Dispatch thousands of couriers and orders across hundreds of grids in milliseconds, with lower latency than existing learning-based methods, due to parallel grid-level decision-making.

• Optimize Both Revenue and Efficiency: Achieve higher platform revenue (e.g., 9.47% GMV improvement over strong baselines in large-scale settings) while reducing courier travel distance (ACTD), striking a better trade-off between business goals and operational cost.

• Maintain Performance Under Uncertainty: Continue to perform well when demand patterns shift (e.g., holidays, weather events) or when courier availability changes dynamically, because the policy learns from local, robust features rather than fragile global assumptions.

• Learn Stable Policies in Non-Stationary Environments: Avoid training instability caused by multiple agents changing their behavior simultaneously, thanks to counterfactual reward shaping that provides consistent, action-specific feedback.