An Agentic AI Framework with Large Language Models and Chain-of-Thought for UAV-Assisted Logistics Scheduling with Mobile Edge Computing

Summary

This paper studies a hybrid scheduling problem in cloud manufacturing (CMfg) where unmanned aerial vehicles (UAVs) support both product collection and mobile edge computing (MEC). The authors state: "In this paper, UAVs collect finished products from manufacturing stations and transport them back to a central depot. Meanwhile, computational tasks generated by industrial sensor devices at these stations are processed locally, at UAVs, or offloaded via UAVs to the cloud. The coupling between physical logistics and computational task scheduling makes the problem challenging because A UAV can provide MEC services only during its service window at a station, so routing decisions directly determine when UAV-assisted offloading is available. Routing decisions also affect the UAV energy budget and the availability of onboard computing and communication resources for computational task execution under task deadline constraints."

The paper identifies two main challenges. Challenge I is that The mathematical modeling of combinatorial optimization problems in CMfg can be significantly more complex than that of traditional single-domain scheduling problems, particularly because "It requires capturing how UAV routing induces time-varying service windows. These service windows, in turn, affect per-slot task offloading and compute/communication allocation under energy budgets, capacity limits, and task deadlines. Challenge II is that Compared with traditional scheduling problems, hybrid task scheduling in CMfg introduces significantly higher solution complexity because logistics- and computation-related decisions must be optimized jointly rather than handled in isolation."

To address Challenge I, the authors propose "an agentic AI framework to address Challenge I. The agent facilitates dialogue-driven mathematical model generation and refinement by combining LLM reasoning with retrieval augmented generation (RAG) and chain-of-thought (CoT) reasoning. Specifically, an LLM converts natural language descriptions into structured modeling logic. Then, RAG extracts pertinent formulations, constraints, and objectives from curated literature and technical documentation. CoT further makes explicit the logic underlying variable definitions, constraint design, and objective construction."

To address Challenge II, the authors propose a hierarchical DRL approach to solve the coupled UAV-assisted collection and mobile edge computing (MEC) scheduling problem. The framework "decomposes the original joint optimization problem into two tractable sequential decision layers, which substantially reduces the effective state/action space while preserving the essential coupling between logistics and computation services. Both layers are formulated as finite-horizon Markov decision processes (MDPs). They are solved using proximal policy optimization (PPO). The upper layer formulates the multi-UAV routing problem as a single-agent MDP. It jointly determines the manufacturing-station assignment for each UAV and the corresponding visiting order under energy, payload, and flight-distance constraints. The lower layer makes per-slot execution decisions—offloading destination (local/UAV/cloud via UAV) and compute/communication allocation—under capacity and deadline constraints. The two layers interact through two signals: The upper layer provides service windows and remaining UAV energy to the lower layer, which uses them to determine the feasibility of UAV-involved processing."

The system model considers a fleet of U homogeneous UAVs dispatched from a central depot with terrestrial MEC servers (referred to as the cloud) to collect finished products from M geographically distributed manufacturing stations. Each station has a collection reward, product weight, and hosts a set of industrial sensor devices (ISDs) that generate computational tasks stochastically over a discretized time horizon. The UAV routing model includes constraints on collection assignment and route consistency, payload and distance budgets, and mission timing feasibility. The MEC service model includes constraints on execution-mode selection, completion-time encoding, computing and communication resource limits, service-window indicators, service-window feasibility for UAV-involved processing, deadline satisfaction, and UAV energy budgeting.

The complete optimization problem is formulated as: P: max ωcol Σ vm cm + ωcmp Σ zk,τ − ωmiss Σ (1 − zk,τ) − ωflow Σ (Tk,τ − τ∆) − ωres Σ (Rtcmp + Rtcom), s.t. Eq. (1) − Eq. (9), Eq. (10) − Eq. (27). The objective promotes product collection and deadline-compliant task completion, while reducing deadline violations, task completion delay, and overall computing and communication resource occupation.

The problem analysis shows that "the UAV routing part is NP-hard since it jointly determines station assignment and visit sequencing under route-consistency, payload, distance, and mission time constraints while maximizing collection value, which amounts to a capacitated value-collecting multi-UAV routing variant. The task offloading part is also NP-hard. Therefore, both subproblems, and thus the full joint formulation, are NP-hard, which motivates our proposed hierarchical DRL approach."

The proposed agentic AI framework includes a RAG module that is used to ground the LLM with task-relevant evidence before model generation. The RAG module organizes an internal knowledge base from two perspectives: UAV routing and computational task offloading. The retrieval process uses cosine similarity to measure relevance between the query and each chunk, and the top-Kret most relevant chunks form the context for generation. The CoT module reasons over the retrieved information and organizes it for downstream mathematical formulation. The framework includes two agents: "the Agentic AI Responder is responsible for reasoning over the user request and providing the final answer. In contrast, the Agentic AI Verifier evaluates another agent's responses, deciding whether to approve or reject it. The CoT workflow decomposes the user request into four steps: objective function formulation, UAV routing constraints derivation, task offloading constraints derivation, and mathematical notation summarization. For each CoT step, the Agentic AI Responder initially formulates a response to the specific CoT-step query; subsequently, the Agentic AI Verifier assesses the generated response prior to the continuation of the reasoning process."

The hierarchical DRL approach uses PPO in both layers. The upper-layer MDP state "summarizes four types of information: (i) per-UAV status, including location, remaining energy, payload, and flight-distance budget; (ii) route-progress information; (iii) per-station attributes; and (iv) mission-level context. The upper-layer action includes a priority score and routing actions, where each UAV selects one discrete routing symbol from stay, add(1),..., add(M), complete. The upper-layer reward includes collection value obtained during the transition," a shaping reward that encourages increasing the coverage ratio of collected stations, balanced route workloads across UAVs, and route termination rewards, minus an aggregated penalty that discourages infeasible decisions.

The lower-layer MDP state "summarizes blocal queues, UAV-side resources, and current service feasibility through six groups of normalized features: (i) per-ISD local statistics; (ii) per-UAV statistics; (iii) global system scalars; (iv) per UAV–ISD active offload status; (v) a UAV–ISD service-window availability mask; and (vi) a fixed-length, zero-padded snapshot of the top-Nq unfinished tasks. The lower-layer action is defined over a cached global-queue snapshot Qtb and specifies the service decisions for the first Kg queue positions. Each action tuple includes a processing choice for the queue item at position j selected from SKIP, local ∪ UAV(u), cloud-via-UAV(u) U u=1 and normalized compute and communication allocations that are quantized by the environment into discrete levels. The lower-layer reward combines positive incentives, penalties, and an end-of-episode settlement," including rewards for task completions, dispatch bonuses, dense progress rewards, shaping terms for backlog and urgency reduction, and penalties for deadline violations, infeasible decisions, avoidable idling, excessive resource usage, and remaining unfinished load.

The complexity analysis shows that the upper-layer training complexity is O(Nup T̄ up (U M + Pθ,ϕ up)) and the lower-layer training complexity is O(Nlo T̄ lo (U Kg + Nq + Kg U + Pθ,ϕ lo)). The authors note that "a monolithic MDP would need to enumerate joint routing and task-scheduling choices, whose size scales with (M + 2)U (2U + 2)Kg before considering resource-allocation levels. Therefore, the proposed hierarchical decomposition keeps the rollout and update costs polynomial in the main system sizes while preserving the service-window coupling between UAV routing and MEC task scheduling."

The training procedure is sequential: "First, the upper layer is trained by PPO for UAV routing... The best-performing upper-layer policy is then fixed to extract the UAV service-window and residual-energy information for the lower layer. Next, the lower layer is trained by PPO for task scheduling and resource allocation under this fixed routing information."

Simulation results are presented for both the agentic AI framework and the hierarchical DRL approach. For the agentic AI framework, the results show that "the proposed framework remains closer to the original modeling logic by preserving the intended meanings of the modeled terms, whereas the baseline tends to reformulate them into a different optimization-oriented representation. For example, In the RAG database, the penalty is defined through the abstract occupation variables Rtcmp and Rtcom, which represent normalized computing and communication occupation. The proposed framework preserves this same interpretation... By contrast, the baseline rewrites this penalty as a direct sum of computing and communication allocation variables. As a result, it changes the penalty from normalized resource occupation to raw resource usage."

For the upper-layer DRL training, "the upper-layer PPO shows good learning ability and convergence. In the early stage, the Raw reward fluctuates sharply, indicating active exploration. With training, MA(50) increases rapidly and then stabilizes around 400, indicating that the policy is improved rapidly and converges to a stable high reward solution. The collection rate quickly approaches 100% and remains at or near 100% in most episodes, indicating that the PPO policy can reliably plan UAV routing and complete almost all collection tasks."

For the lower-layer DRL training, "both PPO and A2C improve the lower-layer policy from highly negative rewards to a converged positive-reward region. PPO exhibits larger fluctuations in the intermediate stage, but its reward quickly recovers and remains relatively stable afterward. In contrast, the total reward under A2C increases more smoothly in the early stage, but several sharp reward drops still appear in the later stage, indicating weaker stability after convergence. Regarding deadline satisfaction, PPO maintains a rate very close to 100% more consistently in the later stage, whereas A2C still experiences several late-stage drops and cannot always sustain the ideal 100% deadline satisfaction rate."

The paper concludes that "the learned policies can achieve strong routing and scheduling performance, while the overall design preserved the essential coupling between logistics and computation through compact cross-layer information exchange. Overall, this work provides a unified framework for both formulating and solving hybrid logistics-computation scheduling problems in CMfg. Future work can extend the framework to larger-scale systems, heterogeneous UAV fleets, and more dynamic manufacturing environments."

Based on the paper, here are the specific improvements I can make to AI systems:

Improvement: Implement a two-agent (Responder + Verifier) architecture with Chain-of-Thought (CoT) decomposition and Retrieval-Augmented Generation (RAG) for automated optimization problem formulation.

Capabilities:

• Parse natural-language system descriptions into structured mathematical models (objective functions, constraints, decision variables)

• Decompose complex formulation tasks into 4 sequential CoT steps: objective definition, routing constraints, offloading constraints, notation summarization

• Retrieve domain-specific modeling knowledge (UAV routing, MEC offloading) from a curated vector database using cosine similarity

• Verify each intermediate reasoning step before proceeding, with automatic rethinking on failure

• Produce interpretable, traceable formulations that preserve semantic fidelity to domain knowledge (unlike single-agent baselines that alter meaning)

These improvements enable AI systems to: automatically generate correct optimization models from natural language, solve coupled logistics-computation problems with high reliability, maintain deadline guarantees in dynamic environments, and scale to larger problem instances with polynomial training complexity.

In cloud manufacturing, unmanned aerial vehicles (UAVs) can support both product collection and mobile edge computing (MEC). This joint operation forms a hybrid scheduling problem, where physical logistics decisions are coupled with computational task scheduling. In this paper, UAVs collect finished products from manufacturing stations and transport them back to a central depot. Meanwhile, computational tasks generated by industrial sensor devices at these stations are processed locally, at UAVs, or offloaded via UAVs to the cloud. This coupling makes the problem challenging. A UAV can provide MEC services only during its service window at a station, so routing decisions directly determine when UAV-assisted offloading is available. Routing decisions also affect the UAV energy budget and the availability of onboard computing and communication resources for computational task execution under task deadline constraints. To address this, we propose an agentic-AI-assisted optimization framework with two components. First, we develop an agentic AI that combines large language models, retrieval-augmented generation, and chain-of-thought reasoning to translate user input into an interpretable mathematical formulation for the hybrid scheduling problem. Second, we design a hierarchical deep reinforcement learning approach based on proximal policy optimization (PPO), where the upper layer learns UAV routing and the lower layer optimizes per-slot task execution and resource allocation. Simulation results show that the proposed framework yields more consistent formulations, while the hierarchical PPO achieves full product collection in 99.6% of the last 500 episodes and maintains a 100% deadline satisfaction rate, with more stable performance than the advantage actor-critic approach.