Online Planning for Sparse Ground Target Search from a High-Altitude UAV under Partial Observability

Unmanned aerial vehicles (UAVs) searching for sparse ground targets from high altitudes face a unique challenge when targets are small and unobservable, necessitating novel sensing strategies. The core contribution of this work is formulating sequential exploration with Pan-Tilt-Zoom (PTZ) operation as a partially observable Markov decision process (POMDP) to optimize target discovery while explicitly modeling observation uncertainty conditioned on the target's scale.

The gist: This work formulates the sequential exploration with PTZ operation as a partially observable Markov decision process (POMDP), in which the agent maintains a belief state over the target’s true location. To solve this POMDP, we deploy partially observable Monte Carlo planning (POMCP), where we condition the sensing reliability on target object scale and deploy selective ensemble detection as an additional reasoning step.

Problem Formulation and Modeling Uncertainty

The search problem is defined by discretizing the search space using a family of windows, parameterized by top-left corner coordinates (r, c) and side length l, denoted as W(L). The state space (S), action space (A), and observation space (O) are instantiated from these windows. Crucially, the transition model is simplified to the identity: T(s′s, a) = 1[s′ = s] for all s, s′ ∈ S, a ∈ A, as the target is assumed static. The observation model uncertainty is explicitly modeled using a Beta-Bernoulli conjugate prior. This uncertainty depends on both the presence indicator of the target (bp) and its apparent scale ratio ρ, where ρ = la / l. The probability of detection is decomposed into two cases: when bp = 1, the observation follows a Gaussian distribution conditioned on the scale ratio, while when bp = 0, it follows a Bernoulli distribution.

POMDP and Planning Framework

The control policy operates directly on the belief space to sequentially actuate PTZ operations to optimize long-term objectives. The belief state (bt) is maintained as an unweighted particle set of K hypotheses and updated via rejection sampling after each observation, approximating the true Bayesian posterior. To solve this POMDP, Partially Observable Monte Carlo Planning (POMCP) is deployed. The planning process involves running NSIM simulations from the current belief node, denoted as root node, traversing the tree using a UCB action selection criterion to balance exploitation and exploration: a∗ = arg max a∈A 'V (ha) + c s ln N(h) / N(ha) for action selection.

Observation Model and Selective Ensemble Detection

The observation model is decomposed based on the presence indicator bp. When bp = 1, the observation likelihood is modeled using: Z(o s, a) = θ+(ρ) · N ([ro, co, so]; [r, c, l], Σ) o ≠ ∅ / 1 − θ+(ρ) o = ∅ for non-null observations. When bp = 0, the model uses: Z(o s, a) = θ−(ρ) Nfp o ≠ ∅ / 1 − θ−(ρ) o = ∅. To enhance reasoning when belief concentration is high, the methodology includes selective ensemble detection. This is triggered when the belief concentration exceeds a threshold τe, where POMCP-PLAN suggests a scan action covering the MAP with scale ratio ρ = 1, we swap the default scan for an ensemble scan involving five models trained on different subsets of the dataset.

Reward Structure and Termination Policy

The reward structure is decomposed into in-tree step rewards and a terminal declaration reward. The in-tree reward function is defined as: R(s, a) = (Rstep + Rdet − Rfp o ≠ ∅ / Rstep o = ∅) where "Rstep < 0 is the per-step cost encouraging minimization of detector inferences. The terminal reward, applied outside the POMCP tree when termination criteria are met, is defined as: RT(ˆst) = (R+ sˆt = s∗ / R− sˆt ≠ s∗), where R+ ≫ R− determines the terminal reward, s∗ denotes the true target state. Target declaration occurs when belief concentration exceeds a rational threshold τ: The agent declares the MAP state sˆt when belief concentration exceeds the rational threshold τ."

Validation and Results

The methodology was validated in a photorealistic simulator built in Unreal Engine under various environmental conditions and vehicle states. Experiments compared POMCP-PTZ against baselines like Fixed Scan, Exhaustive-Crop, and Coarse-to-Fine methods across different altitudes (100m, 200–350m) and camera resolutions (640 × 640).

Here are the specific improvements that can be made to existing AI systems by applying the methodology described in this paper, along with what these improved systems can achieve:

Improvements for AI Systems

1. Transition from Static/Fixed-Scale Detection to Dynamic, Uncertainty-Aware Active Sensing:

2. Integration of Belief State Planning (POMDP) into Perception Pipelines:

3. Adaptive Detection Strategy via Selective Ensemble Deployment:

4. Robustness to Scale and Resolution Variation in Aerial Target Search:

What the Improved AI System Can Do

The improved AI system, based on the POMCP-PTZ framework, can perform the following specific functions:

1. Efficient Sparse Target Localization Under Partial Observability:

Specifically, it can successfully locate small ground targets (like helipads) from high altitudes where standard detection methods fail due to resolution downscaling and limited context. It achieves this by dynamically deciding the optimal Pan-Tilt-Zoom (PTZ) action—balancing wide-scan exploration for initial saliency against precise scans for reliable detection—to navigate the partially observable environment.

2. Sequential, Information-Maximizing Search Policies:

Instead of relying on fixed scanning patterns (like Fixed Scan or Exhaustive Crop), the system can execute a sequence of actions (PTZ adjustments) that are explicitly optimized to maximize the probability of target detection within a minimum number of steps, effectively modeling human-like spatial attention and search efficiency.

3. Adaptive Inference Resource Management:

The system intelligently manages computational load by selectively deploying computationally expensive multi-model ensembles only when the belief state is sufficiently concentrated (i.e., when the confidence threshold is met). This prevents unnecessary high-cost inference during early, uncertain exploration phases, leading to significant reduction in overall computational overhead compared to always running large ensembles.

4. Uncalibrated Confidence Reasoning:

By explicitly modeling observation uncertainty using a Beta-Bernoulli conjugate model conditioned on the target object's apparent scale (derived from PTZ settings), the system can generate more reliable and calibrated confidence scores for detections, even when sensor resolution is severely degraded or highly variable. This allows it to distinguish between true positives and false positives with greater statistical rigor than methods that rely solely on fixed detection thresholds.

5. Robustness Across Varying Environmental Conditions (Altitude/Resolution):

The system demonstrates superior performance in high-altitude scenarios (e.g., 350m) where traditional methods fail due to severe resolution downscaling, and it maintains competitive performance even when camera resolutions are varied, showing a resilience that fixed-slicing or coarse-to-fine methods lack.

Unmanned aerial vehicles (UAVs) searching for ground targets from a high altitude face a unique challenge, particularly when the target is already within the field of view but effectively unobservable because of its small apparent scale. Standard object detectors often underperform in such scenarios because of resolution downscaling and limited context. In contrast, active object search frameworks address this challenge by directing the agent to a suitable pose to gather richer visual information. However, flight regulations in urban airspace often restrict such physical movements for UAVs. As an alternative active sensing approach, the UAV can leverage the pan-tilt-zoom (PTZ) mechanism of the onboard camera to dynamically adjust its field of view and sequentially gather enhanced visual information from specific regions of interest. Once the candidate locations of the target are identified, it can deploy more expensive object detection schemes, such as an ensemble of multiple models, to get better reasoning at a fixed scale. In this work, we formulate the sequential exploration with PTZ operation as a partially observable Markov decision process (POMDP), in which the agent maintains a belief state over the target's true location. To solve the POMDP, we deploy partially observable Monte Carlo planning (POMCP), where we condition the sensing reliability on target object scale and deploy selective ensemble detection as an additional reasoning step. We validate our methodology with experiments in a photorealistic simulator under different environmental conditions and vehicle states, showing detection of ground targets at variable scales with significantly fewer steps and minimal dependence on sensor resolution compared to baseline methods.