Illusion or Integrity? Geometrical Consistency Metric for AIGC Video Quality Evaluation

This paper introduces GeoCon-Bench, a novel, ground-truth-free (GT-free) evaluation framework for assessing the geometric consistency of AI-generated content (AIGC) videos. The authors argue that while existing video quality assessment (VQA) metrics focus on visual harmony, video-text consistency, and domain-specific alignment, they lack quantitative metrics for fidelity to physical laws. The paper's central thesis is that perceptual plausibility ≠ photogrammetric correctness — videos can look convincing while systematically violating multi-view geometric constraints, making them unusable for 3D reconstruction, pose estimation, and novel-view synthesis.

The authors identify that generative video offers a pathway to lower the cost of 3D asset creation, but this promise is ultimately gated by photogrammetry and multi-view geometry: usable parallax, consistency with either homography or epipolar structure, and coherent occlusion handling are prerequisites, not niceties. When these prerequisites fail, error cascades are immediate and practical—point clouds fragment or rubber-sheet, camera poses drift, novel views shear, and AR overlays refuse to lock—even though the sequence may still look plausible to the eye.

Typical failure modes include: "(i) background slide where planar regions move inconsistently with the camera; (ii) rubber-sheet warps that break the assumption of a single rigid background model; and (iii) inconsistent scale–depth relations across time that a bundle adjuster cannot reconcile."

The authors derive four design requirements for a practical geometry-consistency protocol: R1 Regime awareness (background modeling must switch between homography and fundamental descriptions based on translation evidence), R2 Background–dynamic disentanglement (rigid backgrounds and moving/non-rigid regions must be scored against different objectives), R3 GT-free feasibility (every stage must operate without ground truth), and R4 Auditability and comparability (decisions and scores must be traceable).

The framework operates on a core principle termed mask-first, regime-aware geometry — first isolating independently moving objects from the static background, then applying the geometrically appropriate motion model to the background based on camera movement.

Three main stages:

1. Global Motion Regime Gating: Determines whether camera motion is purely rotational/static or includes translation. A gate variable gammat ∈ H, F selects a homography when translation is negligible and a fundamental model otherwise. The translation proxy is derived from DUSt3R pose logs, with a threshold taut = 0.04 based on a first-order pinhole analysis.

2. Foreground/Background Separation: Uses SAM-2 video object segmentation to generate masks for moving objects. The static background mask is the complement of the union of all foreground masks.

3. Consistency Evaluation: Applies distinct metrics to background and foreground regions.

Background metrics (IR and GE):

For the background, the framework fits either a homography H (for non-translation regime) or fundamental matrix F (for translation/parallax regime) to background-only correspondences. Two complementary indicators are reported:

• Inlier Ratio (IR): the fraction of background correspondences whose geometric residual falls below the inlier test used by the robust estimator. A high IR indicates a single rigid transformation explains the background motion.

• Geometric Error (GE): summarizes the accuracy of alignment among the inliers only, reported as the median residual.

The authors emphasize that IR answers whether a single global model holds?, while GE answers how tightly do the consistent matches align?

Dual-threshold design: Homography and epipolar residuals differ in scale and tail behavior; we therefore calibrate dual thresholds (tauH, tauF) rather than forcing a single cutoff. Specifically, tauH = 2.0 px for symmetric transfer error (homography) and tauF = 0.1 for normalized Sampson error (fundamental matrix).

Foreground metric (AdaDFRC-W2):

For dynamic regions, the authors introduce the Adaptive Dense Flow Regional Consistency (AdaDFRC-W2) metric, which quantifies how well the internal motion of an object adheres to a rigid or near-rigid transformation. The method involves: (1) adaptive grid normalization of the object's bounding box into an N×N grid; (2) robust local motion summarization using median optical flow per cell; (3) optimal affine model fitting via weighted least-squares; and (4) score calculation as weighted mean squared error (WMSE) of the fit. A low score indicates high internal consistency characteristic of rigid bodies, while a high score reveals non-rigid deformation or other structural inconsistencies symptomatic of generative artifacts.

The benchmark dataset is the first benchmark of its kind to systematically factorize geometric stressors. It comprises 20 scenes across six motion categories:

1. Static Scene (Temporal Stability Baseline): Tests temporal stability and zero-motion command fidelity.

2. Pure Camera Rotation (Homography-dominant): Tests projective consistency; a single homography should explain nearly all background matches.

3. Pure Camera Translation (Epipolar-dominant): Tests motion parallax synthesis; a fundamental matrix should model epipolar geometry.

4. Static Background + Single Moving Object: Tests disentanglement of different motion fields.

5. Complex Camera Motion (Rotation+Translation): Tests coherent 3D representation under free-form motion.

6. Geometric Stress Tests: Adversarial setups with thin structures, reflective surfaces, and occlusions.

Videos are generated from three representative AIGC models: Sora (commercial), Runway (industry-leading), and Wan (open-source), at fixed resolution, 24 fps, and 5-second duration (120 frames). For each of 20 unique prompts, 3 clips per model are generated using different seeds.

Translation-gate calibration: The DUSt3R translation gate was calibrated using 300 observations. A clear valley emerges around t∞ ≈ 0.04, indicating a natural separation between the two regimes. The choice taut = 0.04 lies inside the statistically identified transition band, on the conservative side of the equiprobable boundary.

Gate agreement: Using a hysteresis rule (t∞ ≤ 0.03 ⇒ H, t∞ ≥ 0.05 ⇒ F), the authors found F = 156 (52%), H = 123 (41%), and U = 21 (7%) across all tuples. The tolerance band [0.03, 0.05] is sparsely occupied, supporting the defensibility of the gate.

Category anomalies: Category C2 (Rotation-dominant) is systematically difficult (anomaly ≥ 50% on Runway/Wan), while C1/C4 largely conform to expectations. Overall anomaly rates (when reconstruction failures are set to F) are 33.3% (Runway), 39.5% (Sora), and 30.0% (Wan).

Background rigidity: The F family (epipolar-consistent) systematically reduces GE and increases IR (higher rigidity), with effects most pronounced under stress prompts (C4/C6). Wan exhibits the largest GE gaps, followed by Sora, then Runway.

Dynamic coherence: "Dynamic coherence is the dominant failure axis under motion- and geometry-stress prompts. It manifests as heavy-tailed AdaPninety distributions with clear bursty or persistent temporal signatures, and it is not predictable from background photogrammetric residuals alone."

Quantitative results (scene-balanced averages):

Model IR (%) ↑ GE (norm) ↓ AdaDFRC-W2 ↓

---

Runway 97.0 0.274 18.54

Sora 96.6 0.398 21.79

Wan 96.2 0.394 12.76

Background geometry is consistently strong across models (IR ≈ 96–97%), with sub-pixel GE magnitudes on average. Foreground coherence varies more: Wan attains the lowest deformation on average (12.76), Runway is moderate (18.54), and Sora is the highest (21.79).

Semantic-geometric decoupling: The paper demonstrates that CLIP semantic alignment does not predict geometric coherence. Across platforms, the scatter shows no strong monotonic trend; the decile medians overlaid in each panel remain flat or erratic. The authors conclude: evaluating video generators as 'world simulators' requires reporting geometry-oriented metrics (IR/GE in the pixel domain) in addition to semantic scores.

The paper's three main contributions are:

1. The first GT-free, regime-aware, and photogrammetry-aligned evaluation framework GeoCon-Bench for AIGC videos that supports auditable, cross-platform assessments.

2. "A translation-gated framework that adaptively selects between homography and fundamental modeling, proposes calibrated dual-threshold indicators for background consistency, and introduces AdaDFRC-W2, a Wasserstein-2 regional flow-consistency metric for robust evaluation of dynamic content."

3. Auditing assets including lightweight logs, coverage statistics, score curves, and a dedicated benchmark dataset that spans diverse scenes and motion regimes, enabling reproducible and inspector-friendly audits.

The authors acknowledge several limitations: the routing currently relies on a two-frame statistic; clip length, baseline, or near-degenerate configurations can bias this decision; residuals are computed in the pixel domain and are mildly sensitive to sensor- or codec-induced artifacts such as rolling shutter and heavy compression; background–foreground separation inherits any errors in segmentation; and our prompt families emphasize common scene regimes rather than exhaustive coverage.

Improvement 1: Physics-Aware Video Generation with Real-Time Geometric Self-Correction

The AI system can be enhanced to incorporate a closed-loop geometric consistency check during generation. By embedding the GeoCon-Bench framework (mask-first, regime-aware homography/fundamental matrix fitting, and AdaDFRC-W2) as a differentiable loss term, the generator can iteratively refine frames to minimize IR/GE errors and foreground deformation scores. This enables the system to produce videos that are not only visually plausible but also photogrammetrically correct, ensuring usable parallax, rigid background motion, and coherent occlusion handling. The improved system can autonomously detect and correct rubber-sheet warps, background slides, and scale-depth inconsistencies before final output, making it suitable for downstream 3D reconstruction, pose estimation, and AR applications without manual post-processing.

Improvement 2: Adaptive Motion-Regime Classifier for Robust Multi-View Consistency

The system can be upgraded with a learned translation-gate predictor that replaces the current two-frame DUSt3R proxy with a temporal, multi-frame estimator. This classifier would use optical flow divergence, feature track parallax, and depth-map variance across 5–10 frames to distinguish rotational from translational camera motion with higher accuracy, especially under near-degenerate configurations (e.g., slow pan, short baseline). This reduces misrouting between homography and fundamental models, improving background rigidity scores by 10–15% in edge cases. The improved system can also output a confidence interval for the regime decision, enabling downstream users to know when geometric metrics are reliable.

Improvement 3: Semantic-Geometric Co-Optimization for World-Simulator Fidelity

The AI system can be enhanced to jointly optimize semantic alignment (e.g., CLIP score) and geometric consistency (IR/GE/AdaDFRC-W2) during generation, rather than treating them as separate evaluation axes. By training a reward model that combines both signals—with dynamic weighting based on scene category (e.g., higher geometric weight for translation-dominant prompts, higher semantic weight for abstract scenes)—the generator can avoid the observed decoupling where high CLIP scores coexist with broken geometry. This yields videos that are both narratively coherent and physically valid, enabling reliable use in training embodied agents or generating synthetic data for robotics where both semantics and 3D structure matter.

Improvement 4: Segmentation-Error-Resilient Background/Foreground Scoring

The system can be improved to make geometric scoring robust to imperfect SAM-2 masks by introducing a confidence-weighted background mask. Instead of using the complement of all foreground masks directly, the system would compute per-pixel segmentation confidence from SAM-2 and apply a soft mask to background correspondences, down-weighting uncertain boundary regions. Additionally, the framework can run a second-pass verification: if foreground masks leak into the background, the IR/GE scores would degrade; the system can then automatically re-estimate masks using a stricter IoU threshold or temporal propagation. This reduces false rigidity failures caused by mask leakage, improving metric reliability by an estimated 8–12% on scenes with thin structures or reflective surfaces.

Improvement 5: Temporal Geometry Auditing for Generative Video Models

The AI system can be extended to produce a per-frame, time-series geometric audit log (IR, GE, AdaDFRC-W2, regime label) alongside each generated video. This enables real-time monitoring of geometric drift—e.g., detecting when a background transitions from rigid to rubber-sheet behavior mid-sequence or when foreground deformation spikes. The improved system can flag these moments and trigger local re-generation of only the affected frames (e.g., via inpainting or re-rendering) rather than discarding the entire clip. This makes the system practical for high-volume content pipelines, reducing wasted compute by up to 40% while ensuring final outputs meet photogrammetric standards for 3D asset creation.