RIGOR: Rig-Informed Geometry for Omnidirectional Reconstruction

arXiv:2609.13504 · cs.CV · Submitted 2026-09-11 · Read on arXiv

Listen

Radio episode about this paper

Transcript

Introduction to the show: ident: AI Radio. Generated commentary on the latest Artificial Intelligence papers.

Tom: I'm Tom, and with me are Jane, Lu, senior AI researcher at Tsinghua, Meng, lead engineer at a mysterious AI startup and Lalam, the in-house Large Language Model.

Jane: Today's paper: "RIGOR: Rig-Informed Geometry for Omnidirectional Reconstruction".

Tom: RIGOR introduces a large-scale reconstruction pipeline for gravity-aligned omnidirectional videos that retains a frozen feed-forward perspective backbone and exploits each panorama as a four-view virtual rig.

Jane: First, who's behind it and why it matters.

Title and authors: Tom: So, Jane, can you explain the central mechanism of RIGOR in a way that really captures how it works step-by-step? I want to understand the process from input to output.

Jane: Certainly. The paper describes assuming gravity-aligned panoramas and then projecting each panorama into four perspective views, which sets up the virtual rig where optical centers coincide and rotations are known, but they don't provide metric scale or stereo parallax.

Lu: That initial setup is crucial because it establishes that geometric foundation; without that known relationship between the views, there's no way to enforce consistency later on.

Meng: So, once we have those four views set up, how does the sequential reconstruction part actually use this rig structure? Does it just plug in camera poses?

Jane: The sequential reconstruction uses Depth Anything three to predict camera poses and depths for blocks of perspective images, but critically, they check if these predictions align with the shared center model under the known virtual rig geometry <ref:2609.13504#pg0>.

Tom: That sounds like a direct test of consistency; what happens when that test fails? Does the system just crash, or does it try to fix it?

Jane: If an inconsistency is detected—if a four-view set has a center residual exceeding bounds or rotation residual exceeding five degrees—the method triggers a repair mechanism to select the best valid subset and fix those local predictions.

Lu: That dynamic selection of the best subset for local repair is where the intelligence really shines; it’s not a rigid rule, but an intelligent decision based on maintaining geometric validity.

Meng: So, they are essentially building in a self-correction loop for pose estimation during sequential reconstruction rather than just relying on the feed-forward model to be perfect.

Lalam: This internal repair loop means the system becomes much more resilient against the inevitable noise that creeps into long sequences, which is a key aspect of robust cultural data processing.

Tom: That resilience is what makes it viable for real-world use; so, once those local repairs are done, how does the paper handle finding revisits in the panorama?

Jane: For loop closure, they use a frozen SALAD encoder to get normalized descriptors for every view and then score all possible cyclic assignments using Equation (four) to find the best similarity score <ref:2609.13504#pg1>.

Lu: The cyclic scoring is what’s special here; it forces them to consider how views relate to each other in a continuous cycle, which is much more meaningful for panoramic data than just pairwise matching.

Meng: That mechanism sounds like it’s designed specifically to capture the holistic relationship between images that makes sense in a panoramic context, not just isolated feature matching.

Lalam: It shows an understanding that loop closure isn't just about finding two visually similar pictures; it has to respect the entire spatial layout of the panorama.

Tom: And once they have these strong candidate pairs, what’s the final step before they commit to them? Are they just taking those candidates blindly into the global optimization?

Jane: No, those candidates undergo a joint DA3 prediction and context gathering to form a joint frame J, which then attaches these similarity transforms Ht and Hs to that frame.

Lu: That intermediate step of creating a verified candidate frame before feeding it into the graph is vital; it’s the verification stage that prevents bad loop closures from corrupting the global map immediately.

Meng: So, they separate detection from verification clearly, which is good engineering practice; first find potential matches, then prove they are good enough to be trusted for optimization.

The paper's summary: Tom: So, Jane, when you look at the comparison they make, what are the main performance metrics that show where RIGOR actually stands ahead? I want some concrete numbers if possible.

Jane: The paper shows that on challenging construction site sequences from the Hilti-Trimble-Oxford dataset, RIGOR achieves better trajectory accuracy compared to baseline methods like DA3-Legacy. Specifically, they reduce the median Absolute Trajectory Error from two point seven zero four meters down to two point three one two meters and the Relative Pose Error over a 10s horizon from one point seven nine seven meters to one point seven one two meters, which is a clear win in accuracy metrics.

Lu: The geometric evaluation also shows strong results, with RIGOR achieving the lowest ROI CD of zero point one three two meters and the highest F@twenty-five score of zero point eight seven three, which are competitive figures for this type of reconstruction task.

Meng: Those numbers speak to real-world utility; when you’re building something that needs tight spatial accuracy, those improvements translate directly into a more reliable system for navigation or mapping tasks in the field.

Lalam: Achieving the lowest ROI CD means the reconstructed geometry is incredibly precise locally, which speaks volumes about how well this method handles small-scale spatial details.

Tom: And what about the ablation studies? Did they show that those specific features they added, like rig repair, actually make a difference in terms of performance?

Jane: They did; the ablation studies confirmed that removing the rig repair feature caused the ROI CD to jump from zero point one three two meters up to zero point two three zero meters and dropped the F@twenty-five score down to zero point eight seven three down to just zero point eight seven three, which shows how much that specific component contributed positively to the final performance numbers.

Lu: That result is quite telling; it confirms that the geometric constraint enforcement isn't just a nice addition; it’s a critical component for achieving the stated accuracy levels in this system.

Meng: It validates that focusing on reinforcing the structure through these constraints pays off, showing us which parts of the reconstruction pipeline are most impactful for improving overall performance.

Lalam: This detailed analysis confirms that when we engineer AI systems, we can pinpoint exactly where adding structural constraints yields the biggest gains in reliability and accuracy for complex tasks.

The paper's improvements: Tom: So, to wrap up the main points of this paper on "RIGOR: Rig-Informed Geometry for Omnidirectional Reconstruction," we have a large-scale pipeline that combines a frozen feed-forward backbone with explicit geometric constraints from the virtual rig, coupled with loop retrieval through cyclic four-view consensus and global optimization for trajectory refinement.

Jane: Essentially, it means they are using the known geometry of those four views to actively detect and repair local errors in predictions before they propagate globally.

Lu: This approach establishes a very strong foundation where the known physical relationships between the views dictate what is permissible during reconstruction, moving beyond simple feed-forward guesswork.

Meng: From my side, this means we can build systems that are significantly more resilient to the kinds of unpredictable visual artifacts common in complex environments because they have these built-in checks.

Lalam: The real implication for us is that we can expect AI to handle long, complex visual streams with a much higher level of spatial consistency, which is a massive step forward for reliable cultural understanding.

Tom: That’s the big picture, folks; RIGOR provides a systematic and rigorous way to tackle the challenges of reconstructing three dee data from omnidirectional videos over extended sequences <ref:2609.13504#pg0>. We’re really looking forward to seeing how this structure informs our next steps in development.

Jane: It's been fascinating dissecting how they used geometric constraints to fix local problems, which is a really practical technique for AI deployment.

Lu: I think the concept of treating the panorama as a virtual rig is an exciting conceptual leap that allows for such tight control over the reconstruction process.

Meng: We’ll take these findings and focus on how we can implement those constraint-based checks efficiently in production environments.

Lalam: Ultimately, this paper shows that when we design AI systems with structural awareness, they become inherently more trustworthy for long-term data processing tasks.

Conclusion: Tom: So, we've seen how RIGOR uses that virtual rig structure to enforce geometric consistency across long panoramic sequences, and now we're wrapping up this deep dive into "RIGOR: Rig-Informed Geometry for Omnidirectional Reconstruction."

Jane: It really shows how by imposing those known relationships between the four views, the AI can proactively fix local errors instead of letting them pile up into a massive global mess.

Lu: I think the creative potential here is huge; envision using these constraints to build truly robust spatial anchors for any kind of large-scale three dee world modeling.

Meng: From an engineering standpoint, that ability to self-correct local predictions before they hit the global graph means we might actually need less heavy post-processing in our reconstruction pipelines.

Lalam: I feel like this work has implications for how we understand and recreate complex spatial information across any medium, which is a core part of how AI can improve our collective cultural understanding of the physical world.

Tom: Exactly, Lalam; the accuracy improvements they showed on those construction site sequences really underscore how much better our navigation and mapping tools are becoming.

Jane: It’s impressive that they managed to maintain that level of precision while dealing with the inherent noise in long-trajectory data.

Lu: And the loop closure mechanism, using cyclic alignment for panoramic descriptors, opens up so many avenues for finding revisits in three hundred sixty-degree imagery that we hadn't even considered before.

Meng: But I wonder about scalability; how does this approach hold up when we move from these controlled datasets to completely unstructured or dynamic real-world video streams where the rig might be harder to define initially?

Lalam: That’s a fair question, Meng; the adaptability of that virtual rig concept is what makes it so compelling for future applications in understanding any complex visual scene.

Tom: It’s definitely something we need to keep thinking about as we look at other reconstruction challenges, but for now, RIGOR gives us a fantastic blueprint on how to enforce structural integrity in AI modeling.

Jane: Absolutely; it’s a great reminder that geometric constraints can be incredibly powerful tools for stabilizing complex AI predictions.

Lu: So, if the next paper tackles something like time-series forecasting challenges with label alignment, we might find some interesting parallels in how they use transformations to guide learning objectives.

Meng: That connection between structural guidance in three dee reconstruction and constraint-based learning objectives for 1D data is something worth exploring down the line <ref:2609.13504#pg0>.

Lalam: I’m really looking forward to seeing how these ideas translate into improving the way AI interprets complex, continuous data streams moving forward.

ETH Zürich

cs.CV

Submitted: 2026-09-11

Updated: 2026-10-06

Code: https://github.com/TangentH/RIGOR

Importance score: 79/100

The gist: RIGOR introduces a large-scale reconstruction pipeline for gravity-aligned omnidirectional videos that retains a frozen feed-forward perspective backbone and exploits each panorama as a four-view

Key concepts

Virtual Rig
Each panorama is projected into four perspective images, forming a virtual rig. These views have known relative rotations but lack metric scale or stereo parallax. This structure allows the system to enforce geometric constraints based on the fixed relationship between these four views during reconstruction.
Rig Correction
When sequential predictions become locally inconsistent (e.g., pose errors exceed bounds), RIGOR triggers a repair mechanism. It selects a valid subset of views to correct the prediction and back-projects 3D points using the fitted rig pose, ensuring local geometric consistency before proceeding.
Four-View Loop Closure
Loop closure is achieved by cyclically aligning descriptors from all four virtual views to find the most similar matches. Only candidates meeting strict similarity thresholds are kept. These verified loop measurements are then used in a global optimization process to correct accumulated drift across the entire trajectory.
Frozen Feed-Forward Backbone
RIGOR retains a pre-trained feed-forward model for predicting camera centers and rotations. This backbone is constrained by the known virtual rig geometry, ensuring that predictions stay close to a shared center model, which stabilizes the sequential reconstruction process.

Terminology

Summary

RIGOR introduces a large-scale reconstruction pipeline for gravity-aligned omnidirectional videos that retains a frozen feed-forward perspective backbone and exploits each panorama as a four-view virtual rig. This method addresses inconsistencies in long trajectories by using the known co-location and relative orientations of perspective views as explicit geometric constraints to detect and repair locally inconsistent predictions, retrieve loop closures through cyclic four-view consensus, and geometrically verify candidate revisits before global optimization.

The gist

RIGOR is a virtual-rig-based method that exploits the known co-location and relative orientations of perspective views as constraints for detecting and repairing inconsistent pose and geometry predictions; we provide a rig-aware capture representation for loop retrieval that cyclically aligns descriptors from all virtual views, followed by a joint geometric verification of the retrieved candidates; our method similarly targets temporally ordered panoramic observations, but combines a frozen perspective feed-forward model with four-view geometric consistency, rig-aware loop retrieval and verification, and global loop-based correction.

Preprocessing and Virtual Rig Setup

The method begins by assuming the input video is gravity-aligned. To suppress dynamic foreground geometry, Grounded Segment Anything is used to mask people, followed by removing the static footprint of the capture device using a separately calibrated mask projected from the equirectangular image into the four perspective views. The core of RIGOR involves projecting each panorama into four perspective images, denoted as Equation (1), where each view has a known yaw rotation relative to the panorama reference orientation. These four views form a virtual rig, characterized by having their optical centers coincide and their relative rotations are known, but they provide neither stereo parallax nor metric scale.

Sequential Reconstruction with Rig Correction

The sequential reconstruction process utilizes Depth Anything 3 (DA3) to predict camera poses, effective intrinsics, and per-pixel depths for consecutive blocks of perspective images. Adjacent blocks share images, and confidence-weighted pointmap alignment estimates a relative similarity transform Sk,k+1 ∈ Sim(3) and places their predictions in one reconstruction frame. For the frozen backbone prediction of camera centers and rotations (Equation 2), RIGOR assumes that these predictions should be close to a shared center model, Ct, Rbv t ≈ UtQv under the known virtual rig geometry. When local inconsistencies are detected, a repair mechanism is triggered: if a four-view set is invalid (center residual exceeds bounds or rotation residual exceeds 5°), the method selects the best valid subset and uses it to repair locally inconsistent predictions. The repaired 3D point corresponding to pixel p in view v∗ is then back-projected using the fitted rig pose (Equation 3).

Four-View Loop Closure and Global Optimization

Loop closure is achieved through a cyclic retrieval mechanism based on descriptors. A frozen SALAD encoder produces a normalized descriptor for every perspective image. For two panoramas, the method jointly scores the four possible cyclic assignments using Equation (4) to find the assignment with the maximum similarity score, denoted as mq(t, s). Candidate pairs are retained only if they satisfy specific thresholds: "mq∗ (t, s) > 0.65 and all four similarities on the selected diagonal are at least 0.50." Retained candidates undergo a joint DA3 prediction and context gathering to form a joint frame J. Similarity transforms Ht and Hs attach J to the corresponding stored blocks (Equation 5). Finally, these verified loop measurements are incorporated into a Sim(3) pose graph G = (V, E), where edges contain sequential and verified loop edges. The optimization minimizes the error of the transformations: G⋆k = arg min X (i,j)∈E rij (Gi, Gj)2, allowing the graph to correct accumulated rotation, translation, and scale drift along the trajectory.

Experimental Validation and Performance

The method is evaluated on challenging construction site sequences from the Hilti-Trimble-Oxford dataset. Trajectory evaluation metrics include Absolute Trajectory Error (ATE) and a position-only translational Relative Pose Error over a 10 s horizon, with all reported trajectories passing the 99% association coverage requirement. Geometry evaluation uses symmetric Chamfer-L1 distance (CD) and the F-score at a distance threshold τ = 0.25 m. Quantitative results show that RIGOR achieves the best trajectory accuracy, reducing median ATE from 2.704 m to 2.312 m and RPE-10s from 1.797 m to 1.712 m relative to DA3-Legacy, while obtaining the lowest ROI CD (0.132 m) and highest F@25 (0.873). Ablation studies confirm that rig repair is beneficial, showing that removing it degrades ROI CD from 0.132 m to 0.230 m and F@25 from 0.

Improvements for AI systems

Based on the provided research paper, RIGOR: Rig-Informed Geometry for Omnidirectional Reconstruction, here are specific, actionable improvements that can be implemented in AI systems using this framework:

)Specific Improvements and Capabilities of the RIGOR System:

  1. A. Robust Long-Trajectory Scene Understanding in Challenging Environments:

The system can reliably reconstruct dense 3D geometry and estimate camera trajectories over long sequences (e.g., hours of continuous video), even in environments characterized by repetitive structures, weak visual textures, or high dynamic content (people/vehicles).

  1. B. Enhanced Global Geometric Consistency via Virtual Rig Constraints:

Instead of relying solely on local feed-forward predictions that drift over time, the system can enforce a rigid geometric structure across all captured views within a panorama (the virtual rig). This prevents accumulated rotation, translation, and scale drift from destroying the global map integrity.

  1. C. Automated Detection and Repair of Local Inconsistencies:

The AI system can actively detect when the feed-forward model makes an error (local inconsistency) by checking it against the known geometric constraints of the four-view rig. It can then automatically repair these local errors in both predicted camera poses and reconstructed 3D point clouds before they propagate to the global optimization stage.

  1. D. Robust Loop Closure Detection for Panoramic Imagery:

The system can efficiently find revisits across long sequences by exploiting the cyclic nature of panoramic imagery using learned descriptors (SALAD encoder). It doesn't just look for visual similarity; it cyclically aligns descriptors from all four virtual views to identify true loop closures, even when the viewpoint changes significantly.

  1. E. Verified Loop Closure Candidates:

Before a potential loop closure is accepted into the global reconstruction graph, the system performs a joint geometric verification using DA3 predictions on the candidate views. This two-stage verification (descriptor matching followed by geometric check) drastically reduces false positives, ensuring that only geometrically sound revisits are used to correct the trajectory.

  1. F. Improved Trajectory Accuracy and Precision:

By incorporating these consistency mechanisms, the system achieves significantly lower Absolute Trajectory Error (ATE) and Relative Pose Error over long horizons compared to baseline feed-forward methods, leading to much more accurate navigation and localization in real-world applications like autonomous driving or robotic exploration.

  1. G. Scalability without Panorama-Specific Training:

The system can be applied to any gravity-aligned panoramic video stream without requiring additional fine-tuning of the underlying deep learning reconstruction backbone, making it highly versatile for new environments where specific training data is unavailable.

  1. H. Adaptive Reconstruction Parameters (Optional):

The framework allows for controlled experimentation with input parameters like the perspective Field of View (FoV) and sequential block size/overlap, enabling researchers to tune the system's robustness based on the specific characteristics of a target environment or camera setup (e.g., choosing 95° FoV for optimal consistency).

Related papers