SurGe: Improved Surface Geometry in Point Maps

Recent feedforward 3D reconstruction methods predict point maps and estimate global 3D geometry remarkably well, but these predictions still exhibit inaccurate local surface geometry, which is clearly visible qualitatively but only weakly reflected in common metrics. This paper introduces SurGe, a model that improves local point map and point map normal evaluations by incorporating a novel metric and architectural changes to better capture the orientation induced by neighboring 3D predictions.

How it works

The authors address the weakness of standard point map metrics, which only measure average point positioning error after global alignment, by introducing a point map normal metric that directly evaluates local surface orientation induced by neighboring point differences. This metric is quantified as MAEnormal, which compares the normals induced by neighboring point differences rather than just the magnitude of underlying residuals.

The proposed solution involves two complementary components:

1. A point gradient matching loss (Lpgm) that supervises depth-normalized 3D finite differences to enforce local consistency and suppress oscillations. This loss is adapted from log-depth gradient matching, making it scale-invariant for each neighboring pair by normalizing the comparison by the nearer endpoint depth.

2. A Neighborhood Attention Decoder (NAD) that replaces fixed convolutional kernels with blocks based on Neighborhood Attention (NA). This allows the decoder to selectively aggregate local evidence unlike convolutional decoders, without incurring the cost of full-resolution self-attention and without producing patch-aligned artifacts.

Key Contributions

The paper enumerates its main contributions as follows:

1. An evaluation metric based on point map normals that better reflects local surface quality than standard point map metrics (MAEnormal).

2. A scale-invariant point gradient matching loss (Lpgm), inspired by log-depth gradient matching [23], for supervising local surface structure in point maps.

3. A decoder based on Neighborhood Attention for dense point map prediction that improves thin structures and local surface geometry (NAD, Sec. 3.1).

Model Architecture Details

SurGe combines a DINOv2 [29]-initialized ViT [9] encoder with the Neighborhood Attention Decoder (NAD). The NAD consists of five stages that progressively upsample the features, with each stage built from nl NAD blocks. Each NAD block is a Transformer-style residual block featuring a Neighborhood Attention layer and a pointwise Feed-Forward Network (FFN). The model utilizes window-matched RoPE [40] on queries and keys to provide consistent representation of relative offsets inside local windows, and it omits the usual pre-attention and pre-FFN LayerNorm layers.

Training Objectives

The full dense-label objective function is defined as:

L = Lglob + Lloc,4 + Lloc,16 + Lloc,64 + 10Lpgm.

Where:

Lglob

is the global affine-invariant point map loss with ROE [50] alignment.

(Lloc,

are local patch losses with diameters of 1/4, 1/16, and 1/64 of the image diagonal.

The authors note that they adapt supervision to label quality: for synthetic labels, they use the full objective (including Lpgm), while for SfM labels, they omit Lloc,64 and Lpgm.

Experimental Results

SurGe achieves the best average rank for global point map AbsRelglob and consistently improves local point map and point map normal evaluations across eight zero-shot monocular geometry benchmarks. Specifically, SurGe gives the lowest AbsRelloc on every evaluated dataset, and MAEnormal shows superior performance in reflecting local surface orientation compared to state-of-the-art methods like MoGe-2 and InfiniDepth [63], which produce oscillatory or bending artifacts on thin structures. The results demonstrate that SurGe improves local surface quality while preserving state-of-the-art global point map accuracy.

Ablation Study Insights

Ablations confirm the effectiveness of both components:

Decoder Design:

The NAD performs better than DPT heads, ConvStack, and ViT decoders in local evaluations (AbsRelloc and MAEnormal), whereas the ViT decoder suffers from patch-level artifacts. The ConvStack-L baseline shows moderate improvement over standard ConvStack.

Surface Loss:

The proposed Lpgm loss is shown to noticeably outperform both point map normals (Lnormal) and log-depth gradient matching (Lgm) in most cases, which is attributed to the fact that the Lpgm signal remains in the same space as the global loss.

Conclusion

SurGe presents a monocular point map model designed to improve local surface geometry rather than only average point accuracy.

Here are the specific improvements that can be made to existing point map prediction systems by adopting SurGe, along with what these improved AI systems will be able to achieve:

---

1. The integration of the proposed Neighborhood Attention Decoder (NAD) into a feedforward 3D reconstruction pipeline.

2. The implementation of the scale-invariant Point Gradient Matching Loss (Lpgm) as a critical component of the training objective.

3. The adoption of the Point Map Normal Metric (MAEnormal) for explicit, fine-grained evaluation of local surface orientation, rather than relying solely on global point position errors (AbsRelglob).

These improvements will enable the following capabilities in AI systems:

1. Predicting and reconstructing thin structures with high fidelity, such as street signs, chair legs, and faucets.

2. Producing 3D models that exhibit significantly smoother surfaces and sharper geometric detail compared to current state-of-the-art methods (like MoGe or InfiniDepth), which often produce oscillatory or bending artifacts in local geometry.

3. Achieving state-of-the-art performance in both global 3D accuracy and local surface quality simultaneously, ensuring that small errors do not compromise the overall scene structure.

4. Developing more robust perception systems for autonomous driving and robotics, where accurate reconstruction of fine geometric details is crucial for collision avoidance and interaction with thin objects in complex environments.

5. Creating a new generation of geometry evaluation metrics (MAEnormal) that explicitly quantify local surface coherence, allowing researchers to make targeted improvements on specific geometric defects in future models.