Robot Learning on Discrete Surfaces: Theory and Applications

All objects are enclosed within surfaces, yet most robot learning and motion generation frameworks treat surfaces as constraints ignoring their intrinsic geometry, creating a gap for polyhedral meshes whose discrete geometric structure remains unexploited. This paper proposes a unified discrete Riemannian framework that enables robot learning directly on polyhedral surface meshes by defining geometrically consistent approximations of key Riemannian operators across faces, edges, and vertices.

The gist

This work develops a discrete Riemannian framework that extends differential geometry to polyhedral surface meshes by constructing geometrically consistent approximations of logarithmic and exponential maps, parallel transport, geodesic distance, and ambient-space projections that remain well-defined across mesh elements.

How it works: Discrete Differential Geometry Foundation

The framework is built upon discrete differential geometry to define operators that are well-defined across the mesh structure. Key components include:

1. Tangent cone: Defined as the limit of sequences of points, identifying three cases based on whether the point lies in a face interior, on an edge, or at a vertex.

2. Logarithmic map: Maps a point on the mesh into a vector in the tangent cone, where the vector's magnitude equals the geodesic distance between points. This is computed using enhanced versions of continuous Dijkstra algorithms to find discrete geodesics.

3. Exponential map: The inverse operation that maps a vector from the tangent space back to a point on the mesh, utilizing an iterative procedure based on straightest geodesic principles and handling edge singularities through angle computations (total angle).

4. Parallel transport: An isometric transformation that moves a vector from one tangent cone to another, encoded as a rotation matrix derived from the geodesic path and surface normal vectors.

How it works: Extension to Learning Paradigms

The unified framework is instantiated in three learning paradigms, each utilizing the discrete operators for improved generalization and stability:

1. Dynamic Movement Primitives (DMPs): The approach uses an improved exponential-map computation and a fixed-tangent-cone forcing-term encoding with parallel transport, which yields better cross-surface generalisation than prior mesh-based methods by replacing local parameterisations.

2. Gaussian Process (GP): This involves a geodesic-based kernel with practical admissibility control, enabling regression at arbitrary mesh locations, including face interiors, without smoothness assumptions or density requirements, overcoming the limitations of Laplace–Beltrami-based kernels.

3. Riemannian Flow Matching (RFM): The framework uses mesh-native operators and geodesic premetrics instead of spectral premetrics to improve generative quality over spectral baselines while reducing training time.

Key Contributions

The main contributions include:

(i) A discrete Riemannian operator framework on polyhedral meshes—including tangent cones, straightest-geodesic exponential maps, and ambient-space projections—that explicitly handles edge and vertex singularities left unaddressed by existing mesh-based methods.

(ii) A reformulation of geometry-aware DMPs on meshes that encodes the forcing term in a fixed tangent cone and reconstructs it via parallel transport, eliminating the limit-cycle drift it induces on curved surfaces.

(iii) A geodesic-distance-based GP formulation on meshes that enables kernel regression at arbitrary surface locations—including face interiors—without smoothness or density requirements, overcoming vertex-restriction and resolution-sensitivity of Laplace–Beltrami-based kernels.

(iv) An extension of RFM to discrete polyhedral surfaces using geodesic premetrics, improving generative quality over spectral baselines while reducing training time.

Validation and Applications

The framework was validated in simulation against state-of-the-art methods and demonstrated on two real-robot scenarios: generalising user-drawn trajectories across different surfaces and planning polishing motions on RGBD-reconstructed surfaces. Experiments showed that the proposed DMPs quickly converge to the limit cycle, whereas MeshDMP exhibits an undesired rotational behavior around the fixed motions centre because it uses a local parameterisation for the forcing term, which is overcome by encoding it in a fixed tangent cone and using parallel transport. For GPs, the geodesic-based kernel performed better than Laplace-Beltrami kernels across various length scales below a critical value of κth. For RFM, the mesh-native operators improved generative quality compared to spectral baselines and reduced training time by approximately half. The methodology was implemented in C++ relying on the CGAL library for polyhedral mesh operations.

Limitations and Future Directions

Limitations identified include:

1. The geodesic pre-computation remains a bottleneck, as it relies on algorithms with complexity O(N2 log N), becoming significant when the surface exceeds 10,000 faces.

2. The geodesic-based kernel cannot be guaranteed to be positive-definite when the lengthscale exceeds some critical value κth, risking inconsistent GPs beyond this threshold.

Here are specific improvements to AI systems derived from this research, along with the capabilities these improved systems would possess:

1. The proposed framework enables robot learning directly on polyhedral surface meshes by defining consistent discrete Riemannian operators (logarithmic/exponential maps, parallel transport) that handle edge and vertex singularities.

2. This allows for the instantiation of three geometry-aware learning paradigms:

3. Improved Dynamic Movement Primitives (DMPs) with fixed-tangent-cone forcing terms encoded via parallel transport.

4. Geodesic-based Gaussian Processes (GPs) using a geodesic distance kernel, enabling regression at arbitrary mesh locations (including face interiors) without smoothness assumptions or vertex restrictions.

5. Riemannian Flow Matching (RFM) using mesh-native operators and geodesic premetrics, which improves generative quality over spectral baselines while reducing training time.

These improvements enable the following specific AI capabilities:

1. Improved motion planning for manipulation tasks on complex, real-world objects (e.g., grasping, polishing). The robot can learn and execute trajectories that generalize robustly across surfaces with varying curvature and topology (e.g., transitioning from a flat plane demonstration to a highly irregular object surface like the Stanford Bunny).

2. Robust online surface reconstruction and real-time motion generation for tasks like automated industrial finishing or polishing. The system can ingest RGB-D data, reconstruct a watertight manifold mesh, and generate physically consistent polishing motions directly on that learned 3D geometry in seconds (e.g., achieving polishing behavior in <5 seconds).

3. High-fidelity regression of sensor data on arbitrary surface locations without the need for dense mesh upsampling or interpolation errors inherent to Laplacian-Beltrami methods. This allows the AI to accurately predict function values (e.g., material properties, deformation fields) at any point on a learned surface, even in regions where the mesh is coarse or non-smooth.

4. More efficient and stable generative modeling for creating new 3D shapes or textures conforming to complex geometries. The RFM approach trains vector fields directly on the mesh structure, leading to faster training and higher generative quality than spectral baselines.

All the objects composing our world are enclosed within surfaces. Yet, most robot learning and motion generation frameworks treat surfaces as constraints ignoring their intrinsic geometry. This gap is acute for polyhedral meshes--the standard output of CAD and 3D reconstruction--whose discrete geometric structure remains unexploited. In this paper, we propose a unified discrete Riemannian framework that enables robot learning directly on polyhedral surface meshes. Using discrete differential geometry, we define logarithmic and exponential maps, parallel transport, and ambient-space projections that remain well-defined across faces, edges, and vertices. We instantiate the framework in three learning paradigms: (i) Dynamic Movement Primitives (DMPs), an improved exponential-map computation and a fixed-tangent-cone forcing-term encoding with parallel transport yield better cross-surface generalisation and stability over prior mesh-based approaches. (ii) Gaussian Process (GP), a geodesic-based kernel with practical admissibility control, enables regression at arbitrary mesh locations without smoothness assumptions. (iii) Riemannian Flow Matching (RFM), mesh-native operators improve generative quality over spectral baselines while reducing training time. The framework is validated in simulation against state-of-the-art methods and demonstrated on two real-robot scenarios: generalising user-drawn trajectories across different surfaces and planning polishing motions on RGB-D-reconstructed surfaces.