Iso-Riemannian Optimization on Learned Data Manifolds

arXiv:2510.21033 · math.OC, cs.LG, math.DG · Submitted 2026-08-14 · Read on arXiv

Listen

Radio episode about this paper

Transcript

Introduction to the show: ident: AI Radio. Generated commentary on the latest Artificial Intelligence papers.

Tom: Next we'll be talking about the paper "Iso-Riemannian Optimization on Learned Data Manifolds".

Jane: The paper was written by Willem Diepeveen and Melanie Weber from University of California, Los Angeles and Harvard University.

Tom: Stay tuned as we take you through the paper and discuss its implications.

Title: Tom: Welcome back, everyone. Today we're looking at a paper that's been making waves in the optimization community, and it's got a mouthful of a title: "Iso-Riemannian Optimization on Learned Data Manifolds." Jane, what do we even mean when we say "learned data manifolds"?

Jane: Great question, Tom. So imagine your data isn't spread out evenly across all of space, but instead it lives on a kind of curved surface, like a sheet of paper that's been crumpled. That surface is the manifold. And in this paper, they don't just assume the surface is given—they learn it from the data itself using something called a pullback geometry, which is a fancy way of saying they find a transformation that flattens the data out.

Tom: And that's where the "iso" part comes in, right? Because once you flatten things, you want your optimization to respect that flattened geometry?

Jane: Exactly. The "iso" refers to the iso-connection, which is a way of measuring distances and moving along the manifold so that you're moving at constant speed in the original space. The authors—Willem Diepeveen from UCLA and Melanie Weber from Harvard—argue that this constant-speed property is what makes optimization on these learned manifolds actually work.

Lu: And I think that's the key insight. In classical Riemannian optimization, you assume the geometry is fixed and you use the Levi-Civita connection, which is the natural way to measure geodesics. But when the geometry is learned from data, that natural connection can distort things. The iso-connection fixes that distortion by ensuring that when you move along a geodesic, you're moving at a constant Euclidean speed.

Tom: So it's like taking a scenic route versus a highway? The scenic route might be more interesting, but the highway gets you there predictably?

Lu: That's a good way to put it. The iso-connection is the highway. It guarantees that the distance you travel in the ambient space is proportional to the parameter along the curve, which makes convergence analysis much cleaner.

Meng: But let me play devil's advocate here. Does this actually matter in practice? I mean, if you're running gradient descent on a manifold, you just need the exponential map, right?

Jane: That's the thing, Meng. The exponential map under the Levi-Civita connection can be expensive and unstable when the geometry is learned. But under the iso-connection, the exponential map has a closed-form expression in terms of the pullback geometry. That means you can compute it efficiently, even in high dimensions.

Tom: And that's what makes this paper exciting. They're not just proposing a theoretical framework—they're showing it works on real data, including MNIST digits. We'll get into the details of their experiments in a bit, but the headline is that this iso-Riemannian approach gives you convergence guarantees that classical Riemannian optimization can't provide when the geometry is learned.

Lu: And that's the gap they're filling. Previous work on optimization over learned manifolds either assumed the manifold was linear or used chart-based approaches that lose global structure. This paper says, let's use the geometry itself, but with a connection that's actually compatible with the Euclidean structure of the problem.

Jane: So the title really captures it. It's about doing optimization on manifolds that are learned from data, but doing it in a way that respects the isometry—the constant-speed property—of the geometry.

Tom: And that's the hook for the rest of our discussion. Next, we're going to look at what they actually propose in terms of new definitions for monotonicity and Lipschitz continuity, and why those are the right tools for this setting.

Summary: Tom: So we've set the stage with the title. Now let's talk about what this paper actually does. Jane, can you walk us through the core ideas?

Jane: Sure, Tom. The paper introduces three new notions: iso-monotonicity, iso-Lipschitz continuity, and iso-convexity. These are analogues of the classical concepts used in optimization, but they're defined using the iso-connection instead of the Levi-Civita connection.

Tom: And why do we need new definitions? Why can't we just use the old ones?

Jane: Because the old ones assume the geometry is fixed and well-behaved. But when the geometry is learned from data, a function that's convex in the Euclidean sense might not be geodesically convex in the Riemannian sense. The paper shows a simple example: the function f(x) = x2/two under a pullback metric with φ(x) = sinh(x+one) is not geodesically convex. That means Riemannian gradient descent can fail or converge very slowly.

Lu: And that's the fundamental problem. The critical points of a function don't depend on the geometry, but the region of convergence does. So even if you have a nice convex function, the learned geometry can make it look non-convex from the Riemannian perspective.

Meng: So they're essentially saying, let's change the connection so that the geometry doesn't fight against the Euclidean structure of the problem?

Jane: Exactly. And they prove that with these new definitions, you can get linear convergence for a descent algorithm they call iso-Riemannian descent. The algorithm is simple: you take a step along the iso-exponential map in the direction of the negative vector field, with a step size that depends on the iso-monotonicity and iso-Lipschitz constants.

Tom: And they have a theorem that guarantees convergence, right? Theorem three in the paper?

Jane: Yes. If the vector field is α-strongly iso-monotone and L-iso-Lipschitz, and you choose a step size r less than 2α/L2, then the sequence converges linearly to the unique zero of the vector field. That's a direct analogue of the classical result, but now it applies in this learned geometry setting.

Lu: And what's nice is that they show how to verify these conditions locally. You don't have to check every pair of points—you just need to check the iso-connection along geodesics. That's Theorem two and it makes the framework practical.

Meng: But hold on. How do you actually compute these iso-manifold mappings? The iso-exponential map, the iso-logarithmic map—those sound like they could be expensive.

Jane: That's the beauty of the pullback structure. For Euclidean pullback manifolds, the iso-manifold mappings have closed-form expressions in terms of the diffeomorphism φ. So you just apply φ, do Euclidean operations, and apply φ−1. That's cheap and scalable.

Tom: And that's why they can apply this to real data sets like MNIST. But before we get to the experiments, let me ask about the applications. What do they actually use this for?

Jane: Two main applications. First, they define the iso-barycentre, which is a generalization of the Riemannian barycentre. It's the point that minimizes the sum of squared iso-distances to the data points. And they show it has nice properties—for example, on a 1D manifold, the iso-barycentre field is one-strongly iso-monotone and one-iso-Lipschitz, which guarantees convergence of their algorithm.

Lu: And the second application is iso-convex optimization, where they show that the l2-projected gradient field of a Euclidean convex function can be iso-monotone under certain conditions. That's the key to solving inverse problems on learned manifolds.

Tom: So we've got the theory. Next, we need to talk about what they actually improved compared to existing methods, and that's where the experiments come in.

Improvements: Tom: Alright, we've covered the theory. Now let's talk about what this paper actually improves in practice. Jane, what did they show?

Jane: The biggest improvement is in clustering. They introduce something called iso-K-means, which is a variant of K-means that uses iso-barycentres instead of Euclidean or Riemannian barycentres. And the results are striking.

Tom: I remember the figures. On the synthetic river and spiral data sets, Euclidean K-means and even Riemannian K-means give weird clusters. But iso-K-means finds both correct clusters and meaningful centroids.

Jane: Exactly. The reason is that the pullback metric doesn't encode our intuition about Euclidean proximity. So when you use the Riemannian distance for clustering, you can get clusters that look wrong. The iso-distance corrects for that by measuring distances along constant-speed geodesics.

Lu: And that's a real improvement. The paper shows that the iso-barycentre field on a 1D manifold has the same strong monotonicity and Lipschitz constants as the classical Riemannian barycentre field. So you get the same convergence guarantees, but with a more meaningful notion of distance.

Meng: But what about the higher-dimensional case? The paper admits that the constants aren't necessarily close to one in practice. How do they handle that?

Jane: They use a line search. Algorithm one combines iso-Riemannian descent with a backtracking line search, initialized at the Riemannian barycentre. That ensures the starting point is within the strongly convex geodesic submanifold containing the data.

Tom: And it works on MNIST too, right? They compute the iso-barycentre of the digit data and it looks like an eight which is interpretable.

Jane: Yes, and the Euclidean mean is just a blurry mess. The Riemannian and iso-barycentres both give something that looks like an eight but the iso-barycentre is computed with the constant-speed property, which makes it more robust.

Lu: The other improvement is in solving inverse problems. They show that for a 1D manifold, if the function is strongly convex on the tangent bundle and the geometry doesn't introduce too much non-convexity, then the l2-projected gradient field is iso-monotone and iso-Lipschitz. That gives you convergence of their algorithm to the unique minimizer.

Meng: And they demonstrate this on denoising MNIST digits. They constrain the optimization to a twenty-dimensional subspace of the learned manifold and show that their algorithm converges to a nearly noiseless representation in just ten iterations.

Jane: That's the practical payoff. The theory tells you when and why the algorithm works, and the experiments show that it actually works in high-dimensional settings.

Tom: So the improvements are threefold: new theoretical tools, better clustering, and efficient inverse problem solving. But there are also limitations, right? The paper mentions numerical issues with the MNIST experiments.

Jane: Yes, they note that the combination of single precision in PyTorch and numerical approximations for the iso-manifold mappings can cause the algorithm to stall when seeking high accuracy. That's a practical limitation, but it's not a fundamental one.

Lu: And they also note that convergence of iso-K-means is still open. The Lloyd's algorithm adaptation doesn't inherit convergence guarantees from the classical setting. So there's room for future work.

Tom: That's a good segue. Let's bring in Lalam to talk about the bigger picture. Lalam, what do you see as the most impactful vision from this paper?

Conclusion: Tom: We've covered the theory, the applications, and the improvements. Let's wrap up with the big picture. Lalam, what's your take?

Lalam: I see this paper as a bridge between two worlds that have been disconnected for too long. On one side, you have the Riemannian optimization community, which has developed powerful tools for fixed geometries. On the other side, you have the machine learning community, which learns geometries from data but doesn't have the optimization tools to use them effectively. This paper builds that bridge.

Jane: And the bridge is the iso-connection. It's a simple idea—ensure constant-speed geodesics—but it has profound consequences for optimization.

Lalam: Exactly. And the cultural impact is significant. Think about applications like medical imaging, where you want to solve inverse problems on manifolds learned from patient data. Or recommendation systems, where the geometry of user preferences is learned from interaction data. This framework gives you provable convergence guarantees in those settings.

Meng: But what about the practical challenges? The numerical issues they mention—are those going to be a barrier?

Lalam: They're a barrier for high-precision applications, but not for the core methodology. As the paper notes, higher-order numerical approximations and better precision handling can mitigate these issues. And the fact that the algorithm converges in ten iterations on MNIST denoising shows that the approach is already practical.

Tom: So let me try to summarize. The paper "Iso-Riemannian Optimization on Learned Data Manifolds" introduces new notions of convexity, monotonicity, and Lipschitz continuity based on the iso-connection. It proves convergence of iso-Riemannian descent, demonstrates applications in barycentre computation and clustering, and shows how to solve inverse problems on learned manifolds.

Jane: And the key takeaway is that when you learn a geometry from data, you need to be careful about which connection you use for optimization. The iso-connection reconciles the learned geometry with Euclidean convexity, making optimization both theoretically sound and practically efficient.

Lu: I'd add that this opens up a whole research agenda. The paper mentions future work on retractions, higher-order methods, and non-smooth optimization. There's a lot of room to build on these foundations.

Meng: And from an engineering standpoint, the fact that the manifold mappings have closed-form expressions for pullback geometries makes this implementable today. That's a big deal.

Lalam: The cultural impact is that we can now build systems that respect the intrinsic geometry of data while maintaining the guarantees that make optimization reliable. That's a step toward more interpretable and trustworthy AI systems.

Tom: Well said. We've had a great discussion about "Iso-Riemannian Optimization on Learned Data Manifolds" by Willem Diepeveen and Melanie Weber. Thanks to everyone for joining us—Lu, Meng, Lalam. And thanks to our listeners for tuning in. We'll be back with the next paper soon. Until then, keep exploring the geometry of your data.

Willem Diepeveen, Melanie Weber

University of California, Los Angeles · Harvard University

math.OC, cs.LG, math.DG

Submitted: 2026-08-14

Updated: 2026-08-18

Code: https://github.com/Weber-GeoML/Iso-Riem-Opt

License: http://creativecommons.org/licenses/by-nc-nd/4.0/

Importance score: 82/100

Key concepts

Learned Data Manifolds
Data is not spread evenly but lives on a curved surface, or manifold. The paper learns this surface from the data using a pullback geometry to find a transformation that flattens the data out.
Iso-connection
This connection measures distances and movement along the manifold so that movement occurs at a constant speed in the original space. It fixes distortions caused by learned geometries, ensuring optimization respects Euclidean structure.
Iso-monotonicity
A new notion of monotonicity defined using the iso-connection instead of the classical Levi-Civita connection. This allows functions to be convex in a way that respects the learned geometry, preventing Riemannian gradient descent from failing.
Iso-barycentre
A generalization of the Riemannian barycentre that minimizes the sum of squared iso-distances to data points. It is shown to have properties like strong iso-monotonicity and iso-Lipschitz continuity on 1D manifolds.

Terminology

Summary

arXiv ID: 2510.21033v2 [math.OC], 6 May 2026

The paper addresses the challenge of performing optimization on learned data manifolds. The authors state: High-dimensional data with intrinsic low-dimensional structure is ubiquitous in machine learning and data science. While various approaches allow one to learn a data manifold with a Riemannian structure from finite samples, performing downstream tasks such as optimization directly on these learned manifolds remains challenging.

The central problem is that Euclidean convex functions cannot be assumed to be geodesically convex, and the associated Riemannian gradient fields are generally not monotone in the classical Riemannian sense. As a result, existing Riemannian optimization theory neither identifies a canonical vector field to use in first-order schemes nor guarantees their convergence in this setting.

The authors formally consider the problem of finding a vector x̄ ∈ M in a geodesic submanifold M ⊂ R d with respect to a Riemannian metric (R d, (·, ·)) on the ambient space that solves:

inf f(x), x ∈ M,

where f: R d → R is a Euclidean convex function. They aim to solve this via a first-order algorithm of the form:

x(k+1):= exp iso x(k)(−rΞ x(k)), r > 0, x(0) ∈ M,

where exp iso x is the iso-exponential mapping generated by (R d, (·, ·)), and Ξ ∈ X(M) is an appropriately chosen vector field with Ξ x̄ = 0.

The paper makes three main contributions:

The authors "extend classical first-order optimization techniques to the iso-Riemannian setting by introducing generalized notions of monotone and Lipschitz vector fields, together with characterizations that yield convergence conditions for the iso-Riemannian descent algorithm. They obtain linear convergence rates in terms of newly defined iso-monotonicity and iso-Lipschitz constants, and prove existence and uniqueness of zeros of the associated vector fields."

The authors "introduce an isometrized generalization of the Riemannian barycentre and show that the iso-barycentre field is naturally characterized by our iso-monotonicity and iso-Lipschitz notions – while not necessarily satisfying either in the classical Riemannian sense –, leading to existence and local uniqueness results and convergence of the iso-Riemannian descent algorithm to the iso-barycentre. Building on this, they formulate an isometrized variant of Riemannian K-means clustering, which exhibits marked improvements over both Riemannian and Euclidean K-means on synthetic and real datasets."

The authors formalize a notion of convexity based on the iso-connection and show how this newly defined iso-convexity is governed by the interaction between Euclidean convexity of a function f and (extrinsic) data manifold geometry. They identify "conditions under which the l2-projected gradient field of f is iso-monotone and iso-Lipschitz – again while not necessarily satisfying either in the classical Riemannian sense –, which ensures convergence of the iso-Riemannian descent algorithm to the minimizer."

The paper reviews standard notions from differential and Riemannian geometry, including smooth manifolds, tangent spaces, Riemannian metrics, the Levi-Civita connection, geodesics, exponential and logarithmic maps, and parallel transport.

For Euclidean pullback manifolds (R d, (·, ·) φ) generated by a diffeomorphism φ: R d → R d pulling back the standard Euclidean structure, the paper notes the following closed-form manifold mappings:

  • d φ R d(x, y) = ∥φ(x) − φ(y)∥2

  • γ φ x,y (t) = φ−1((1−t)φ(x) + tφ(y))

  • exp φ x(Ξ x) = φ−1(φ(x) + D xφ[Ξ x])

  • log φ x(y) = D φ(x)φ−1[φ(y) − φ(x)]

  • P φ y←xΞ x = D φ(y)φ−1[D xφ[Ξ x]]

The paper describes how normalizing flow training can be used to learn such diffeomorphisms by minimizing the negative log likelihood loss:

L(θ):= E X∼p data[−log p θ(X)] + (λ/2)∥θ∥22

where p θ(x):= (1/√(2π) d) e(−½∥φ θ(x)∥2) det(D xφ θ).

The key insight is that "if p data is feasible, i.e., there exists some θ* such that p θ* = p data, minimizing the loss will find this θ*. In other words, if we have p θ* = p data this means that geodesics γ φ x,y between data points move through regions with higher likelihood than the end points."

The paper reviews classical Riemannian optimization, noting that a vector field Ξ is monotone if:

(Ξ q − P q←pΞ p, P q←plog p(q)) q ≥ 0,

α-strongly monotone if the above is ≥ α·d M(p,q)2, and L-Lipschitz if ∥Ξ q − P q←pΞ p∥ q ≤ L·d M(p,q).

The classical convergence result (Theorem 1) states that for an α-strongly monotone and L-Lipschitz vector field with a singularity at p̄, the Riemannian gradient descent scheme converges linearly when 0 < r < 2α/L2.

The paper introduces the iso-connection under (R d, (·, ·)), defined as:

∇ iso Φ xΞ:= (1/∥Φ x∥2)∇ Φ x∥P(·)←xΦ x∥2Ξ

The manifold mappings under the iso-connection are related to those under the Levi-Civita connection via:

  • γ iso x,y (t) = γ x,y(τ x,y(t))

  • exp iso x(Ξ x) = exp x(τ x(Ξ x)Ξ x)

  • log iso x(y) = (∫01∥γ̇ x,y(s)∥2ds / ∥log x(y)∥2) · log x(y)

  • P iso y←xΞ x = (∥log x(y)∥2/∥log y(x)∥2) · P y←x(Ξ x)

The iso-distance is defined as d iso R d(x, y):= ∫01∥γ̇ x,y(s)∥2ds.

The paper defines:

Definition 1 (iso-monotone vector fields): A vector field Ξ is iso-monotone if for every x ≠ y ∈ M:

(Ξ y − P iso y←xΞ x, P iso y←xlog iso x(y))2 ≥ 0,

and α-strongly iso-monotone if the above is ≥ α·d iso R d(x,y)2.

Definition 2 (iso-Lipschitz vector fields): A vector field Ξ is L-iso-Lipschitz if:

∥Ξ y − P iso y←xΞ x∥2 ≤ L·d iso R d(x,y).

Definition 3 (locally iso-monotone vector fields): A vector field is locally iso-monotone if for any x ≠ y ∈ M and t ∈ [0,1]:

(P y←γ x,y(t)∇ iso γ̇ x,y (t) Ξ(·), P y←xlog x(y))2 ≥ 0.

Definition 4 (locally iso-Lipschitz vector fields): A vector field is locally L-iso-Lipschitz if:

∥P y←γ x,y(t)∇ iso γ̇ x,y (t) Ξ∥2 ≤ L·∥log y(x)∥2.

Lemma 1 (iso-Fundamental Theorem of Calculus): For a smooth vector field Ξ on a geodesic submanifold M:

Ξ y − P iso y←xΞ x = ∫01 (∥γ̇ x,y(t)∥2/∥log y(x)∥2) · P y←γ x,y(t)∇ iso γ̇ x,y (t) Ξ dt.

Theorem 2 (implications of local properties): Local iso-monotonicity implies iso-monotonicity; local α-strong iso-monotonicity implies α-strong iso-monotonicity; and local L-iso-Lipschitzness implies L-iso-Lipschitzness.

Proposition 1: Iso-geodesic velocity fields are iso-monotone.

Proposition 2: On any 1-dimensional strongly convex geodesic submanifold, the vector field −log iso(·)(w) is 1-strongly iso-monotone and 1-iso-Lipschitz.

Lemma 2 (unique zeros): An α-strongly iso-monotone vector field with at least one singularity has a unique singularity.

Theorem 3 (Convergence of iso-Riemannian descent): For an α-strongly iso-monotone and L-iso-Lipschitz continuous vector field with a singularity at x̄, the iso-Riemannian descent algorithm converges linearly to x̄ when 0 < r < 2α/L2.

Definition 5 (Iso-Riemannian barycentre): A point x̄ ∈ R d is an iso-barycentre of a data set x i i=1 n if:

(1/n)Σ i=1 n log iso x̄(x i) = 0.

Proposition 3 (Iso-geodesic midpoint): The iso-barycentre of two points x1, x2 connected by a unique length-minimizing geodesic satisfies x̄ = γ iso x1,x2 (1/2).

Proposition 4 (iso-barycentres on R): The iso-barycentre of any data set on R with respect to any Riemannian structure satisfies x̄ = (1/n)Σ i=1 n x i, i.e., it equals the Euclidean mean.

Theorem 4 (Local existence): For points in a simply connected strongly convex geodesic submanifold, there exists at least one iso-barycentre.

Theorem 5 (Properties of 1D iso-barycentre fields): On a 1-dimensional strongly convex geodesic submanifold, the vector field −(1/n)Σ i=1 n log iso(·)(x i) is 1-strongly iso-monotone and 1-iso-Lipschitz.

Corollary 5.1: Under the assumptions of Theorem 5, there is a unique iso-barycentre.

The algorithm initializes at the Riemannian barycentre and iterates: computing the iso-barycentre field, performing iso-Riemannian descent with line search, and checking convergence.

Algorithm 2 (Iso-K-means): The algorithm adapts Lloyd's algorithm by using iso-distances for cluster assignment and iso-barycentres for centroid updates.

The paper notes: "only iso-K-means is able to find both correct clusters while maintaining meaningful centroids, which can be attributed to the fact that the underlying pullback metric does not encode our intuition on l2-proximity, which deteriorates clustering performance when not corrected for (through isometrization)."

Definition 6 (Iso-convex functions): A function f: R d → R is iso-convex on a strongly convex geodesic submanifold M if the vector field (x ↦ P x∇f(x)) ∈ X(M) is iso-monotone, and α-strongly iso-convex if it is α-strongly iso-monotone, where P x is the l2-projection onto the tangent space at x ∈ M.

Proposition 5 (Equivalences on R): On R, (α-strong) convexity of f is equivalent to (α-strong) iso-convexity, and L-Lipschitzness of f′ is equivalent to L-iso-Lipschitzness.

Theorem 6 (properties of 1D l2-projected gradient fields): For a smooth function f satisfying certain second-derivative bounds and geometric assumptions on a 1-dimensional submanifold, the vector field P(·)∇f is (α+β)-strongly iso-monotone and (L+M)-iso-Lipschitz, where α, β, L, M are constants related to the function's convexity and the manifold's geometry.

Corollary 6.1: Under the assumptions of Theorem 6, the l2-projected gradient iso-Riemannian descent converges linearly to the unique point satisfying P x̄∇f(x̄) = 0.

The algorithm iterates: computing the projected gradient, performing iso-Riemannian descent with line search based on function value decrease, and checking convergence.

The paper considers inverse problems of the form:

inf x∈M (1/2)∥Ax − b∥22

For overdetermined A (d′ ≥ d, rank(A) = d), the manifold constraint can lead to faster convergence if the manifold lies within a subspace orthogonal to eigenvectors with large eigenvalues.

For underdetermined A (d′ ≤ d, rank(A) = d′), the manifold constraint turns an ill-posed problem into a well-posed one: "if α + β > 0, the manifold constraint has turned the ill-posed problem into a well-posed one."

For compressed sensing with restricted isometry property (1−δ)d2 ≤ ∥A·log iso x(y)∥2 ≤ (1+δ)d2, one gets α = 1−δ and L = 1+δ.

The paper uses two synthetic data sets:

  • River data set: φ river(x):= (x1 − β·sin(x2), sinh(η·x2)) with β = 5, η = 0.25

  • Spiral data set: φ spiral with β = 0.25

Results show:

  • Both Riemannian and iso-barycentres yield more interpretable data representations than the Euclidean mean

  • Only iso-K-means finds both correct clusters while maintaining meaningful centroids

  • The iso-monotonicity ratio suggests α is much smaller than 1, while L tends to be much larger

For MNIST with a learned pullback structure:

  • Both Riemannian and iso-barycentres give similar, interpretable representations (looking somewhat like an 8), whereas the Euclidean mean yields a blurry mix

  • For denoising the digit four using a rank-20 approximation at the iso-barycentre, the algorithm converges to a nearly noiseless representation in just 10 iterations

The paper notes numerical challenges: numerical errors eventually become significant enough to disrupt the optimization process due to single precision in PyTorch and numerical approximations for iso-manifold mappings.

The paper concludes: This study demonstrates the necessity of rethinking optimization on Riemannian manifolds when using learned data geometries, and argues for a framework based on the iso-connection.

Future directions include:

  • Systematically quantifying effects of numerical approximation and rounding errors

  • Developing higher-order numerical approximations

  • Replacing the iso-exponential map by a retraction

  • Enhancing optimization with higher-order methods

  • Advancing non-smooth optimization techniques (e.g., iso-medians)

  • Learning geometries that are inherently more favorable for optimization

MW was partially supported by NSF awards CBET-2112085 and DMS-2406905, and an Alfred P. Sloan Research Fellowship in Mathematics. Some numerical computations were run on the FASRC Cannon cluster at Harvard University.

Improvements for AI systems

Based on the paper, here are specific improvements that can be made to AI systems, along with what the improved systems can do:

Improvement: Replace standard Riemannian gradient descent with iso-Riemannian descent (IRD) using the iso-connection instead of the Levi-Civita connection when optimizing functions over data-driven geometries.

What the improved system can do:

  • Converge linearly to minimizers of Euclidean convex functions constrained to learned manifolds, even when the function is not geodesically convex under the learned metric

  • Guarantee convergence with explicit step-size bounds derived from iso-monotonicity and iso-Lipschitz constants (Theorem 3)

  • Solve inverse problems (e.g., denoising, compressed sensing) with provable convergence rates, where standard Riemannian methods fail or require impractically small step sizes

  • Handle cases where the geodesic subspace is open in the ambient space (not strictly lower-dimensional), which is common in clustering and regularization tasks

Abstract

High-dimensional data with intrinsic low-dimensional structure is ubiquitous in machine learning and data science. While various approaches allow one to learn a data manifold with a Riemannian structure from finite samples, performing downstream tasks such as optimization directly on these learned manifolds remains challenging. In particular, Euclidean convex functions cannot be assumed to be geodesically convex, and the associated Riemannian gradient fields are generally not monotone in the classical Riemannian sense. As a result, existing Riemannian optimization theory neither identifies a canonical vector field to use in first-order schemes nor guarantees their convergence in this setting. To address this, we introduce notions of convexity, monotonicity, and Lipschitz continuity induced by a connection different from the Levi-Civita connection, namely the recently proposed iso-connection. Within this iso-Riemannian framework, we propose an iso-Riemannian descent algorithm and provide a detailed convergence analysis. We then show, for several downstream tasks - including iso-Riemannian barycentre computation and the optimization of Euclidean convex functions over learned data manifolds - that iso-convexity, iso-monotonicity, and iso-Lipschitz continuity form the right set of assumptions to reconcile learned geometry with Euclidean convexity. Experiments on synthetic and real datasets, including MNIST, endowed with a learned pullback structure, demonstrate that our approach yields interpretable barycentres, improved clustering, and provably efficient solutions to inverse problems, even in high-dimensional settings. Taken together, these results show that iso-Riemannian optimization provides a natural geometric framework for designing and analyzing algorithms on learned data manifolds.

Sources

Related papers