Learning Transfers: Kan Extensions of Topological Invariants of Learned Representations
summary
The gist
The gist A representation transfers if it stays usable once the task has changed, and this paper makes that structure explicit and computable by defining transfer discrepancy as the supremum over
In short
The paper introduces a mathematical framework to quantify how well a learned representation transfers between different tasks. It defines 'transfer discrepancy' as the largest distance between an invariant forced by task change and the actual observed target invariant. This allows researchers to measure structural transfer, moving beyond simple accuracy scores to understand what part of the source structure is preserved.
Key concepts
- Left Kan Extension (LanJ F)
- This mathematical tool constructs a new invariant for a target task based on the source representation and how the task changes. It forces the target invariant to incorporate both the original source information and any structural changes imposed by moving from one task to another, ensuring compatibility.
- Transfer Discrepancy
- This is a metric used to score transferability. It measures the maximum distance between this constructed target invariant (the Kan extension) and the actual invariant observed in the target task. A smaller discrepancy indicates better structural transfer.
- Functorial Invariants
- These are mathematical objects that assign an invariant structure to data representations. The paper uses these functors to model how a source representation transforms when subjected to a change of task, allowing for rigorous comparison between different learned models.
Terminology used across episodes
This episode discusses
- Learning Transfers: Kan Extensions of Topological Invariants of Learned Representations · Paper Radio
- Persistence Images: A Stable Vector Representation of Persistent Homology
- Latent Space Oddity: on the Curvature of Deep Generative Models
- Kan Approximations of the Persistent Homology Transform
- Representation Topology Divergence: A Method for Comparing Neural Network Representations
- Topological Signatures of ReLU Neural Network Activation Patterns
- A Topology Layer for Machine Learning
- Statistical topological data analysis using persistence landscapes
- A Topological Regularizer for Classifiers via Persistent Homology
- Neural Ordinary Differential Equations · Paper Radio
- Augmented Neural ODEs
- Connectivity-Optimized Representation Learning via Persistent Homology
- Kan Extension Transformers: A Categorical Unification of Attention, Diffusion, and Predict-Detach Self-Conditioning
- Learning Is a Kan Extension
- Topological Data Analysis of Decision Boundaries with Application to Model Selection
- Neural Persistence: A Complexity Measure for Deep Neural Networks Using Algebraic Topology
- Kan Extensions in Data Science and Machine Learning
The paper
Learning Transfers: Kan Extensions of Topological Invariants of Learned Representations · Read on arXiv
Luciano Melodia
Friedrich-Alexander University Erlangen-Nürnberg
When a learned representation is reused for a new task, which of its structure should survive? A task is a small category, a change of task is a functor J, and the invariants of a trained representation form a functor F. The left Kan extension Lan JF is the invariant that the source and the change of task force on the target: by its universal property, every comparison of the source with an observed target invariant factors through it. The transfer discrepancy is the largest distance, over the target objects, between this extension and the observed invariant. For persistent homology the extension is a cokernel over comma categories, and a functorial version of the discrepancy, which also compares the maps between target objects, is 1-Lipschitz under interleavings. On finite samples the discrepancy is a bottleneck distance, computed without approximation. For data glued from two pieces whose exclusive parts lie farther apart than the filtration scale, a Mayer-Vietoris sequence identifies what extending homology loses, while extending chains loses nothing. For a merge of labelled point clouds, the degree-zero discrepancy vanishes if and only if some 1-Lipschitz classifier has a margin above the filtration scale at every sample point. In experiments on sampled manifolds, trained encoders and pretrained image models, every exact prediction holds on every sample. The discrepancy of truncated invariants recovers a hidden merge of separated parts far above chance and tells the intended target of a merge from a permuted control, but across pretrained models it does not track the accuracy of a linear probe.
Transcript
Introduction to the show: ident: AI Radio. Generated commentary on the latest Artificial Intelligence papers.
Tom: Today's paper: "Learning Transfers: Kan Extensions of Topological Invariants of Learned Representations".
Jane: The gist A representation transfers if it stays usable once the task has changed,
Tom: First, who's behind it and why it matters.
Title and authors: Tom: We’ve got the title "Learning Transfers: Kan Extensions of Topological Invariants of Learned Representations." It sounds like they are connecting topology, which is pretty abstract, to how AI representations move between different tasks.
Jane: They’re using this idea to make that structural map between tasks explicit and computable for the first time in this way. It moves beyond just looking at the objects themselves.
Lu: The authors are showing how you can model a task as a small category A, the change of task as a functor J from A to B, and the representation as a functor F into a category of invariants.
Meng: So they’re using these mathematical tools to define what structural information must be preserved when you shift from one problem domain to another in an AI system.
Tom: And the main result is that this structure the target has to exhibit is actually the left Kan extension of the source functor along that change of task.
Jane: They are defining their transfer discrepancy as the supremum over target objects of the distance between that extension and what you observe in your target invariant, G(b).
The paper's summary: Tom: So to summarize, they’re modeling transfer learning by setting up this categorical structure. They define a change of task as a functor J from A to B and the representation as F into invariants.
Jane: Then they define the target-side invariant LanJ F evaluated at b as the left Kan extension of F along J evaluated at b, which records all source invariants over that point along with the identifications from J.
Lu: The paper proves several things about these Kan extensions. For instance, for forgetting structure, they show that LanJ F evaluated at a target object is equivalent to the colimit of all the source invariants in V.
Meng: That’s interesting because it shows how you can compute this extension using just the coproduct in V for forgetting structure. It connects the abstract Kan extension to something concrete.
Tom: And they also look at class refinement, where for a unique object c, it shows that LanJ F evaluated at d is just F evaluated at c.
Jane: That means when you have a unique refinement, the transfer structure simplifies considerably because it just depends on the source representation at that point.
The paper's improvements: Tom: One key improvement they suggest is defining an invariant target as a triple (V, S, dV). V needs to admit all small colimits, and S is a class of objects closed under isomorphism where the distance dV is an extended pseudometric on that class.
Jane: Then the transfer discrepancy CompJ (F, G) is defined as the supremum over b in B of the distance between LanJ F evaluated at b and G evaluated at b.
Lu: For chain complexes, they use an invariant target of (ChK, Sch, dch), where dch only takes two values: zero and one. This keeps things very simple for those settings.
Meng: They also give examples for persistence modules, where the invariant target is (PersK, Persq-tame K, dI), and the distance metric they use is called the interleaving distance.
Tom: And for modules of finite type, they show that this discrepancy score computes exactly the bottleneck distance without any approximation. That’s a big deal for models with finite type modules.
Conclusion: Jane: So wrapping up, this paper adds a structural map between tasks and the invariant it forces on the target, which is what they call transfer discrepancy CompJ (F, G). It reads the morphisms of the source and not just its objects.
Lu: The vanishing of this discrepancy is strictly weaker than exact transferability, which means you can have low discrepancy without forcing an isomorphism between functors.
Meng: They also show that the colimit is computable through a cokernel presentation, a Lipschitz bound, and exact evaluation by the distance dV. These are all practical computational tools for this framework.
Tom: It sounds like they’ve established that the discrepancy is non-expansive in both arguments, as shown by Theorem thirteen. And because of this, you get a score that is computable without approximation for every n and b.
Jane: The findings are validated across three experiments involving sampled manifolds and learned latent clouds, so it’s not just theoretical math in isolation.
Lu: They tested these concepts on sampled manifolds and learned latent clouds, showing how the discrepancy behaves in practice, which gives us a feel for its utility.
Meng: It seems like this framework provides a concrete way to detect structural changes rather than just looking at distributional shifts, which is what we need in deployment.
Tom: So that’s it on "Learning Transfers: Kan Extensions of Topological Invariants of Learned Representations." This paper gives us a new way to understand how representations move between different tasks.
More episodes
- 2607.04209-Ball Differential Privacy: How to Mitigate Data Reconstruction with Less Noise
- 2605.19156-How Far Are We From True Auto-Research?
- 2606.03031-AUDITFLOW: Executable Symbolic Environments for Structured Financial Reporting Verification
- 2607.23500-Formalizing Flag Algebras in Lean
- 2606.31552-Improving multichannel speech enhancement through accurate room-acoustic simulations
- 2607.10203-Does Depth Survive Composition, and Can a Router Use It? Adaptive Compute in Latent World Models
- 2607.08234-RhyMix: A Lightweight Adaptive Multi-Rhythm Network for Long-Term Time Series Forecasting
- 2607.09015-Correlation-Aware Contextual Bandits with Surrogate Rewards for LLM Routing
- 2608.05219-When Privileged Guidance Misaligns: State-Matched Routing and Contextualized Self-Distillation for Multi-Turn Agents
- 2110.13601-DAG Card is the new Model Card