DeltaNet-PC: Test-Time Memory Without Backpropagation Through Time

summary

The gist

The gist: DeltaNet-PC develops a predictive coding system that learns to perform test-time verbatim memorization of sequential information without using backpropagation through time.

In short

DeltaNet-PC develops a predictive coding system that learns to memorize sequential information during testing without needing backpropagation through time. It treats per-step memory updates as latent variables, enabling test-time verbatim memorization of single sequences, multiple sequences, and key-value pairs with high accuracy.

Key concepts

Latent Variables: Instead of learning slow weights that represent memory updates, DeltaNet-PC treats the per-step memory update (∆St) as a latent variable. This allows the system to learn how to store information directly during testing, rather than relying on traditional, slow parameter learning methods.
Predictive Coding: The system uses predictive coding principles where memory is modeled through an energy function based on the difference between predicted and actual values. This framework allows the model to infer missing or sequential information by minimizing prediction errors across a sequence.
Recursive Least Squares (RLS): The update rule for the latent memory variable ∆St is derived as a Recursive Least Squares algorithm. This specific update mechanism is computationally efficient because it avoids needing to compute an inverse at every time step, making the system feasible for online learning scenarios.
Rotary Position Code (RoPE): DeltaNet-PC uses a rotary position code with learned per-band frequencies to encode the position of keys within a sequence. These learned frequencies allow the system to adaptively use positional information, where interference between tokens at different positions drives the frequency learning.

Terminology used across episodes

Reinforcement Learning (RL) — explained in 25 episodes
Sample Complexity — explained in 20 episodes
Retrieval-Augmented Generation (RAG) — explained in 18 episodes
Federated Learning — explained in 17 episodes
Quantum Fisher Information (QFI) — explained in 16 episodes
Flow Matching — explained in 15 episodes

This episode discusses

The paper

DeltaNet-PC: Test-Time Memory Without Backpropagation Through Time · Read on arXiv

Todd Morrill, Richard Zemel

Columbia University

Test-time memorization of sequential information is a foundational capability for any intelligent system that needs to rapidly store and retrieve data, as demonstrated by the attention mechanism in large language models. However, Transformers and linear attention variants are trained via backpropagation through time (BPTT), which has drawbacks such as its enormous memory requirements. Predictive coding (PC), an algorithm that optimizes a prediction-error energy function with local updates, offers a compelling solution but still lacks test-time memory for sequential data. We develop a solution to this problem by treating the per-step memory update ΔS t as a latent variable, as opposed to a slow weight to be learned. By taking the log probability of a standard PC-based probabilistic model, we derive a PC-native solution for verbatim sequence memory, and relate our update rule to common choices in the linear attention literature (e.g., DeltaNet). We present an extensive set of empirical tests to probe our system's properties, including its scaling capacity, memory footprint, and ability to infer a context from evidence (e.g., bank (river) vs. bank (money)). We show how DeltaNet-PC can be used for storing single sequences of data, multiple sequences of data, and key-value associative memory pairs with perfect recall accuracy.

Transcript

Introduction to the show: ident: AI Radio. Generated commentary on the latest Artificial Intelligence papers.

Tom: I'm Tom.

Jane: And I'm Jane, here with Lu, Meng and our in-house large language model, Lalam.

Tom: Today's paper: "DeltaNet-PC: Test-Time Memory Without Backpropagation Through Time".

Jane: The gist: DeltaNet-PC develops a predictive coding system that learns to perform test-time verbatim memorization of sequential information without using backpropagation through time.

Tom: First, who's behind it and why it matters.

Title and authors: Tom: To summarize what DeltaNet-PC does, it takes a standard predictive coding model and modifies the energy function to achieve test-time memorization by focusing specifically on that latent variable update mechanism.

Jane: Basically, they use a joint distribution p(k one:L, v one:L,S) restricted to the memory model over a sequence of length L, and the energy function is set up like ridge regression on all key and value pairs for every step > <ref:2610.15185#pg1>.

Lu: The energy function involves that ridge term epsilon twoS 2F plus terms for stored pairs and a value-orthogonality regularizer that keeps values from interfering with each other too, which is detailed in section B <ref:2610.15185#pg1>.three > <ref:2610.15185#pg1>.

Meng: And they derive the update rule for the latent variable DeltaSt by setting the gradient of E t-one(S t-one +) to zero, which results in a recursive least squares update rule > <ref:2610.15185#pg1>.

Lalam: That RLS update is what allows them to get an update like S t = S t-one + g t T (p t-1k t), which is equivalent to the recursive least squares algorithm but without needing an inverse at every time step > <ref:2610.15185#pg1>.

Tom: That efficiency gain with RLS is a key part of the mechanism that makes this approach viable for online learning, unlike methods that might require iterative optimization to settle these latents >

Jane: And they show this works on tasks like memorizing a song where the chorus is followed by different bridges, requiring non-Markovian memory in a single sequence >

Lu: They also address how context uncertainty in multi-episode settings can be resolved through contextual inference, using an energy function like E p(c) = one/two enc(x p) - S L R W k enc(x p-one), p theta + psi(c) squared > <ref:2610.15185#pg1>.

The paper's summary: Tom: The paper highlights a few key improvements right here. First, the use of RLS instead of iterative optimization to settle the latents is a big step because it’s more efficient for online scenarios >

Jane: And they also show how they can resolve context uncertainty in multi-episode settings through this contextual inference mechanism, which is essentially inferring the context latent from observed transitions using that energy function >

Lu: They're also showing that they can learn how to use positional information adaptively by having rotary frequencies learn based on task needs, so sequential recall queries each transition at its own position >

Meng: I'm interested in the part about adaptive positional learning because it suggests that the binding of a written sequence just doesn't exert force, but interference with other occurrences of the same token at different positions moves those frequencies >

Lalam: And they also show that for key-value memory, they can achieve one hundred percent test set accuracy in as few as thirty-two training examples, which is a significant improvement over the one thousand twenty-four needed by DeltaNet >

The paper's improvements: Tom: So to wrap up on DeltaNet-PC, the main implication is that treating matrix-valued updates as latent variables instead of slow weights allows for test-time verbatim memorization, which was a capability previously hard to achieve without backpropagation through time >

Jane: It’s positioned well for online learning and in-memory hardware because of those strict requirements for local learning in space and time and bounded memory in the sequence length >

Lu: The work demonstrates how predictive coding natively provides linear attention machinery, allowing it to be positioned for scenarios where we need rapid context inference from observed transitions >

Meng: From a deployment standpoint, the RLS update rule makes this feasible because you don't need heavy iterative optimization to settle those latents before you can use them in production >

Lalam: And the whole paper is about showing that DeltaNet-PC achieves test-time verbatim memorization of sequential information by treating matrix-valued updates as latent variables, and it sets a new benchmark for this capability.

Conclusion: Tom: So to wrap up on DeltaNet-PC, we’re talking about Test-Time Memory Without Backpropagation Through Time. The main implication is that treating matrix-valued updates as latent variables instead of slow weights allows for test-time verbatim memorization, which was a capability previously hard to achieve without backpropagation through time >

Jane: It’s positioned well for online learning and in-memory hardware because of those strict requirements for local learning in space and time and bounded memory in the sequence length >

Meng: From a deployment standpoint, the RLS update rule makes this feasible because you don't need heavy iterative optimization to settle those latents before you can use them in production >

Tom: It’s about moving the learning from being a slow parameter update to being an inference problem on these latent variables, which is what makes it potentially runnable in real-world scenarios >

Jane: And they show this architecture can handle within-episode structure tests, multi-episode settings, and key-value binding tests >

Meng: That means if you have a song with a chorus followed by different bridges that are non-Markovian, the model can handle that because of how the memory is structured >

Lalam: They also address how to use positional information adaptively by having rotary frequencies learn based on task needs, so sequential recall queries each transition at its own position >

Tom: That RLS update is what allows them to get an update like S t = S t-one + g t T (p t-1k t), which is equivalent to the recursive least squares algorithm but without needing an inverse at every time step >

Jane: They also show how they can resolve context uncertainty in multi-episode settings through this contextual inference mechanism, which is essentially inferring the context latent from observed transitions using that energy function >

Meng: I'm interested in the part about adaptive positional learning, because it suggests that the binding of a written sequence just doesn't exert force, but interference with other occurrences of the same token at different positions moves those frequencies >

Tom: So to wrap up on DeltaNet-PC: the main implication is that treating matrix-valued updates as latent variables instead of slow weights allows for test-time verbatim memorization, which was a capability previously hard to achieve without backpropagation through time >

Jane: It’s positioned well for online learning and in-memory hardware because of those strict requirements for local learning in space and time and bounded memory in the sequence length >

Meng: From a deployment standpoint, the RLS update rule makes this feasible because you don't need heavy iterative optimization to settle those latents before you can use them in production >

Jane: And they show this architecture can handle within-episode structure tests, multi-episode settings, and key-value binding tests >

More episodes

← Home