2608.07436-Post-Grokking Collapse at the Representation–Readout Interface in Muon-Trained Transformers

Introduction to the show: ident: Paper Radio. Generated commentary on the latest Artificial Intelligence papers.

Tom: Next we'll be talking about the paper "Post-Grokking Collapse at the Representation–Readout Interface in Muon-Trained Transformers".

Jane: The paper was written by Ali Janati, Kaoutar El Maghraoui, Andrei Kanavalau and Anass Belfatmi from Data Science Institute, Columbia University and Department of Computer Science, Columbia University and Stanford University and CentraleSupélec.

Tom: Stay tuned as we take you through the paper and discuss its implications.

Paper Summary: Tom: We just introduced a paper about grokking on modular arithmetic, and the headline is genuinely surprising. The Muon optimizer reaches the grokking threshold faster than AdamW, but the solutions it finds don't hold — every one of nine Muon configurations generalizes and then loses generalization.

Jane: And that's not a quirk of Muon alone, because four of their seven grokking AdamW configurations also collapse afterward. The paper argues the failure sits at the interface between the hidden representation and the readout layer, where the residual stream has no privileged basis and the loss stops constraining how the two sides align.

Lu: Right, once training accuracy hits 100 percent, the gradient drops to tiny values, but the two optimizer groups keep moving at different rates. They measure an elasticity of minus 0 point 03 for the Muon group versus plus 1 point 5 for the AdamW groups, meaning the hidden step doesn't track the gradient while the readout does, and the groups separate eight times faster per parameter.

Meng: That separation is what they call hidden–readout misalignment. And the elegant part is they can freeze either side and the collapse disappears, and if you anchor the embeddings and the readout after the circuit forms, they get zero sub-threshold evaluations across five seeds and more than four hundred fifty thousand post-grokking steps.

Lalam: The bigger picture is that a circuit can be present, correct, and completely masked. They project onto the Fourier family that computes modular addition, and in isolation that family gives 100 percent even when the full model has collapsed to 45 point 85 percent. Rescaling just that family, with no retraining, brings the model back to 99 point 9 percent.

Tom: So the usual progress measures would tell you the circuit is intact right at the moment it stops working. The support of the dominant frequencies is unchanged, the power distribution has cosine similarity 0 point 9899, and the function is dead.

Jane: That's the piece that matters for mechanistic interpretability, because it means checking whether a circuit is present isn't enough. You also need to know whether the rest of the representation is drowning it out.

Lu: And they show the collapse is different from the anti-grokking phase reported before, because training accuracy falls with test accuracy, from 100 percent to 21 percent. It's a joint failure of the representation and the readout, not a generalization-only failure.

Meng: They even trace it through depth, where the instability spreads and gets worse, and they localize where the Fourier code appears — at an MLP activation, not at attention writes. That's a concrete legible circuit with a concrete failure mode.

Lalam: For eye training practice, this suggests split-optimizer routing has a hidden cost that shows up after convergence. Muon still wins on speed, but you need to stabilize the coordinates it leaves unconstrained, or the speed advantage evaporates when the model falls off the cliff.

Tom: And the fix they propose is remarkably simple — freeze the embedding and unembedding after the circuit forms. We'll get into the details, starting with the abstract and the first page.

Jane: Good place to start, because the abstract already packs in the nine-point sweep, the 27 point 59 percent AdamW failure, and the central claim about the representation–readout interface.

Page 1 of the Paper: Tom: We've set the scene with the headline, so now let's look at what page 1 actually claims in the abstract. It says under the standard split, Muon reaches the grokking threshold on modular addition in fewer steps, but the solutions don't hold.

Jane: And the abstract already gives the sweep: nine configurations on addition mod 113, all nine grok, all nine subsequently lose generalization. The selected AdamW reference also isn't stable across seeds — it falls below threshold on four of five, down to 27 point 59 percent.

Lu: What caught me is the phrase "the failure arises at the interface between the representation and the readout, which the residual stream leaves identified only jointly, up to an invertible map the loss does not select." That asymmetry is the engine of the whole paper.

Meng: The abstract also reports the step elasticity numbers and the eightfold separation. And it previews the Fourier analysis: the addition family reaches exactly 100 percent in isolation while the full model gets 45 point 85 percent, then rescaling restores 99 point 9 percent.

Lalam: For listeners who haven't seen the abstract, the deeper point is that grokking itself might be a special case of what they call masking resolving upward. The same quantity that buries a working circuit before grokking is the one that buries it after collapse.

Tom: Right, and page 1 launches the introduction by reminding us that grokking is the standard testbed for mechanistic accounts. Modular addition is the canonical instance because the algorithm is known — transformers compute in a Fourier basis.

Jane: And that known algorithm is what lets them do functional decomposition instead of spectral heuristics. Instead of just looking at dominant frequencies, they read the representation in the basis the model actually computes in, and compare against the number of frequencies the algorithm needs.

Lu: One thing I'd underline from page 1 is that Muon's speedup isn't just a curiosity; it's an instrument. Because the grokked circuit appears within a few thousand steps, you can observe the post-grokking regime repeatedly inside single runs, branch matched trajectories from bit-identical states, and intervene cheaply.

Meng: And they flag that the instability isn't dependent on the specific setting. Two moduli, two widths, two training fractions, two operations, depths 1, 2, and 4 — it all holds.

Lalam: The intro also distinguishes their failure from the existing literature. Training accuracy falls with test accuracy, so it's not the anti-grokking phase where generalization collapses while the training set stays solved. That distinction shapes everything that follows.

Tom: And it sets up the residual stream argument, which is the theoretical backbone. We'll see that spelled out on page 2.

Jane: Absolutely — the next page has the figure showing the collapse at ten-step resolution, with spectral similarity flat while accuracy plunges.

Page 2 of the Paper: Tom: We left off with the residual stream claim, and page 2 makes it concrete. The logits depend on the final residual and the unembedding only through their product, so any invertible rotation of the stream can be absorbed by rotating the unembedding the other way, and the function stays identical.

Jane: That means the pair is identified only jointly, and the loss has no reason to prefer one coordinate system over another. With split-optimizer routing, Muon manages the hidden matrices while AdamW manages the embeddings and the readout, so the two sides co-adapt without any shared constraint.

Lu: Page 2 also has the key figure, and it's pretty dramatic. At ten-step resolution, test accuracy falls from 100 percent to 19 point 04 percent while the addition-family Fourier support has Jaccard index 1 point 0000 and power cosine 0 point 9899. The standard progress measures report an intact circuit at the exact step the model stops computing the task.

Meng: And then it recovers to 99 point 9 percent within 300 steps, but on a different set of frequencies — the power cosine to the pre-collapse state is only 0 point 55. So the model doesn't go back to what it had; it re-solves the task elsewhere.

Lalam: That's a profound point for interpretability. A circuit can be present and correct, and the representation around it can change so that its contribution is outvoted. Support and power measures are invariant to the change of basis that kills the function.

Tom: The page also introduces the elasticity measurement. Over the 691 steps before a collapse, the training loss sits at 1 point 5e-7, gradient norms are around 1e-6 and 1e-5, and the hidden group's applied step has elasticity minus 0 point 026 versus plus 1 point 5 for the embeddings and readout.

Jane: So when the gradient drifts, Muon's normalized orthogonalization keeps the step size constant while AdamW's step grows with the gradient. The two groups separate at eight times the rate per parameter, and neither side can decode the other's representation afterward.

Lu: That's where the term hidden–readout misalignment comes from. And it's not caused by a new task or distribution shift — the training set stays solved throughout the quiet window, and both accuracies fail together.

Meng: Page 2 also previews the family argument: the task selects the Fourier family. Trained on subtraction, the model uses the (k, -k) modes at 100 percent in isolation while (k, k) falls to chance.

Lalam: For the bigger picture, the residual stream basis symmetry isn't just a theoretical curiosity. It tells you why freezing one side works. If a coordinated change of basis leaves the loss unchanged, then nothing pulls the two sides back together once they start moving at different rates.

Tom: And that's the conceptual foundation for the causal experiments later. The next page moves from the framework to the contributions, with the bullet list of what they actually established.

Jane: Right, page 3 lays out the claims about localizing the failure, anchoring it, and separating the two collapse modes.

Page 3 of the Paper: Tom: Page 3 is the contribution list, and the first bullet is about localizing the failure. They branch from bit-identical states, freeze either group, and both freezes suppress the collapse. Within the auxiliary group, the unembedding is the component whose motion the failure requires.

Jane: And long-run prevention is striking: holding embeddings and readout fixed once the circuit forms leaves no post-grokking evaluation below 95 percent across five runs, 451,400 steps, and 4,519 evaluations. Meanwhile the unfrozen arm records 137 to 321 sub-threshold evaluations on each paired seed.

Lu: The second bullet is the task-selected family. Across 43 solved checkpoints spanning five seeds and three regimes, projection onto the (k, k) Fourier family gives exactly 100 percent at every one, and ablating it leaves 1 point 71 percent. Subtraction flips it exactly — (k, -k) gives 100 percent and (k, k) drops to chance.

Meng: The third bullet says the update rule shapes how widely the representation is spread. Muon occupies 326 effective conjugate pairs on addition versus AdamW's 4 point 95, and the family holds 28 percent of non-constant power versus AdamW's 91 percent. Ablating Muon's normalized orthogonalization collapses that to 4 point 11.

Lalam: That's a really clean separation: what the model computes is chosen by the task, but how the computation is distributed across the spectrum is chosen by the optimizer. The ablation doesn't change which family is sufficient, it just concentrates the representation.

Tom: Then the fourth bullet introduces the two collapse modes. Filtering distinguishes circuit failure, where the isolated family no longer solves the task, from circuit masking, where it still gives 100 percent while the full model reaches 45 point 85 percent.

Jane: And the masking case decomposes exactly: the family's margin through the unembedding is +5 point 72, the adversarial remainder is −6 point 43, and rescaling the family alone lifts the model to 99 point 9 percent. Grokking is that same masking resolving upward.

Lu: The fifth bullet separates construction and alignment in depth. Fourier-specific sensitivity appears at an MLP in ten of eleven checkpoints, at no attention write. Direct readout of a post-block residual reaches 95 percent only at the final block.

Meng: So the circuit is built in one block and made legible in another. That structural separation explains why depth makes the instability worse — the alignment machinery doesn't scale with the number of blocks.

Lalam: For eye training, what stands out from this page is the asymmetry between speed and stability. Muon gives you faster grokking, but you inherit an unconstrained direction that AdamW keeps under control by being inefficient. The paper's fix is to freeze the unconstrained coordinates.

Tom: And that naturally raises the question of whether the setup itself is doing something special. The next page starts the methods section, describing the exact task, data, and architecture.

Jane: Yes, page 4 lays out modular arithmetic with p=113, the 30 percent training split, and the decoder-only transformer with no normalization layers. All of that matters for the invariance argument.

Page 4 of the Paper: Tom: We've covered the contributions, so now the methods. Page 4 describes the task: modular addition with p=113, token sequence

a, b, =: , a vocabulary of 114 tokens including the equals sign, and 113 output classes. The full grid is 12,769 pairs, and they train on a fixed random 30 percent, full batch with cross-entropy.

Jane: The architecture is a decoder-only transformer with d_model 128, four heads of size 32, MLP width 512, ReLU, and sequence length 3. Crucially, it has no normalization layers, so the residual stream is a plain sum of embeddings, attention writes, and MLP writes.

Lu: And that absence of normalization matters for the analysis. LayerNorm is the one operation that would distinguish a coordinate system in the residual stream. Without it, the stream is exactly invariant to an invertible change of basis absorbed by the surrounding matrices.

Meng: They also list the exact readers and writers. Four matrices write into the stream — token embedding, position embedding, attention output projection, MLP output projection — and three read from it — fused QKV, MLP input, unembedding. That's the interface the whole paper studies.

Lalam: Even if you did have RMS normalization, the invariance isn't fully removed; it's restricted to orthogonal transformations. The paper notes that would be an 8,128-dimensional symmetry at this width, which shows the size of the unconstrained direction they're dealing with.

Tom: There's a neat design detail in how they vary depth — nested initialization. Embeddings, block zero, and the unembedding are shared across depths, and deeper models start from a strict superset of the shallower initial state. That makes depth comparisons clean.

Jane: And the parameter counts are small enough for full-batch training: depth one has 226,048 parameters, depth two 422,656, depth four 815,872. That's why they can run the causal branches cheaply.

Lu: One thing I appreciate is the explicit statement that every projection is bias-free. It's not just a simplification; it ensures the unembedding is a pure linear map, which is what lets them decompose the margin exactly later.

Meng: The dataset split is generated once from the run seed and shared by all matched comparisons. That means any difference between matched runs is attributable to the optimizer or intervention, not to sampling variability.

Lalam: For the bigger picture, this minimal setup is what lets them use modular arithmetic as a testbed. The algorithm is known, the architecture is stripped down, and the basis freedom is exposed rather than hidden under normalization.

Tom: And the next page gets into the optimizer routing, which is the other half of the experimental design. Muon gets the hidden matrices, AdamW gets embeddings and the unembedding, with the readout kept as its own group.

Jane: Exactly, page 5 has the parameter group table and the operational definitions for grokking and stability, plus the seed replication strategy.

Page 5 of the Paper: Tom: We've seen the architecture, so now page 5 explains how the parameters are split. The hidden group holds the QKV, attention output, MLP input, and MLP output matrices per block, and that's the only group Muon ever receives. The auxiliary group holds token and position embeddings, and the unembedding sits in its own readout group.

Jane: The AdamW baseline places everything under one AdamW instance, with a single learning rate and weight decay. That comparison is important because it isolates the effect of the split routing from the effect of the optimizer itself.

Lu: Page 5 also gives the Muon mechanics: a momentum buffer with beta 0 point 95, a Nesterov-style update, Frobenius normalization, then five quintic Newton–Schulz iterations to approximate the orthogonal factor. The normalization is the key property because it makes the step size insensitive to the gradient magnitude.

Meng: And the operational definitions are strict. Sustained 95 percent test accuracy means the first evaluation followed by five evaluations at or above 95 percent, so a configuration that touches the threshold and falls away doesn't count. Strict stability means no evaluation below 95 percent anywhere in the remaining hundreds of thousands of steps.

Lalam: Those strict criteria matter because the whole object of study is a trajectory that crosses the threshold multiple times. If you used first crossing, you would credit configurations that collapse immediately and miss the phenomenon entirely.

Tom: The selected configurations are interesting. Muon uses hidden learning rate 0 point 03, weight decay 0 point 1, and the auxiliary and readout groups at 1e-3 and 2 point 5e-4 with weight decay 1 point 0. AdamW baseline is lr 1e-3, wd 3 point 0, chosen as the fastest configuration that was stable in the initial sweep.

Jane: And there's a careful note about nondeterminism. On their accelerator backend, replay isn't bitwise deterministic, so the matched branches are constructed by branching from a single in-memory state within one process. That gives zero parameter difference by construction.

Lu: The seed replication adds another layer. For the five-seed comparison, they use CPU where the implementation is deterministic, and they verify the two arms agree to zero difference before the freeze. That makes the paired comparison airtight.

Meng: It's a good example of how to handle reproducibility in messy training setups. Instead of pretending the backend is deterministic, they build the causal comparisons around exact-state branching and use deterministic replay for the seed study.

Lalam: For eye research, this level of methodological care is what separates a suggestive pattern from a robust finding. The instability is measured against strict thresholds, the branches are exact, and the seed runs are paired.

Tom: And that groundwork pays off on page 6, where they run the sweeps and quantify the speed advantage.

Jane: Right, the next page has the AdamW and Muon sweeps with all nine Muon configurations grokking and none of them stable.

Page 6 of the Paper: Tom: Page 6 opens with the sweeps and how they chose baselines, which turns out to be delicate because the fastest AdamW configuration isn't usable. At lr 1e-2 with wd 1 point 0, AdamW reaches sustained generalization in 6,300 steps, then spends 918 evaluations below threshold and ends at 36 point 82 percent.

Jane: So they take as the baseline the fastest configuration that both groks and stays stable in the single-seed sweep: lr 1e-3, wd 3 point 0, at 8,200 steps. For Muon, no configuration is stable, so they just take the fastest, which is lr 0 point 03, wd 0 point 1 at 5,400 steps.

Lu: The speed comparison shows Muon reaches sustained generalization in 5,400 against 8,200 at this seed, a factor of 1 point 52. But over the whole sweep, successful AdamW configurations average 30,486 steps and Muon averages 13,011, a factor of 2 point 34 computed over AdamW's successes alone.

Meng: Across five seeds, the median advantage is 1 point 54, and Muon is faster on four of five. The interesting thing is AdamW's variance: its slowest seed takes nearly three times its fastest, while Muon's five seeds span just one hundred steps.

Lalam: That variance itself is a finding. AdamW has a wide spread of grokking times, Muon is tightly concentrated, which suggests the normalized orthogonalization controls the dynamics more tightly.

Tom: Table 6 varies modulus, training fraction, and width. Muon groks in all three variants, AdamW in two, and where both succeed, the ratios are 4 point 92 and 4 point 79 rather than 1 point 52. So the main condition is actually the narrowest margin they measure.

Jane: There's also a detail about step speed: Muon costs 1 point 75 times an AdamW step, so the depth-2 advantage in steps is 2 point 22 but only 1 point 28 in elapsed time. That's going to return later when they talk about depth.

Lu: The baseline selection story also sets up the instability section. At high learning rates AdamW becomes unstable, and Muon lives permanently in that regime. The moment you push for faster grokking, you inherit the collapse.

Meng: And the sweep results are stark: all nine Muon configurations reach sustained 95 percent, none is strictly stable, with minima as low as 0 point 78 percent and sub-threshold counts from 2 to 592. The count doesn't track either hidden hyperparameter monotonically.

Lalam: That non-monotonicity is important. If the instability were a tuning issue, you'd expect a gradient where smaller learning rates or different weight decay fix it. Instead, every point on the grid fails, which points to something structural.

Tom: And the next page digs into that instability, including the fact that AdamW isn't exempt and that severity tracks learning rate.

Jane: Right, page 7 has the full instability analysis, including the five-seed replication where neither selected setting is strictly stable.

Page 7 of the Paper: Tom: We've established the speed advantage, so page 7 pushes on instability. Four of AdamW's seven grokking configurations are also unstable, but severity tracks the learning rate: at 3e-4 and 1e-3, no grokking configuration falls below threshold at this seed; at 3e-3 it's mild, two and four evaluations; at 1e-2 it becomes severe, 258 and 918.

Jane: And the five-seed replication of the selected settings is the sharpest statement. Muon records between 137 and 321 sub-threshold evaluations on all five seeds, with minima from 16 point 27 percent to 76 point 05 percent. AdamW records one or two on four of five, with a minimum of 27 point 59 percent. So neither is strictly stable.

Lu: The paper says what separates the optimizers is severity, by two orders of magnitude, not whether the failure occurs. That's a strong claim because it means the stability of the AdamW baseline is a property of a single seed, not of the configuration.

Meng: The subtraction result also appears here: trained on (a - b) mod 113, the Muon run records 18 post-grokking evaluations below 95 percent with a minimum of 1 point 52 percent, while AdamW records none. That reproduces the asymmetry of the addition pair.

Lalam: Two further sweeps on the output head confirm the failure isn't an artifact of the readout's optimization. Varying readout learning rate over a factor of five changes the timing and depth of collapse but leaves every setting unstable. Varying readout weight decay, including zero, also leaves it unstable.

Tom: The zero weight decay case is worth pausing on. If decay were pulling the readout away, removing it should help, but the collapse remains. So the paper concludes that what pulls the readout is not what drives the failure.

Jane: At lr 1e-2, the replication is harsher than the single-seed sweep. Three of five seeds never reach sustained 95 percent, the two that do record 805 and 956 sub-threshold evaluations, and every one ends within a tenth of a point of chance. The single-seed figure of 36 point 82 percent was the best of five, not typical.

Lu: That's a cautionary tale about single-seed sweeps. The configuration that looked moderately bad in the sweep is actually catastrophic across seeds.

Meng: And this is all before the localization experiments. Page 8 sets up the diagnostic window, tracking the quiet steps before a collapse where the loss is at 1 point 5e-7 and the gradients are tiny.

Lalam: The instability is the motivation for the causal analysis. Once you know it's ubiquitous across configurations and seeds, you need to identify which parameters are responsible, and that's exactly what the matched branches do.

Tom: So the next page is the move from correlation to causation, with the exact-state branching and the elasticity regressions.

Jane: Yes, page 8 has Table 7 and Figure 2, following the 691 steps before a collapse.

Page 8 of the Paper: Tom: Page 8 is where the mechanism gets quantified. Over the 691 steps before a collapse, training loss sits at 1 point 5e-7 while the hidden gradient norm rises by a factor of 3 point 92 and the non-hidden gradient by 2 point 60. The hidden group's gradient-driven step changes by 0 point 97, so it's essentially flat.

Jane: The elasticity regression is the key: minus 0 point 026 for the hidden group, plus 1 point 51 for embeddings, plus 1 point 47 for readout. The hidden step doesn't track the gradient at all, while the AdamW groups track it more than proportionally. That's why the two separate.

Lu: There's a beautiful detail in the bottom panel of Figure 2. The gradient-driven component of the hidden step is 0 point 1499 and the weight decay component is 0 point 1475, opposing it. The net displacement is 0 point 0462, and that residual grows by 64 percent across the window while the step producing it stays flat.

Meng: So the hidden matrices aren't stationary; they're walking along a direction the loss doesn't constrain, and the readout isn't keeping pace. The training loss falls by a fifth over the same window, so the loss reports steady improvement right before the collapse.

Tom: Then the matched branches from step 44,000 to 46,000. The control collapses at 44,700, freezing hidden prevents it, freezing auxiliary prevents it. Within the auxiliary group, freezing token or position embeddings only delays the collapse, but freezing the unembedding prevents it outright.

Jane: And if you leave hidden matrices and the unembedding free while freezing the two embeddings, the failure still happens at step 45,290. That shows the two groups the mechanism names are together sufficient.

Lu: The paper frames it well: each freeze removes one ingredient. Fixing the readout means any departure raises the loss and produces a gradient opposing it. Fixing the hidden matrices means nothing traverses the unconstrained direction in the first place. With both free, nothing opposes the separation.

Meng: The branches also show the failure requires functional incompatibility, not just a fitted change of basis. Two branches from the same state each decode their own representation at 100 percent and the other's at chance, with readout matrices reaching cosine similarity minus 0 point 028.

Lalam: For the bigger picture, this is a clean causal design. Instead of correlating statistics with failure, they intervene on exact branches and show the necessity of both moving components.

Tom: The page also sets up the ablation of Muon's normalized orthogonalization, which comes on page 9 along with the terminal failures.

Jane: Right, the ablated runs never show recurrent collapse, but all six end in non-finite loss. That's a different failure mode, and it tells you the normalization path is actually what creates the survivable instability.

Conclusion: Tom: We've come to the end of the paper, and the conclusion draws the pieces together. The task chooses the family, the optimizer chooses the spread, and the readout interface is where the failure lives.

Jane: The cleanest summary is that a transformer can hold a circuit that solves the task perfectly and still answer incorrectly. Before grokking the circuit occupies one percent of the representation; after collapse its share has fallen from 84 percent to 20 percent. Between those states the model works.

Lu: And in both failure states the circuit is outvoted, not absent. The family produces a positive margin on every example, including every one the model answers wrongly, while the rest of the representation contributes a negative margin of nearly equal size.

Meng: Rescaling the family alone restores the model to 99 point 9 percent with no retraining. That's the most direct evidence that the code is intact inside a failing model.

Lalam: The broader lesson for eye training is that orthogonalizing optimizers like Muon are fast because they ignore the magnitude of the gradient, but that same property leaves them insensitive to a coordinate drift that the loss doesn't penalize. The solution they propose, freezing the inputs and readout after the circuit forms, works across five runs and five paired seeds.

Tom: And the future work is well specified. They want to fit the transformation between healthy and collapsed representations to test whether the drift is really a change of basis, and they want to separate orthogonalization from step scaling in the ablation.

Jane: They also predict that removing hidden weight decay should triple the net motion and bring collapse forward, which is a concrete empirical test. And the idea that a decoder fit from scratch on a collapsed representation should recover the task is a sharp way to check whether the information survives.

Lu: What stays with me is the warning about progress measures. Support and power are invariant to the exact change of basis that kills the function, so a spectral audit can report an intact circuit at the moment the model stops computing the task.

Meng: At the same time, the paper gives you the tools to catch it. The margin decomposition, the family filtering, and the cross-readout substitution all separate the circuit's integrity from its legibility.

Lalam: For the field, this reframes grokking as a competition on amplitude rather than a binary switch. The same quantity that buries the circuit before generalization is the one that buries it after collapse, and that unification is a genuinely useful lens.

Tom: And that's where we'll leave it. The paper is a reminder that a learned representation only matters insofar as something can read it, and the readout can drift even when the training set is perfectly solved.

Jane: We've covered the speed, the instability, the Fourier decomposition, and the intervention that fixes it. Next up on the channel we'll look at a different paper, but this one gives us a lot to compare against.

Tom: Thanks for listening, and we'll see you at the next discussion.