Finding Usable Weight Mechanisms with Tiled SVD

arXiv:2608.06969 · cs.AI · Submitted 2026-08-07 · Read on arXiv

Listen

Transcript

Introduction to the show: ident: Paper Radio. Generated commentary on the latest Artificial Intelligence papers.

Tom: Next we'll be talking about the paper "Finding Usable Weight Mechanisms with Tiled SVD".

Jane: The paper was written by Ash Manvi and Samreena Tajreen from Aquin Labs.

Tom: Stay tuned as we take you through the paper and discuss its implications.

Paper summary: Tom: We've got a fresh one from Aquin Labs — Ash Manvi and Samreena Tajreen — and it's about dragging mechanistic interpretability out of the proxy world. Instead of training sparse autoencoders to name features, they go straight into the weight matrices.

Jane: And I love the phrase they use — "mechanism mounts." Each one is a triple: a trigger, a write, and a strength. The identity is the weight rule itself, not a verbal label from a dictionary.

Lu: So this is SVD-based. Column-tiled SVD, actually — they slice each weight matrix into tiles and factor each tile separately. Whole-matrix SVD spreads everything out; tiling keeps local structure.

Meng: Right, and then they test it hard. They don't just check whether a tile reconstructs itself — that metric is rigged. They measure full-write energy lift on real forwards.

Jane: On Gemma-2-2B, all seven linear maps per layer. Every one of the 182 site-layers passes. Residual writes get the full treatment — chunking, coverage, and a steer-into-the-residual-stream causal check.

Tom: 52 of 52 residual site-layers with A/B/C, 130 of 130 others with A/B. The headline number is genuinely clean: 182 out of 182.

Lalam: And why does this matter? Because interpretability research mostly describes a model through an external codebook. This paper says: look at the weights themselves. The question "what does this direction mean?" gets answered by the weight rule that writes it.

Lu: They're careful not to overclaim, though. No human-readable concept names, and no claim to replace sparse autoencoders for concept discovery.

Jane: They're honest that novelty is thinner — singular vectors of transformers have been looked at before. The wedge is fair chunking, and a measurement stack you can reuse.

Meng: So the deliverable is less "we found features" and more "here's a pre-registered suite that judges mounts honestly." They release code, corpus builder, tests.

Tom: And we're going to walk through it page by page, because the interesting stuff is in the details — how they build the mounts, and how they dodge the tautological metrics.

Jane: Next up: the first page, where they set up the problem. Let's see how they frame the whole thing.

Page 1: Jane: Where we left off: the paper promises a measurement suite that judges weight mechanisms directly, not through a proxy dictionary. Page one is where they motivate that move.

Tom: They open with the standard tool — sparse autoencoders trained on activations, labeled from max-activating text. Those atlases are useful, but the identity lives in the learned dictionary, not in the network weights.

Lu: Right, and that's the crack they're prying at. If interpretation lives in a separately trained codebook, then "what this direction means" gets answered in proxy space. The weight rule that actually writes into the residual stream stays implicit.

Meng: Then they mention prior work — singular vectors of MLP and attention matrices, read through the unembedding, form interpretable token clusters. So SVD as a lens isn't new.

Jane: Exactly. They cite Millidge and Black on SVD being interpretable, then Xue and Andrzejak framing singular modes as detector-effector units. Their point: those used SVD as a lens or circuit primitive, not as a fair test of which chunking of W yields usable on-distribution mechanisms.

Tom: And there's the keyword — "chunking." You have a big weight matrix. How do you slice it so the pieces are actually usable mechanisms?

Lu: Because the answer isn't obvious. A global SVD gives you abstract directions, but a transformer writes through local column blocks. Tiling respects that locality.

Meng: I like that they name the object early — a mount is a triple: trigger, write, strength. The trigger lives in the input columns, the write lands in the output space, and σ says how strong the coupling is.

Jane: And identity is the weight rule, not a phrase like "the king direction." That's a philosophical commitment, honestly. Meaning is in the mechanism.

Tom: They also preview the evaluation — full-write energy lift rather than tile-local lift, because tile-local metrics can be gamed. That's going to be a recurring theme.

Lu: The page ends before the method details, but the thesis is already sharp: don't label the dictionary; find the mounts in the weights and test whether they actually write.

Jane: Which sets up page two beautifully — they introduce the model, the seven linear maps, and which ones get the full causal treatment.

Page 2: Tom: Picking up: we've got the thesis — mounts in the weights, not labels in a proxy. Page two makes it concrete: the model, the sites, the exact linear maps.

Jane: Gemma-2-2B, 26 layers. Each layer exposes seven linear maps. Two of them are residual writes — mlp.down and attn.o. Those get the full suite, A/B/C.

Lu: And the other five — mlp.gate, mlp.up, attn.q, attn.k, attn.v — get A/B only. The reasoning is clean: their outputs aren't residual writes, so aligning to the unembedding isn't the right causal metric.

Meng: The figure on this page marks them in orange and green — ABOnlyLinear versus ResidualWriteLinear. Nice touch: the residual writes are the ones that directly change the residual stream.

Tom: For the corpus, they use WikiText-2 raw train. They collect all tokens — that's 86,109 — then subsample to 16,384 with seed 0 to keep memory in check. Simple, reproducible, pre-registered.

Jane: What I appreciate is the honesty about defaults — site-aware tile sizes. Residual and MLP maps get tile width 512, attn.k and attn.v get 256, attn.q gets 128 with a 64 fallback. Those aren't magic numbers; they're tuned per site.

Lu: It smells like a lot of engineering went into making the test fair. They're not picking one tile size and hoping it works everywhere.

Meng: And the mention of "effective path mounts" already peeks through the text — for mlp.up and attn.v, the raw module weights won't cut it. The map used on-distribution isn't the raw matrix.

Jane: Right — that's coming in a later page, but it's the most interesting design decision in the paper, honestly. They bake the gate activation into the up-projection.

Tom: So page two sets the stage: fixed model, fixed corpus, clear split between residual writes and everything else. The machinery comes next.

Lu: And it's a lot of machinery — tile-SVD mounts, trigger coefficients, the energy lift. That's page three.

Page 3: Jane: So we're on page three, and this is where the math gets real. Column-tiled SVD: slice W's input columns into tiles, factor each tile with SVD.

Tom: Each tile yields modes — the left singular vector becomes the write direction u, the right singular vector is the trigger v, and the singular value σ is the strength. A mount is just that triple with site and layer metadata.

Lu: The clever bit is the trigger coefficient: a t,j = x t · v j. You run a real forward, you get site inputs x, and the trigger coefficient tells you how strongly this mount fires on that token.

Meng: Then the tile write is the corresponding column block transposed. So the trigger coefficient times the write direction reconstructs the tile's contribution. It's a lightweight, linear story about how the weight behaves.

Jane: And they're upfront that SVD identity holds almost always by construction — correlation above 0.99, slope error below 0.05. That's a sanity check, not a proof. It can't separate usable mounts from unused ones.

Tom: Then comes the crucial turn: tile-local energy lift is a trap. They show it favors one-column tiles tautologically — the lift sits around 0.999 for column sampling.

Lu: That's the negative result in miniature. If you measure inside the tile, you'll always win. So they switch to full-write energy lift — measuring against the site's actual write tensor ∆h on real forwards.

Meng: Random directions seeded with a fixed seed — 10007 plus j times 997 — give you the baseline. The mount has to beat random directions in the full residual write, not in its own little tile.

Jane: Four constructions compete under a matched mount budget: per-tile SVD, whole-matrix SVD, high-norm column sampling, and random. That's the arena.

Tom: And it's a fair arena, because the budget is matched. Same number of mounts, same test, and the tile SVD has to earn its lift in the full write.

Lu: The hook for next page: how do they know the coverage isn't just redundancy? That's the saturation analysis — and the causal steer check.

Page 4: Tom: We've got the mounts, we've got the full-write lift. Page four asks: does coverage actually saturate, and can you steer with these things?

Jane: Coverage saturation — they measure what fraction of the weight's Frobenius norm the per-tile reconstruction keeps, then build a sparse dictionary of mount directions. Select top 8 mounts per token, reconstruct by least squares.

Lu: The key metric is coverage lift versus a random dictionary of matched size. Saturation means the lift peaks early — by one or two modes per tile — and doesn't collapse afterward.

Meng: The floors are site-dependent: 0.25 for residual writes, 0.15 for other maps, 0.08 for the effective up and v paths. And the early sweep window has to sit within 0.08 of the peak. If extra modes just add redundancy, you've saturated.

Tom: Then Experiment C — the causal steer. For residual writes only, they read the final-logit geometry of a write direction through the unembedding: approximate final RMSNorm, then t = Wlm u.

Jane: Steering adds α=2 on the post-attention or post-FF RMSNorm module — not on the projection alone. They average last-token logit changes over eight texts and compute Spearman ρ between Δlogits and the unembed readout.

Lu: The pass bar is modest — ρ ≥ 0.05 or top-20 Jaccard ≥ 0.05 — but it's required only for residual writes in layer 6 or above. Early layers report C but don't fail when alignment is weak.

Meng: That depth-conditioned rule is a design choice they flag honestly. The curve justifies it; they'll show the numbers.

Tom: And then the effective-path mounts — my favorite part. Raw mlp.up and attn.v weights fail the residual-shaped tests because the on-distribution map isn't the raw matrix.

Jane: For mlp.up they build W* = diag(ḡ)W up, using corpus-mean gate activations. For attn.v, they do a ridge least-squares fit from x to mixed-v. You mount the path that actually gets used.

Lu: It's a subtle point. The raw v-projection writes something, but by the time it goes through attention's mixing and the output projection, the effective write is something else. So you extract from the composed map.

Meng: The pass criteria table at the bottom ties it together — A1 through A4, B1, C1 — with explicit margins. A1 needs tile lift above random plus 0.005, A2 within 0.002 of column sampling, and so on.

Tom: Next page, they stop defining the suite and actually run it. We get the first real numbers from Experiment A.

Jane: And I'm curious whether the chunking claim survives contact with the full model.

Page 5: Tom: Page five, and the suite goes live. They build the WikiText-2 corpus, run the full thing: all layers, all sites, on CUDA. 16,384 tokens subsampled from 86,109 collected.

Jane: The headline: all seven sites × 26 layers pass — 182 of 182. Residual A/B/C are 52 of 52, others A/B are 130 of 130. But the per-experiment numbers are where it gets interesting.

Lu: Experiment A is the chunking test. Tile SVD beats whole-matrix SVD, column sampling, and random at every depth — except mlp.down layer 25, where tile and whole are basically tied at 0.013.

Meng: That's the honest outlier. They don't hide it — the table shows layer 25 mlp.down tile at 0.013 and whole at 0.013. A4 still passes on ratio, but the margin is dead flat.

Jane: Meanwhile layer 18 attn.o is the big winner — tile at 0.383 versus whole at 0.102. Column sampling barely registers, 0.011. Random is essentially zero everywhere.

Tom: What's the intuition? A single global SVD spreads local structure across directions that don't align with how the network writes. Tiling respects the column blocks the attention head or MLP neuron actually uses.

Lu: And they're careful that column sampling — which wins the tile-local metric — loses badly on full-write lift. That's the tautology exposed: locally perfect, globally useless.

Meng: The "fewer mounts" detail is nice, too. attn.o has input dimension 2048, so it gets fewer mounts than mlp.down, and it still wins A everywhere. The tiling advantage isn't just about mount count.

Jane: So chunking works for residual writes on real forwards. The next question is coverage — does the write energy saturate quickly, or do you need many modes? That's Experiment B, on page six.

Tom: And given the table on this page, I'd bet saturation kicks in early.

Page 6: Jane: Page six, Experiment B: coverage versus modes per tile. The sweep goes k = 1, 2, 4, 8, 16. Does adding modes keep buying write coverage?

Tom: The answer is no — it saturates almost immediately. Residual writes show high coverage lift at m=1 or m=2, then flat. Extra modes mostly add redundancy.

Lu: Look at layer 0 mlp.down: m=1 gives 0.225, m=2 jumps to 0.279, m=4 stays 0.279, m=8 goes to 0.302, m=16 drops slightly to 0.298. You've basically extracted everything by the second mode.

Meng: Layer 6 attn.o is even more dramatic — 0.779 at m=1, peaks at 0.793 at m=2, then drifts down. The early window rule catches exactly this: the peak is in the first couple modes, and it doesn't collapse.

Jane: So all 52 residual site-layers pass B1, and the saturation rule does real work — it rejects a curve that only climbs with more modes, because that would mean the tiling isn't capturing the write structure.

Tom: Then Experiment C — the depth curve. They inject after post-norm RMSNorm, steer with α=2, and measure Spearman against the unembed readout of the write direction.

Lu: And the curve is beautiful. On mlp.down, early layers sit near zero — roughly −0.03 to 0.07, waived — then onset layers 6-8 climb to 0.52, mid layers 9-12 around 0.15 to 0.54, and late layers 18-24 reach 0.69. Final layer, 0.91.

Meng: attn.o is shallower but still rises — mid layers around 0.53 to 0.55, final layer 0.75. The key point: mid-depth attn.o no longer fails C1. With post-norm injection, the alignment is genuine.

Jane: And the waiver for early layers is motivated by that curve — ρ hovers near zero until roughly layer 6, then takes off. It's not a free pass; it's reading the data.

Tom: The aggregate verdict on page seven pulls all this together: 182 of 182, with the tier structure spelled out.

Lu: I want to see how they frame what they did and didn't contribute — the discussion on novelty.

Page 7: Tom: Page seven lays out the aggregate verdict — the tier table. Residual A/B/C: mlp.down 26 of 26, attn.o 26 of 26. Other A/B only: 130 of 130. Total 182 of 182 GO.

Jane: And then the discussion starts, and I appreciate the restraint. They say the measurement stack is the durable part of the work — not the discovery of features, but the honest way of testing them.

Lu: They're blunt about novelty being thinner. Singular vectors of transformer weights, detector-effector units, unembed readouts — all exist. They cite Millidge and Black, Xue and Andrzejak, the circuit work from last year.

Meng: The wedge is fair chunking, a negative result about tile-local metrics, coverage saturation, and the depth-conditioned causal check. Those four pieces packaged as a reproducible suite.

Jane: And they explicitly don't claim human-readable concept names, and don't claim to replace sparse autoencoders for concept discovery. That's a clean boundary — this is about mechanisms, not concepts.

Tom: The scope stays sharp, too. Experiment C applies only where u is a residual direction. Raw mlp.up and attn.v weights fail residual-shaped A/B by design — the supported object is the effective path.

Lu: Honest scope is rare in this literature. They're telling you exactly which claims are load-bearing and which sites need special handling.

Meng: All numbers are for Gemma-2-2B on a WikiText-2 subsample — single model family, single corpus. That's not a weakness they hide; it's a limitation they name.

Jane: Which brings us to page eight, where the limitations get their full airing. I've got a feeling the hardest part is the meaning question — energy lift is not human meaning.

Tom: And that's the gap between "we can steer it" and "we know what it means."

Page 8: Jane: Page eight is the limitations section, and they don't soften anything. Single model family and size — Gemma-2-2B only. Experiment C only for residual writes. The WikiText-2 subsample may bias which mounts look strong.

Tom: Then the big one: energy lift is not human meaning. Mounts carry no semantic labels. You can have a mount that writes hard, but the paper won't tell you what it "means."

Lu: And the steering protocol is narrow — short texts, fixed α=2, last-token logits after post-sublayer RMSNorm. That's a thin slice of the behavioral space, and they know it.

Meng: The C1 waiver for early layers is flagged as a design choice. They say it's principled from the depth curve, but early ρ should always be reported. Transparency over convenience.

Jane: mlp.down layer 25 is called out as marginal on A4 — tile equals whole — and attn.o runs with fewer mounts than mlp.down. The table on page five already showed us that.

Tom: Raw mlp.up and attn.v module weights fail residual-shaped A/B — that's a strong statement. If you don't build the effective path, you don't get a passing mount.

Lu: I think that's actually a gift to the field. Negative results about naive approaches save people months of dead-end work.

Meng: So the limitations aren't disclaimers for show. They'd rather shrink the claim than inflate it.

Jane: And the conclusion on page nine ties it together — they extract tile-SVD mounts, score them with full-write lift, coverage saturation, and the depth-conditioned steer check. 182 of 182, with effective-path mounts for up and v.

Tom: The release includes library code, corpus builder, experiment entrypoint, unit tests. Reproducibility is the point.

Lu: Let's wrap this up properly — how would we tell someone why this paper matters?

Conclusion: Tom: So we land the plane. The paper gives us a repeatable way to pull mechanisms out of raw weights — tile-SVD mounts — and a test suite that doesn't fool itself.

Jane: Tile beats whole-matrix SVD, column sampling, and random under a matched budget. Coverage saturates by one or two modes. Post-norm steering aligns with the unembed more and more as depth increases. 182 of 182 site-layers pass.

Lu: The biggest thing they're handing the field is a measuring stick that refuses the easy win. Tile-local metrics are out; full-write energy lift is in.

Meng: And the effective-path move — mounting mlp.up through the mean gate, attn.v through ridge least-squares — that's the part I'll steal. Raw weights lie about what the network does.

Jane: They didn't discover new features with human labels. They built the scaffold for finding mechanisms that actually write, and they shipped the code.

Tom: It's a benchmark, a negative result, and a philosophical stance in one package: identity is the weight rule, not a phrase.

Lu: The limits are real — one model, one corpus, no semantic labels. But the protocol generalizes. Run it on another family, another size, and you learn something.

Jane: Goodbye to this paper, then — and a nice handoff, because the next one on our list picks at the same question from the distribution side.

Tom: Exactly — if mounts are the mechanisms, the next question is how they organize across a model's lifetime. Stick around.

Ash Manvi, Samreena Tajreen

Aquin Labs

cs.AI

Submitted: 2026-08-07

Updated: 2026-08-10

Comments: 8 pages, 6 figures. Finding usable weight mechanisms via column-tiled SVD; pre-registered A/B/C eval on Gemma-2-2B (WikiText-2)

License: http://creativecommons.org/licenses/by/4.0/

Importance score: 95/100

Key concepts

Mechanism mounts: A mount is a triple of trigger, write, and strength. It describes a weight rule: an input trigger direction, an output write direction, and a coupling strength. Its identity is the weight rule itself, not a verbal label from a dictionary.
Column-tiled SVD: Instead of applying SVD to an entire weight matrix, the matrix is sliced into column tiles and each tile is factored separately. This preserves local structure, because whole-matrix SVD spreads local structure across directions that may not match how the network actually writes.
Full-write energy lift: A measurement that checks whether a mount improves reconstruction of a site's actual write tensor on real forwards, compared with random directions. It avoids tile-local energy lift, which can be tautologically won by any single-column tile and does not prove global usefulness.
Effective-path mounts: For mlp.up and attn.v, the raw module weights do not match the map used on-distribution. The paper extracts from the composed or gated path instead, such as using corpus-mean gate activations for mlp.up or a ridge fit for attn.v, so the mount reflects the effectively used write path.

Terminology

Alignment — 7× in this paper · explained in 1 episode
Interpretability — 7× in this paper · explained in 15 episodes
Transformer — 6× in this paper · explained in 2 episodes
Mechanistic interpretability — 5× in this paper · explained in 2 episodes
Steering — 4× in this paper · explained in 2 episodes
Core — 2× in this paper · explained in 2 episodes
Redundancy — 2× in this paper · explained in 2 episodes
Adam — 1× in this paper · explained in 2 episodes

Summary

The paper addresses a core limitation of dominant mechanistic interpretability approaches: "The dominant approach to mechanistic interpretability trains proxy dictionaries such as sparse autoencoders and labels features from max-activating text. The best such atlases identify concepts, but that identity lives in the learned dictionary rather than in the network weights themselves. The authors note that Aligning interpretation to a separately trained codebook, rather than to a particular weight matrix inside the network, means that 'what this direction means' is answered in the proxy space. The weight rule that actually writes into the residual stream is left implicit. Prior work has shown singular vectors of MLP and attention matrices often form interpretable token clusters and can be edited, and has framed singular modes as detector-effector units, but SVD is used as a lens or circuit primitive, not as a fair test of which chunking of W yields usable on-distribution mechanisms."

Model and sites. The evaluation uses "google/gemma-2-2b (26 layers, indices 0,..., 25). Each layer exposes seven linear maps. Two are residual writes: mlp.down (W ∈ R 2304×9216) and attn.o (W ∈ R 2304×2048). These receive the full suite of chunking, coverage, and causal checks (A/B/C), with Experiment C injecting after Gemma-2 post-sublayer RMSNorm so the steered direction lands in the residual stream. The remaining five maps (mlp.gate, mlp.up, attn.q, attn.k, attn.v) receive A/B only because their outputs are not residual writes, so unembed alignment is not the right causal metric. Corpus text comes from WikiText-2 (raw train) with a subsample of 16,384 tokens with seed 0."

Tile-SVD mounts. The core proposal is extracting mechanism mounts directly from linear sites by column-tiled SVD: each mount is a triple (v, u, σ) read as trigger, write, and strength; identity is the weight rule itself. Default tile widths are site-aware: T=512 for residual and MLP maps, T=256 for attn.k/attn.v, and T=128 for attn.q (with a 64 fallback on hard layers). Experiments A and C use k=2 modes per tile; Experiment B sweeps k ∈ 1, 2, 4, 8, 16. The method partitions the input columns of W into tiles [st, et) and factor each tile Bt = W:,st:et = Ut Σt Vt⊤, giving u = Ut[:,i]/‖Ut[:,i]‖2, σ = Σt[i], v = (Vt⊤)[i,:]. Identity is this weight rule, not a verbal label. Four constructions are compared under a matched mount budget: per-tile SVD, whole-matrix SVD, column sampling (high-norm columns as u; one-column tiles), and a random baseline.

Full-write energy lift. Trigger coefficients are computed with real forward site inputs: at,j = xt, sj:ej · vj, and tile writes use the corresponding column block: ∆htile = x:,s:e B⊤. A key methodological contribution is rejecting tile-local energy lift, which favors one-column tiles tautologically (Ltile ≈ 1 for column sampling). Instead, pass criteria use full-write energy lift against the site's write tensor ∆h: Lfull(u) = E t[(∆ht·u)2/(‖∆ht‖22+ε)] − E t[(∆ht·ũ)2/(‖∆ht‖22+ε)], with random directions ũ seeded by 10007 + j·997. SVD identity (sanity, not proof) requires correlation of a with ∆htile·u above 0.99 and relative slope error below 0.05... it cannot separate usable mounts from unused ones.

Coverage saturation. Weight coverage is the fraction of ‖W‖2 F kept by per-tile rank-k reconstructions. Sparse write coverage builds a dictionary of unit mount directions, selects the top-kactive (default 8) mounts per token, reconstructs ∆h by least squares, and reports explained energy, with coverage lift being the gap versus a random dictionary of matched size. B1 requires "the lift peak to meet a site-dependent floor (0.25 residual, 0.15 other maps, 0.08 effective up/v paths), the early sweep window to lie within 0.08 of the peak... and the final point not to collapse more than 0.10 below the peak."

Causal steer versus unembed. For residual writes only, "we read the final-logit geometry of a write direction u by an unembed lens: approximate final RMSNorm, then t = Wlm u. Steering adds αu with α=2 on the post-attention or post-FF RMSNorm module... averages last-token ∆logits over up to eight texts, and reports Spearman ρ(∆, t) and top-20 Jaccard. C1 passes if ρ ≥ 0.05 or J20 ≥ 0.05, and is required only when the site is a residual write and layer l ≥ 6. Early residual layers still report C; they do not fail when alignment is weak."

Effective-path mounts. Raw mlp.up and attn.v module weights fail residual-shaped A/B: the map that is used on-distribution is not the raw matrix. For mlp.up, W⋆ = diag(ḡ) Wup with corpus-mean gate activations ḡ (fallbacks: gate-mixture tile SVD; compose-through-down W↓ diag(ḡ)Wup). For attn.v, the method uses ridge least-squares x → mixed-v (fallback: W† = Wo expandGQA(Wv)).

Pass criteria. A site-layer passes when all applicable checks pass: A1 (Lfull(tile) > Lfull(rand) + m, with m=0.005 or 0.002 on effective paths); A2 (Lfull(tile) ≥ Lfull(cols) − 0.002); A3 (SVD-identity fraction ≥ 0.9 or corr(id, lift) < 0.3); A4 (tile full-write ≥ whole within site slack / ratio floor); B1 (coverage saturation); C1 (steer ρ ≥ 0.05 or J20 ≥ 0.05, required iff residual write and l ≥ 6).

The experiments were run with python scripts/build corpus.py and python scripts/run paper experiments.py with defaults of site-aware tile sizes, k=2 for A/C, modes sweep 1, 2, 4, 8, 16 for B, nsteer=8, α=2, and 16,384 tokens subsampled from 86,109 collected.

Reported run: All seven sites × 26 layers pass: 182/182 (residual A/B/C 52/52; other A/B 130/130).

Experiment A (chunking). On both residual-write sites, tile full-write lift exceeds whole-matrix SVD, column sampling, and random at every depth except mlp.down layer 25, where tile ≈ whole and A4 still passes on ratio. Example values: at layer 0, mlp.down tile lift is 0.126 vs whole 0.012, cols 0.006, rand ≈0; attn.o tile 0.192 vs whole 0.080; at layer 25, mlp.down tile 0.013 vs whole 0.013, attn.o tile 0.146 vs whole 0.055. The paper concludes: Local column structure in W is not well summarized by a single global SVD for on-distribution write energy. Tiling recovers higher-energy write directions under a fixed mount count on both residual writes.

Experiment B (coverage versus modes). On residual writes, coverage lift is already high at m=1 or m=2 and then flat: B1 passes all 52 residual site-layers under the early-saturation rule. Extra modes mostly add redundancy. For example, layer 0 mlp.down coverage lift rises from 0.225 (m=1) to 0.279 (m=2), then to 0.302 (m=8) and 0.298 (m=16); layer 0 attn.o goes 0.508 → 0.617 → 0.633 → 0.635.

Experiment C (causal depth). "With post-norm residual injection, mean Spearman versus unembed(u) rises with depth on both residual sites. Mid-depth attn.o no longer fails C1. Final layers reach ρ ≈ 0.91 (mlp.down) and ρ ≈ 0.75 (attn.o). The C1 waiver for l < 6 remains motivated by this curve." Early layers (0–5, waived) show ρ ≈ −0.03 to 0.07 for mlp.down and ≈ 0.00 to 0.53 for attn.o; onset 6–8: ≈ 0.07–0.52 and ≈ 0.29–0.51; final layer 25: ≈ 0.91 and ≈ 0.75.

Aggregate verdict: Residual A/B/C 52/52 (mlp.down 26/26, attn.o 26/26); other A/B only 130/130 (mlp.gate/attn.q/attn.k 26/26 each; mlp.up via mean-gate/compose-down 26/26; attn.v via lstsq mixed-v 26/26). Total: 182/182 GO.

"The measurement stack is the durable part of this work. Full-write energy lift separates tile SVD from whole-matrix SVD, column sampling, and random under a matched budget, without rewarding the one-column tautology that inflates tile-local lift. Coverage saturates early on residual writes. With post-norm injection, steer versus unembed alignment is a clear depth curve on both mlp.down and attn.o, and the same judge accepts all seven linear maps once mlp.up and attn.v are mounted from their effective write maps. That is an honest multi-site protocol, not an MLP-only demo."

The authors are candid about novelty: "Novelty is thinner. Singular vectors of transformer weights, detector-effector units, and unembed readouts already exist. We do not claim human-readable concept names, and we do not claim to replace sparse autoencoders for concept discovery. The wedge is fair chunking, a negative result about tile-local metrics, coverage saturation, and a depth-conditioned causal check packaged as a reproducible suite. Scope is sharp: Experiment C applies only where u is a residual direction. Raw mlp.up / attn.v module weights fail residual-shaped A/B by design; the supported object is the effective path. All numbers are for Gemma-2-2B on a WikiText-2 subsample."

The study "uses a single model family and size (Gemma-2-2B). Experiment C applies only to residual-write sites; gate, up, q, k, and v are judged on A/B alone. The WikiText-2 subsample (16,384 of 86,109 tokens) may bias which mounts look strong. Energy lift is not human meaning: mounts carry no semantic labels. Steering uses short texts, fixed α=2, and last-token logits after post-sublayer RMSNorm. The C1 waiver for l < 6 is principled from the depth curve but remains a design choice; early ρ should always be reported. Raw mlp.up and attn.v module weights fail residual-shaped A/B and require effective-path mounts. mlp.down layer 25 is marginal on A4 (tile ≈ whole) yet still passes; attn.o runs with fewer mounts than mlp.down."

"We extract tile-SVD mechanism mounts (v, u, σ) from linear sites of Gemma-2-2B and score them with full-write energy lift, coverage saturation, and a depth-conditioned steer check against the final unembedding on residual writes. Under a matched mount budget, tiling beats whole-matrix SVD, column sampling, and random; write coverage saturates by one to two modes per tile; and post-norm residual steers agree with unembed(u) increasingly with depth. Across all seven linear maps and 26 layers the suite passes 182/182 site-layers once mlp.up and attn.v use effective-path mounts. We release the library, corpus builder, experiment entrypoint, and unit tests, with mount identity defined as the weight rule itself."

The paper releases library code, the corpus builder, the experiment entrypoint, and unit tests, with the judge implemented in src/atlas/mount/paper eval.py.

Improvements for AI systems

Improvements derived from the paper:

Extract interpretable weight-local mechanisms directly from linear maps.

Instead of relying on proxy sparse-autoencoder dictionaries, apply column-tiled SVD to each linear site (mlp.down, attn.o, mlp.gate, mlp.up, attn.q/k/v) and treat each triple (v, u, σ) as the actual mechanism: v is the input trigger pattern, u is the write direction into the residual stream, and σ is the strength. The improved system can localize behavior to concrete weight matrices rather than to a separately trained codebook, making edits and audits more direct.

Use full-write energy lift instead of tile-local energy lift to avoid tautological scoring.

Score candidate mount directions by comparing the energy of the full site write tensor projected onto the candidate versus random directions, not by tile-local reconstruction energy (which trivially favors one-column tiles). The improved system can fairly rank mechanisms across different chunkings, identify genuinely high-energy write directions, and avoid false positives from local SVD artifacts.

Automatically choose non-trivial tile chunking and show it outperforms whole-matrix SVD, column sampling, and random baselines.

Use site-aware tile widths (e.g., 512 for residual and MLP maps, 128–256 for attention maps) and a matched mount budget. The improved system can recover higher on-distribution write energy with the same number of mechanisms, enabling more compact and more useful weight decompositions.

Detect redundancy via coverage saturation.

Build a dictionary of unit mount directions per tile, select top-k mounts per token, reconstruct the write tensor by least squares, and measure explained energy versus a random dictionary. The improved system can determine how many modes per tile are actually needed, stop adding redundant mechanisms beyond 1–2 modes on residual writes, and allocate interpretability or editing capacity more efficiently.

Mount the effective write path, not the raw matrix, for non-residual sites.

For mlp.up, use diag(ḡ) W up with mean gate activations (or compose through the down-projection); for attn.v, use ridge least-squares from input to mixed value output (or W o expandGQA(W v)). The improved system can correctly judge and edit sites whose raw weights are not directly residual-shaped, avoiding the false failure modes of raw matrix analysis.

Use a depth-conditioned causal steering check against the final unembedding.

For residual writes, inject the steering direction after Gemma-2 post-sublayer RMSNorm, measure Spearman correlation and top-20 Jaccard between final-logit deltas and unembed(u), and require alignment only at layer ≥ 6. The improved system can validate that extracted directions genuinely steer model output in the predicted direction, with clear depth scaling, and can automatically waive early-layer checks where alignment is not yet expected.

Deliver a reproducible, multi-site GO/NO-GO protocol.

Run the full suite across all seven linear maps and all 26 layers, requiring: A1–A4 for energy-lift competition, SVD identity, and whole-matrix comparison; B1 for coverage saturation; C1 for causal steering on residual sites with l ≥ 6. The improved system can report per-site-layer pass/fail status, identify marginal cases (e.g., mlp.down layer 25), and provide a trustworthy audit signal before any weight editing is attempted.

What the improved AI system can do:

Explain why a model produces a given behavior by pointing to specific tile-SVD (v, u, σ) triples inside specific layer matrices, rather than to abstract SAE features.
Steer model outputs with interpretable residual directions: add αu after post-sublayer RMSNorm and observe logit changes matching W lm u, with reliability improving in deeper layers.
Edit or ablate individual mechanisms with a fair, calibrated importance score based on full-write energy, not proxy-space activation.
Audit any transformer linear site (including gate, up, q, k, v) using the effective-path mounts, catching weights that would otherwise be misjudged.
Avoid over-parameterized interpretability dictionaries by detecting early coverage saturation and choosing only the needed modes per tile.
Reproduce the entire evaluation on new models/corpora with the released library, corpus builder, experiment entrypoint, and unit tests, yielding 182/182 site-layer verdicts or clearly identifying where the new model diverges.

Abstract

The dominant approach to mechanistic interpretability trains proxy dictionaries such as sparse autoencoders and labels features from max-activating text. The best such atlases identify con- cepts, but that identity lives in the learned dictionary rather than in the network weights them- selves. We propose extracting mechanism mounts directly from linear sites by column-tiled SVD: each mount is a triple (v,u, sigma) read as trigger, write, and strength. Identity is the weight rule. We evaluate mounts with a pre-registered suite judged on full-write energy lift rather than tile-local lift. On Gemma-2-2B with WikiText-2 (16,384-token subsample), all seven linear maps are scored: residual writes (mlp.down, attn.o) receive full A/B/C with steer after post-sublayer RMSNorm and pass 52/52 site-layers; other maps receive A/B only (mlp.gate/attn.q/attn.k/effective mlp.up/attn.v 26/26 each). Aggregate: 182/182 GO. We release library code, the corpus builder, the experiment entrypoint, and unit tests.