Beyond Adjacent Layers: Graph-Guided Layer Fusion for Compressing Large Language Models

Introduction to the show: ident: AI Radio. Generated commentary on the latest Artificial Intelligence papers.

Tom: Today's paper: "Beyond Adjacent Layers".

Jane: The gist The FCM framework exploits full pairwise Centered Kernel Alignment similarity matrices to identify non-adjacent functional communities within large language models,

Tom: First, who's behind it and why it matters.

Paper summary: Tom: So we're looking at this paper, "Beyond Adjacent Layers: Graph-Guided Layer Fusion for Compressing Large Language Models." Basically, they are testing the idea that just looking at next-door layers isn't enough for compressing these big models.

Jane: That makes sense. The core thesis here is that functional redundancy in these large language models isn't just local to adjacent layers; it’s actually spread out across non-adjacent ones too, and this paper shows how to find that structure.

Lu: What they claim is that when you compute full pairwise Centered Kernel Alignment similarity matrices for thirteen different LLMs from six different architectures, non-adjacent layers show a much higher degree of similarity than adjacent ones, with three point six to fifteen point nine times more high-similarity pairs than adjacent ones <ref:2610.15159#pg1>.

Meng: So they’re saying the assumption that functional redundancy is local is wrong, which means existing methods that only look at neighbors are missing a lot of potential compression opportunities.

Lalam: Exactly. The paper calls this structure a "structure that prior similarity-based layer merging has not systematically exploited" <ref:2610.15159#pg1>. They propose Fusion via Community Mining, or FCM, to handle this non-local relationship.

Tom: FCM builds the whole L by L similarity graph and then uses spectral community detection to find these functional clusters of layers that are equivalent, and then they keep the most important layer in each one.

Jane: That sounds like a pretty sophisticated way to group things, but what makes this approach different from just looking at simple layer importance scores?

Lu: They use Block Influence, or BI

Men et al., two thousand twenty-five: , which measures the cosine distance between each layer's input and output hidden states, quantifying how much a layer actually transforms its input <ref:2610.15159#pg4>. This score helps them select the most important member within each community.

Meng: I wonder what that means for deployment. If we are compressing to K layers, which one do they keep from that community? That choice seems critical for maintaining performance at scale.

Lalam: They select the layer with the highest Block Influence score in each detected community, which is called the most important layer in that cluster <ref:2610.15159#pg4>. This is then fed into a LoRA recovery fine-tuning process to reconstruct the model <ref:2610.15159#pg4>.

Tom: And there's this detail about how they decide whether to use cosine similarity or CKA similarity, which depends on some rule computed only on calibration data <ref:2610.15159#pg4>. That's a practical step that grounds the theory in actual application.

Jane: So, what are the actual numbers they show? Because I need to know if this isn't just theoretical fluff.

Tom: At two times compression, FCM lowers me perplexity below ShortGPT on twelve out of thirteen models, which is significant on ten of them, and it achieves gains up to fifty percent on Qwen3-4B Table three.

Meng: Fifty percent gain is substantial. But how does that translate when you think about the cost of running these compressed models compared to the original ones?

Lu: The paper shows that FCM attains the best perplexity on ten out of thirteen models against FlattenGPT, SWM, DeltaLLM, and MKA under matched calibration and recovery Table four <ref:2610.15159#pg1>.

Jane: So they're beating several existing methods in terms of accuracy on this specific benchmark setup. What does that imply for the real world application?

Tom: The gains carry over to downstream tasks Section four point six, and the compression sweep reveals a crossover point beyond which FCM’s advantage keeps growing up to three times and persists at the 14B scale Table three.

Meng: Three times that factor is big. But what about the limitations they admit? What's not covered by this method, or where does it break down?

Lalam: They state that FCM casts grouping as community detection on the full L by L similarity graph, which is a specific way of doing things <ref:2610.15159#pg3>. It also mentions that MKA groups greedily by pairwise similarity and merges weights rather than retaining a phase-covering representative <ref:2610.15159#pg3>.

Jane: So, the method relies heavily on the quality of that full similarity graph construction. If the initial profiling step is flawed, does the whole compression strategy fall apart?

Lu: The paper does mention that existing methods for depth compression score layers individually or group only adjacent or contiguous layers, which implicitly assumes that functional redundancy is local <ref:2610.15159#pg1>. FCM directly tests this assumption by computing those full pairwise CKA similarity matrices <ref:2610.15159#pg2>.

Tom: So the main thing here is moving from a local view of layers to a global view defined by these non-adjacent functional relationships, which leads to better compression and better results for the models.

Jane: This paper, "Beyond Adjacent Layers: Graph-Guided Layer Fusion for Compressing Large Language Models," seems to suggest that we need a more holistic way of thinking about how these massive AI models are structured internally.

Lu: It pushes the idea that understanding the functional equivalence between layers, not just their physical adjacency, is key for effective model compression <ref:2610.15159#pg3>.

Meng: From an engineering standpoint, if we can reliably identify these functional communities across different model architectures using this graph approach, it could lead to much more efficient and accurate distillation techniques.

Lalam: I'm confident in the findings because they are based on testing thirteen LLMs across six families, which gives a broad view of how this structure behaves <ref:2610.15159#pg1>.

Conclusion: Tom: So we've seen how this paper uses full pairwise similarity matrices to find functional groups of layers in large language models, and now we're looking at what the authors are actually trying to claim with this "Beyond Adjacent Layers" work <ref:2610.15159#pg4>.

Jane: The title suggests they're moving past just looking at neighbors and realizing the important connections are spread out <ref:2610.15159#pg3>. But what does that actually mean for how we build these models?

Lu: They are showing that functional equivalence isn't just local; it happens across non-adjacent layers too, which is a structure that prior similarity-based layer merging hasn't systematically exploited <ref:2610.15159#pg3>. It suggests the compression gains can go way beyond what you get from only looking at adjacent layers.

Meng: So if this graph structure is real, it means we could potentially compress these models much further than current techniques that rely on local assumptions <ref:2610.15159#pg1>. But how do we know this structure holds up when you try to deploy the compressed version?

Lalam: The method involves finding communities through spectral clustering on the similarity graph, then picking the most important layer in each community based on its Block Influence score <ref:2610.15159#pg4>. That selection process is what dictates which layers we keep for the compressed model.

Tom: And they show that this approach doesn't just compress; it actually improves performance, with lower perplexity and better results on downstream tasks compared to other methods <ref:2610.15159#pg7>. That’s a measurable win here.

Jane: So the big idea is that by understanding these deep functional relationships, we get both smaller models and models that don't lose too much accuracy <ref:2610.15159#pg8>. But the authors also have to be careful about how they define those communities because if the initial similarity calculation is off, the whole grouping fails.

Lu: Yeah, they acknowledge that this process relies on building that full L by L similarity graph correctly, and it's a specific way of doing things <ref:2610.15159#pg3>. That initial step is where the main computational effort goes into making sure the structure is sound.

Meng: For us in production, the cost comes with that initial profiling step on a calibration dataset D, which you need to collect first <ref:2610.15159#pg2>. I wonder how much data that upfront profiling actually costs compared to just running a standard compression sweep.

Lalam: The paper shows that FCM uses cosine similarity by default unless it hits a certain degeneracy threshold, which is defined by specific fixed thresholds for beta and mu <ref:2610.15159#pg4>. That gives the grouping decision some concrete rules to follow.

Tom: So, in short, this paper proposes a method that looks at non-adjacent layers to find functional groups, picks the best representative layer from each group based on influence, and then recovers the model with fine-tuning <ref:2610.15159#pg7>. It's about finding better structure for smaller models.

Jane: So it’s less about making a single perfect model and more about finding a smarter way to represent the same knowledge in fewer, more meaningful parts <ref:2610.15159#pg4>. This makes sense for scaling up AI systems that are currently becoming massive.

Lu: It really pushes the idea that understanding how layers functionally relate to each other, rather than just their physical position on the stack, is key for effective model compression <ref:2610.15159#pg3>. That's a major shift in how we think about representation learning in these models.

Meng: If this technique scales well across different architectures, it changes the whole compression toolkit for deploying large language models to edge devices <ref:2610.15159#pg4>. It shows that structural understanding is a viable path to efficiency.

Lalam: The implication is that we can move from just cutting layers haphazardly to strategically fusing functionally related parts of the model, which leads to better compression and better results <ref:2610.15159#pg7>. That's what this work is pointing toward.