PageWeaver: KV-Guided Query Unions for Sparse Attention

Introduction to the show: ident: AI Radio. Generated commentary on the latest Artificial Intelligence papers.

Tom: I'm Tom, and with me are Jane, Lu, senior AI researcher at Tsinghua, Meng, lead engineer at a mysterious AI startup and Lalam, the in-house Large Language Model.

Jane: Today's paper: "PageWeaver: KV-Guided Query Unions for Sparse Attention".

Tom: The gist The authors present PAGEWEAVER, an execution design that uses selected-page affinity to assemble query groups while preserving each query’s original support and complete output ownership.

Jane: First, who's behind it and why it matters.

Paper summary: Tom: So we’ve seen they are trying to improve how queries are grouped based on page affinity while keeping the original support and output ownership intact. The thesis is that this design uses selected-page affinity to assemble query groups, and it does this while preserving each query’s original support and complete output ownership.

Jane: What that means in plain terms is they are finding a way to organize the computation around how queries use data pages, so they don't have to waste time reordering things or losing information when grouping them up.

Lu: The mechanism involves a bounded GPU search to produce query IDs, and then an ID-aware two-CTA kernel that consumes these IDs without materializing reordered Q tensors or cross-page partial outputs. That avoids the problem of having to materialize those intermediate steps, which is a big win for memory usage and speed.

Meng: So if we think about the practical impact on a system, this seems like it’s about making the hardware utilization much tighter when dealing with sparse attention patterns where queries are not all accessing the same data.

Tom: Exactly. They also contrast this with a direct KV-page union implementation to see what happens when you just rely on disjoint page pairs and full membership to form common or exclusive query lists, which shows a complementary design study of nonlocal reuse and reduction cost.

Jane: And they show that shared structure alone doesn't guarantee a profitable traversal, suggesting that the organization has to be smarter about how it uses the page relation.

Lalam: The paper also highlights that adjacent query groups mix two page communities, while distant queries might share them, revealing a page-side relation that shows nonadjacent queries sharing support. That’s a key insight into query affinity.

Tom: It seems like PageWeaver is focused on using that affinity to change the members of a query-owned group within a bounded window, greedily collecting queries whose selected sets add few new pages.

Jane: It sounds like they are optimizing that greedy collection process to achieve better grouping without introducing any reduction in crosspage output, which is a critical part.

Meng: From an engineering standpoint, the claim that this trades an online grouping cost for a smaller union without introducing crosspage output reduction is important because it means we aren't sacrificing correctness for speed.

Lu: They also mention the fixed execution size is set at g equals eight, independently of the search window W, which makes the execution size predictable regardless of how much searching they do.

Tom: So what we’re seeing here is a design that connects query group size, page-set affinity, and output ownership in one common model. It’s building an ID-aware FP8 sparse-attention path on Hopper for this work.

Jane: And the result is that with FP8 KV throughout, the H200 Union8 implementation achieves a one point seven times geometric-mean complete-call speedup over the FlashInfer path on six captures, which is a solid number to look at <ref:2610.11201#pg1,with FP8 KV throughout, the H200 Union8 implementation achieves a 1>.

Conclusion: Tom: So to wrap up on PageWeaver: it uses the relation between selected queries and KV pages to improve execution groups while retaining each query’s full attention output. The central design principle they are focusing on is optimizing the cost of executing a fixed sparse relation, including the cost of constructing its execution groups.

Jane: It seems like the authors are showing that you can get performance gains by paying for the cost of constructing those execution groups in a smarter way, rather than just doing it naively. The direct KV-page execution and the B300 comparison show why you need to evaluate reuse alongside both preparation and output costs.

Lu: This paper connects query group size, page-set affinity, and output ownership in a common execution model with direct KV-page union as a measured design alternative that is really useful for understanding these interactions.

Meng: For me, the implication is that when designing sparse attention systems, we need to look at how the data pages interact with the queries because simply having many queries accessing data isn't enough; it’s how they access it.

Tom: Yeah, so this PaperWeaver design is about finding a way to optimize that fixed sparse relation cost by using those page relations intelligently. It’s a concrete execution strategy for handling complex sparse attention patterns.

Jane: The authors also emphasize that operator evidence emphasizes selected inputs, and they say independent natural-text validation should precede a general scheduling rule before you deploy this.

Lu: It's about connecting the dots between query group size, page-set affinity, and output ownership in a common execution model for sparse attention.

Meng: I think the practical impact is that this gives engineers a specific mechanism to try when they're trying to squeeze out more performance from existing hardware architectures for these kinds of workloads.

Tom: That’s right, it’s about optimizing the cost of executing a fixed sparse relation by constructing those execution groups efficiently. That’s what PageWeaver is all about.