- Rating
- 2
- Confidence
- 3
- Informativeness
- 2
- Presentation
- 3
- Soundness
- 1
Read review
Paper summary
HySparse2 adapts HySparse to long-context agentic workloads through two levels of KV reuse. At the outer level, a YOCO-style architecture divides the model into a first-half self-decoder and a second-half cross-decoder. KV Bridging applies layer-specific K/V projections to the input hidden states of the self-decoder’s sole FA layer, producing distinct caches for four cross-decoder FA layers. At the inner level, sparse layers reuse the preceding FA layer’s KV cache and selection indices, as in HySparse. HySparse2 replaces selection in 64-token blocks with selection of 1,024 individual global tokens. It also forces the 128 most recent tokens into the same sparse-attention computation instead of maintaining a separate SWA branch. Prompt KV-cache construction can therefore stop after the first 25 layers, while generation still runs the full model. HySparse2 reports higher long-context retrieval and agentic-task scores than HySparse and Hybrid SWA. At 1M tokens, Figure 3 gives 2.92× and 5.02× fewer prefill FLOPs, respectively, and an FP8 KV cache of 2.69 GB versus 6.72 GB and 12.09 GB.
Critical strengths
S1. FA-only KV Bridging and the forced local window enable prompt KV-cache early exit. The four cross-decoder FA layers obtain their K/V from self-decoder hidden states, and subsequent sparse layers reuse those caches for both global and recent tokens. Unlike a separate SWA branch, this arrangement does not require local K/V derived from cross-decoder hidden states. The self-decoder/cross-decoder split follows YOCO, but its integration with HySparse’s within-group KV Reuse is a meaningful architectural adaptation.
S2. The reported long-context gains are substantial at a large training scale. Relative to HySparse, the 80B-A3B HySparse2 model raises pretraining RULER from 84.89 to 90.77 and NoLiMa from 40.27 to 49.76. After the common post-training stage, its average gains over HySparse are 11.30 points on MRCR-v2 and 19.81 points on RULER-v2.
Critical weaknesses
W1. The reported KV-cache reduction is explained by KV width rather than KV Bridging. YOCO lets its cross-decoder layers share one projected global K/V cache. HySparse2 instead gives each cross-decoder FA layer its own K/V projections and a distinct cache. Including the self-decoder FA layer, HySparse2 retains five global caches, as HySparse does. KV Bridging changes when these caches can be constructed, enabling prefill early exit, but does not reduce their number relative to HySparse. In Figure 3’s FP8 comparison at 1M tokens, HySparse’s per-token cache width is , whereas HySparse2’s is . This 2.5× ratio matches the reported 6.72-to-2.69-GB difference. The paper should distinguish the storage benefit of the MQA configuration from the prefill benefit of KV Bridging.
W2. The prefill benefit is not measured. Figure 3 reports calculated FLOPs and cache size, not elapsed prefill time. The paper also does not quantify token-level selection and gather overhead, end-to-end latency, or decode throughput in a serving implementation. Without these measurements, the practical speed benefit for the proposed agentic workload remains unverified.
Questions for authors
Q1. Please decompose the 1M-token cache sizes in Figure 3 by cached FA layers, KV-head count, K/V dimensions, and any local cache. After fixing KV-head count and dimensions, does KV Bridging itself reduce cache storage relative to HySparse? If not, please attribute the reported cache reduction to the changed attention configuration rather than to KV Bridging.