- Rating
- 2
- Confidence
- 1
- Informativeness
- 1
- Presentation
- 2
- Soundness
- 3
Read review
Paper summary
This work proposes a new perspective on the depth of transformer architectures in the context of sharing keys and values across layers. The authors introduce the concepts of "read depth" and "broadcast depth" and show that the read depth survives KV sharing while the broadcast depth does not. They also analyze the theoretical lower bounds of the length of the hard prompts that a model would fail.
Critical strengths
- The authors discuss the theoretical expressiveness of transformers with KV sharing, which is a critical aspect of understanding the limitations and capabilities of modern transformer architectures.
- The analysis and proofs are sound and well-structured. The authors provide a clear theoretical framework that builds on the communication complexity of transformers and give a rough estimate of the lower bounds of the length of hard prompts that a model would fail.
Critical weaknesses
Disclaimer: I am not an expert in this area, and my comments are only an educated guess based on my understanding of the paper.
- The paper is well-written and the results are theoretically meaningful. However, the theoretical results are too loose to be informative, either for model design or for understanding the limitations of transformers. A prompt with 7M tokens is not a practical scenario currently for any models.
- The paper discusses the impact of "read depth" and "broadcast depth" on the expressiveness of transformers. In the context of KV sharing, the place where the keys and values are shared across layers is more crucial than their counts. For example, if we make DeepSeek-V4.1-Flash have the first 4-8 layers with full attention and the rest either reuse, reindex or use sliding window attention, the model is expected to perform much worse than the standard one. While in the paper these two models are considered almost equivalent, as they have the same read depth and broadcast depth. I understand this is perhaps beyond the scope of the paper, but missing this point keeps the hypothesis of the paper away from the practical reality of transformer architectures.
Questions for authors
See the weaknesses section.
Response to Reviewer zk4P
Thank you for the review. Prompts of this length are beyond deployed contexts. What the results establish is a growth statement: for joins, one shared KV generation needs read bandwidth that grows linearly with the length, while two generations need two layers of logarithmic size at every length. The 7.6M figure is where that growth overtakes the bandwidth of one published model.
W1 (looseness and the 7M-token prompt).
What the number measures. For the YOCO baseline (32 layers, 20 heads of dimension 128, p=16), the join bound applies to every N of at least 7,603,697, i.e. to prompts of about 1.52⋅107 tokens. At p=133, which represents every BF16 value, it applies from N≥4.65⋅107 (Section 5, Appendix G.2). The figure is a simplified upper bound, 2L(w−1)+L⋅Bdense, on the bits exchanged across the cut between the two blocks in our simulation of the 16 cross-decoder reads. The simulation's own count is smaller and already rules out every N of at least 5,116,657. How close a model of this width can come to the bound is not known; the bound concerns one-generation models (YOCO, YOIO), not DeepSeek-V4.1-Flash, which has four generations.
How sharp the bounds are.
The first row is tight in read bandwidth up to a constant factor. The second is tight in read depth.
What it implies for design.
Below the thresholds our results neither predict nor exclude a capability gap. We have added this to Section 5 of the revised PDF (paragraph "What the ledger says for design", in blue), together with where the thresholds come from.
W2 (where sharing happens).
We agree that the ledger invites the reading that equal depths mean equal models, and that placement can matter a great deal for a trained model. The depths are necessary conditions, not a ranking. Too little read depth forces failure on composition and pointer chasing at the stated lengths, and a single KV generation forces read bandwidth linear in N for joins. Neither statement orders models with enough of both.
Take k Full layers at the bottom (4≤k≤8), with the remaining global layers reusing or re-selecting within the candidate pool. That variant has broadcast and read depth k, at least the released model's 4, so our bounds give it no disadvantage. They neither predict nor exclude the performance difference you describe. The reason is a modelling choice, made as in Chen et al. (2025): token-wise maps are arbitrary functions, so the finite capacity of the layers between and after the reads is outside the model. For models with several KV generations, such as both models in your example, our bounds do not depend on where the fresh reads sit. For one-generation models, placement enters only through the layer σ0 that computes the shared cache and, with sliding windows, through the window-cone sizes, both in (4.1).
We have added a sentence on placement to Section 5 that states this scope.
If these clarifications address your concerns, we would appreciate it if you would reconsider your rating.