- Rating
- 2
- Confidence
- 3
- Informativeness
- 3
- Presentation
- 3
- Soundness
- 2
Read review
Paper summary
Adamas uses a Hadamard transform and two-bit query/key codes to select KV tokens. It then runs attention on the selected original KV states. The method needs no training and uses custom kernels.
Critical strengths
- Quality is better than the tested sparse-attention baselines at small token budgets.
- The paper measures kernel and decoding time, and tests the transform and quantization choices.
Critical weaknesses
- Practical gains are limited. The largest speedups come from MHA, while end-to-end speedup on GQA is only about 1.1x. The paper does not show whether selection overhead is worthwhile for MLA or hybrid models with fewer full-attention layers.
- Quest expands GQA to MHA in this implementation, which adds cost and causes OOMs. The runtime gap is therefore partly an implementation issue, not just a better selection method.
Questions for authors
- Can you separate selection and attention costs, with native GQA support for Quest?
- Does the method provide meaningful end-to-end gains on MLA or hybrid attention models, including selection overhead?
- What batch sizes were used? Do the GQA gains hold with batching?
4 comments
Replies to Reviewer Wrje (2)
Q3: Efficiency Gains across Batch Sizes
Thank you for this question. All latency experiments in the original paper were conducted with a batch size of 1. To evaluate whether the speedups persist under batched inference, we conducted additional experiments with batch sizes of 1, 2, 4, and 8, covering the batch sizes that fit within the 80 GB memory capacity of an NVIDIA H800. The results are reported below, where the speedup is calculated as the latency of Full Attention divided by that of Adamas.
| Model | Context Length | B=1 | B=2 | B=4 | B=8 |
|---|---|---|---|---|---|
| Llama-3.1 | 8K | 0.972× | 0.999× | 1.071× | 1.128× |
| 16K | 1.014× | 1.079× | 1.231× | 1.414× | |
| 32K | 1.095× | 1.219× | 1.526× | 1.889× | |
| LongChat | 4K | 1.031× | 1.110× | 1.212× | 1.333× |
| 8K | 1.145× | 1.293× | 1.554× | 1.849× | |
| 12K | 1.241× | 1.476× | 1.881× | 2.286× |
The results show that Adamas maintains its efficiency benefits under batched inference for both GQA (Llama-3.1) and MHA (LongChat) models. Moreover, the speedup generally increases with batch size. For example, Adamas achieves 1.89× speedup on Llama-3.1 at a batch size of 8 and 32K context length, and 2.29× on LongChat at a batch size of 8 and 12K context length.
These results demonstrate that the efficiency gains of Adamas extend beyond batch-size-1 inference and remain effective across a range of batch sizes. We will include these additional results in the revised manuscript.
Thanks for the additional experiments. They help clarify the trade-offs, though I still find the gains at near-full quality modest. I’m keeping my weak reject recommendation.
Thank you for the follow-up and for acknowledging that the additional experiments help clarify the trade-offs. We agree that the speedups at near-full-attention quality are relatively modest. However, we believe the broader results in the revised manuscript demonstrate the practical value of Adamas beyond this operating point. In particular, the additional batching experiments show that its speedups increase with batch size, reaching 1.89× for Llama-3.1 (GQA) at 32K and 2.29× for LongChat (MHA) at 12K, while the hybrid-model experiments further demonstrate its applicability to models with hybrid attention architectures. Together with the quality–latency comparisons, these results provide a more comprehensive picture of Adamas across different inference settings and operating points. We hope these additional results help clarify the scope and practical significance of our contribution.
Replies to Reviewer Wrje (1)
Thank you for the thoughtful comments. Below we provide our detailed responses to each concern. For closely related questions and concerns, we provide a combined clarification.
W1 & Q2: End-to-End Efficiency and Accuracy on Hybrid Attention Models
Thank you for raising this concern. To evaluate Adamas on hybrid attention architectures, we conducted additional experiments using Qwen3.5-9B, a hybrid attention model that combines Gated DeltaNet (GDN) and full attention layers in a 3:1 ratio. We replaced all full attention layers (i.e., only one-quarter of the model's attention layers) with Adamas and evaluated both accuracy and end-to-end latency.
Accuracy. We evaluated the model on six LongBench datasets. The results are reported below:
End-to-end latency. We further measured the end-to-end speedup, including selection overhead, at different input lengths. We report results for context lengths of 32K and above:
Overall, Adamas maintains accuracy close to full attention, with the average LongBench score ranging from 51.83 to 53.07, compared with 52.82 for full attention. More importantly, despite replacing only one-quarter of the model's attention layers, Adamas achieves consistent end-to-end speedups at context lengths of 32K and above, reaching up to 1.138× at 128K. These results demonstrate that Adamas can provide practical efficiency gains in hybrid attention architectures, even when only a fraction of the attention layers are eligible for acceleration. We will include this analysis in the revised manuscript.
W2 & Q1: Native-GQA Quest Comparison at Matched Quality
Thank you for raising this concern. We agree that comparing Adamas against Quest's original MHA fallback may introduce implementation-related overhead and thus does not provide a fair assessment of their relative efficiency under GQA. To address this issue, we implemented a native-GQA version of Quest, which performs selection and attention directly over shared KV heads without expanding them into separate query heads. We then re-evaluated the decoding latency of both methods.
The updated decoding latency results are reported below:
Adamas consistently achieves lower decoding latency than native-GQA Quest across all evaluated context lengths and token budgets.
To further compare the two methods at closely matched task quality, we selected two pairs of operating points based on their LongBench normalized scores: Adamas-256 (92.92%) vs. Quest-1024 (91.15%), and Adamas-512 (97.95%) vs. Quest-4096 (97.20%). Their corresponding decoding latencies are shown below.
These results show that Adamas achieves comparable LongBench performance with substantially smaller token budgets. Even at closely matched quality, Adamas consistently achieves lower decoding latency than native-GQA Quest, with speedups ranging from 1.011× to 1.131× across the evaluated context lengths.
Overall, these additional experiments demonstrate that Adamas's efficiency advantage persists when compared against a native-GQA implementation of Quest, and is therefore not solely attributable to the MHA expansion overhead in the original Quest implementation. We will include this comparison in the revised manuscript.