- Rating
- 3(+1)
- Confidence
- 3
- Informativeness
- 2
- Presentation
- 2
- Soundness
- 2
Read review
Paper summary
This paper introduces Turnout, which preserves the original full-attention path and trains a hybrid linear branch (GDN plus sliding-window attention and sink tokens) via distillation. A router trained with reinforcement learning makes input-dependent, per-layer, per-token decisions about whether and when to irreversibly switch to the linear branch, aiming to reduce KV-cache memory while preserving accuracy on long reasoning generations.
Critical strengths
- Turnout enables input-dependent, layer-wise switching between full and linear attention, providing a flexible approach to balancing reasoning accuracy and memory efficiency.
- The method achieves a favorable accuracy–memory trade-off across five reasoning benchmarks.
Critical weaknesses
1.It remains unclear what fundamental challenges make simpler heuristic or search-based switching strategies insufficient, and why reinforcement learning is a particularly appropriate choice for this problem.
-
The baselines mainly consider fixed layer subsets or uniform switching positions, without relevant prior work on hybrid layer selection. The current comparisons may therefore overstate the advantage of learned adaptive routing over simpler alternatives.
-
Section 3 reveals substantial differences between Qwen3-8B and Llama 3.1-8B in their responses to linearization and sliding-window attention. However, the proposed RL-based routing is evaluated only on Qwen3-8B, leaving its effectiveness across different model architectures unverified.
-
With 36 layers and a rank-128 LoRA adapter on each output projection, standard independent adapters alone would introduce approximately 37.75M parameters, exceeding the reported 9.09M additional parameters for the entire linear branch. The paper does not clearly explain whether parameter sharing, weight merging, or another implementation accounts for this discrepancy.
-
No implementation code is provided, and critical details of the dynamic attention-switching mechanism, particularly how the GDN recurrent state is constructed when transitioning from full attention, remain insufficiently specified.
Questions for authors
- Does linear-branch distillation include trajectories that transition from full to linear attention? If not, how is the GDN recurrent state initialized at the switching point, and what evidence supports the frozen branch's robustness to mixed-attention hidden-state distributions?
- How frequently do all-incorrect rollout groups occur under the accuracy-scaled reward, and how are these zero-advantage groups handled during training? Are they skipped, resampled, or simply included without contributing gradients?
3 comments
Reply [2]
W3. Llama result We trained Turnout routers for Llama 3.1-8B-Instruct and added the results to Table 1:
| Method | GSM8K | GPQA-D | MATH-500 | MBPP+ | HE+ | Avg. | Cache saving (%) |
|---|---|---|---|---|---|---|---|
| Full attention | 78.2 | 23.7 | 48.6 | 73.0 | 58.5 | 56.4 | 0.0 |
| All linear | 51.2 | 8.1 | 28.0 | 31.2 | 25.6 | 28.8 | -68.4 |
| Turnout (, Mem.) | 78.1 | 23.2 | 49.2 | 71.4 | 57.9 | 56.0 | -1.4 |
| Turnout (, BE-Cos.) | 77.9 | 26.0 | 52.0 | 71.7 | 60.4 | 57.6 | 1.3 |
Both routers keep the accuracy of full attention, while the all-linear model loses 27.6 points. The cache saving is small because Llama 3.1-8B-Instruct does not generate long reasoning traces. One linear layer uses the memory of 584 KV-cache tokens (512 for the GDN state, 64 for the sliding window and 8 sink tokens). The prompt and response are shorter than 584 tokens for 97.5% of GSM8K, 99.5% of MBPP+ and 91.5% of HumanEval+ samples. For these samples, full attention uses less memory than the linear path, which is why the all-linear model has a negative saving. The router learns to keep full attention on such short sequences, which is the intended effect of the break-even point in the reward (Section 4.3).
We add this explanation to Section 5.2.
W4. Parameter count
Thank you for noticing this inconsistency. The parameter count in Table 2 did not include the LoRA adapters, because we assume that the adapter of a layer is merged into its output projection when the layer switches. This is safe because a layer never switches back to full attention (Section 4.2). After the switch, the layer uses only the linear path, so the original projection is no longer needed.
As a result, the LoRA parameters add no compute and no weight transfer during decoding. Before the switch, the full path does not use the adapter. After the switch, the layer reads one merged projection layer of the original size. The only extra cost is a one-time merge per layer.
We agree that this must be stated explicitly. Table 2 now has an extra row for Turnout with unmerged LoRA adapters:
| Method | Added parameters | Overhead | Weights (GiB) |
|---|---|---|---|
| Turnout (LoRA merged) | 9.74M | 0.12% | 15.28 |
| Turnout (LoRA not merged) | 47.49M | 0.58% | 15.42 |
The 36 rank-128 adapters add 37.75M parameters, as the reviewer computed. Section 5.3 now explains the merge.
W5/Q1. Code and GDN state transition We have done our best to include all the implementation details and hyperparameters in the paper and appendix. We are working on releasing the code and plan to do it upone paper acceptance. We also expanded the description of the switching mechanism in Section 4.1 and added Appendix B.2, which gives the equations of the state computation.
The linearization stage uses the linear path throughout the training sequence and there is no switching mechanism at this stage. However, after linearization both attention paths share the key and value projections. While a layer runs in full attention, the KV cache stores its keys and values, and we also store the two scalar GDN gates of each token and head. At the switch, we compute the GDN state from the cached keys, values and gates with the chunked GDN kernel of Flash Linear Attention library. Given the same layer inputs, this is exactly the state that the linear path would have after the same prefix. We then keep only the 8 sink tokens and the last 64 tokens of the KV cache, and remove the rest.
After a switch, the layer inputs come from a mix of full and linear layers. The linear path was not trained on this mix. Our empirical results suggest that the frozen paths handles inconsistency. Turnout (, BE-Cos.) runs in this mixed mode on every response and reaches 82.4 average accuracy, 2.7 points below full attention and 36.5 points above the all-linear model (Table 8).
Q2. All-incorrect rollouts. We do not treat all incorrect group specifically. We did not observe that happens often, but if it does, the advantage becomes zero and so does the gradient. We have added the note about it in the Section 4.4.
Thank you for the detailed response, which answers my concerns. I would suggest that you present the motivation for RL more clearly in the paper and this strengthens the rationale of your method. I will raise my score to 3.
Reply [1]
We thank the reviewer for the feedback and for recognizing Turnout's input-dependent, layer-wise switching, its flexibility in balancing reasoning accuracy and memory efficiency, and its favorable accuracy–memory trade-off across five reasoning benchmarks. We also uploaded updated version of the paper with all the changes marked with blue color.
W1. Motivation for reinforcement learning and simpler switching strategies. In our opinion, these properties of the problem make heuristics and search hard:
Search is possible only over fixed schedules, because each candidate must be evaluated by generating and grading full responses on a dataset. Manual switching and per-layer hybridization are such fixed schedules. Turnout is more accurate than both (Table 8):
We chose RL since it fits the required properties. GRPO needs only a reward for each sampled response, and it does not need gradients through the generation. It compares K=4 routings of the same prompt, so the advantage does not depend on how hard the prompt is. Training is cheap: the router has 0.65M parameters and is trained for 100 steps on four GPUs. We would appreciate if the reviewer could elaborate the details of a simpler heuristic-based approach that can satisfy these three non-trivial criteria.
W2. Prior work on hybrid layer selection.
The per-layer hybridization baseline in the paper already selects layers based on data. It keeps the layers with the largest final activation-distillation loss (MSE between full and linear path) in full attention (see Section 5.2 and Appendix A.1). These are the layers that are the hardest to linearize.
We now add FlashMorph (Lan et al., 2026) as a baseline from prior work. It selects the full-attention layers with trained per-layer gates. We use two layer rankings for it:
For both, we skip the fine-tuning step after layer selection, so all baselines differ only in which layers keep full attention. With 50% of the layers in full attention accuracies are:
FlashMorph (paper) is close to our per-layer selection, while FlashMorph (ours) is much lower on GPQA-D, MATH-500 and code, because its gates keep the first layers in full attention (see Table 9 in the paper). Turnout is more accurate than per-layer hybridization and both FlashMorph variants.
We added both FlashMorph variants to Figure 5 and Section 5.2, selected layers are listed in Appendix B.4 (Table 9). We did not find other post-hoc layer-selection methods that work without retraining the model. If the reviewer has a specific method in mind, we are happy to add it.