- Rating
- 2
- Confidence
- 3
- Informativeness
- 2
- Presentation
- 2
- Soundness
- 2
Read review
Paper summary
This paper finds that a reasoning model’s first generated token can change the thousands of tokens that follow, and that output length still changes when only the keys and values cached at that position (the “opening memory”) are exchanged while the visible token is held fixed. The authors then introduce Opening-Memory Steering (O-Steer), which edits the opening token’s KV cache entries once at inference time, using a shared memory anchor and a small problem-dependent controller to adapt the edit to each problem. Across 1,919 problems from six math benchmarks, O-Steer reduces mean output length with accuracy decreases.
Critical strengths
This paper is well written and easy to understand, fun to read from a interesting motivation.
Critical weaknesses
- The core claim feels mixed between an interpretability study and an efficient-reasoning method. In my reading, the central contribution is essentially “steering the first token’s memory to control the long reasoning trace.” Given this, the paper would be stronger if it engaged more directly with the recent LLM-reasoning literature, which has grown substantially since EMNLP 2025 / NeurIPS 2025 / ICLR 2026, rather than framing the contribution around “analyzing the role of memory” (as i quote from L453)
Recommend to read:
[1] Training-Free Intermediate-Budget Reasoning via Token-Level Triggers. ACL 2026. https://arxiv.org/abs/2601.07036
[2] Wait, We Don’t Need to “Wait”! Removing Thinking Tokens Improves Reasoning Efficiency. EMNLP 2025 https://arxiv.org/abs/2506.08343
[3] Echoes as Anchors: Probabilistic Costs and Attention Refocusing in LLM Reasoning. ICLR 2026. https://arxiv.org/abs/2602.06600
[4] Demystifying Reasoning Dynamics with Mutual Information: Thinking Tokens are Information Peaks in LLM Reasoning. NeurIPS 2025. https://arxiv.org/abs/2506.02867
- The experimental evidence is insufficient.
-
I did not find the mechanistic-interpretability evidence convincing, despite “mechanistic interpretability” appearing in the keywords and “interpretability and explainable AI” being the selected primary area. Section 2 and Appendix A show that exchanging the opening keys or values changes output length, but not how; the authors themselves note that the analysis covers one model and one token pair (L473) and does not identify which memory entries carry the effect (L875–876).
-
The experiments section reports only Table 1 (page 8) for the accuracy–length trade-off, with a single operating point per method. Viewed within the LLM-reasoning literature (see W1), this result is relatively incremental: on DeepSeek-R1-Distill-Qwen-1.5B, SEAL is better on both accuracy and length (L367–368), and on both models the accuracy drop exceeds the fitting tolerance ε = 0.04 (L1240–1241). Extending it with deeper analysis, such as accuracy–length curves at matched budgets and a breakdown of where the saved tokens come from, would substantially strengthen the paper.
Questions for authors
-
Layout: is there a reason why L478–485(in page9) are left blank? Given the page limit, that space could be used for the additional analysis mentioned in the weaknesses
-
Do you plan to open-source the problem-dependent controller and code?
7 comments
W1. Framing and recent work
Neither Okay nor To supplies a mathematical step, yet the choice of opening can change solution length by thousands of tokens. We use differences in checks, calculations, and repetitions across openings to construct first-token KV edits. We test them on complete development solutions, then fix one opening per model and one edit per objective.
We discuss all four suggested studies in §6. Mid-Think combines an early reasoning cue with a closing pattern, while NoWait suppresses reflection-associated tokens. Echoes as Anchors examines question restatement, and Demystifying Reasoning Dynamics with Mutual Information links reflection and transition tokens to peaks in representation–answer dependence. In our experiments, reversing a checking edit from negative to positive increased both checking frequency and length on Qwen3-1.7B. On 8B, reversing a repeat edit barely changed repetition (Table 31). Omitting the two Chinese openings changed the 8B repeat–length cosine from −0.206 to +0.681 (§5.3).
W2a. What the cache exchange changes
In Qwen3-1.7B, replacing just the values at Alright with those from Yes shortened output by 1,757 tokens on average. The first two output tokens agreed. In the 235 paired draws that reached 128 reasoning tokens under all four conditions, we counted 49 checking-intent expressions with the native cache and 9 after the exchange. Detected calculations rose from 1 to 45. These are freely generated prefixes matched in length (Appendix B.2).
In a separate experiment, we replayed identical text with the Okay and Yep opening caches. Before processing token 128, we set the opening KV to Okay in both conditions. Changing only entries 2–127 from the Okay to the Yep replay shortened the remaining output by 368 tokens on average. The later entries therefore carry part of the opening’s effect. We have yet to identify the responsible heads or layers (§5.4 and Table 16).
Thank you for the four references and for asking where the saved tokens come from. We now measure reasoning and final-answer length separately and compare paired solutions to see which checks survive a shorter trace.