acceptodds
Under review as a conference paper at ICLR 2027

Opening Moves: Understanding and Steering Language Model Reasoning

Abstract

A reasoning model’s first generated token can change thousands of tokens that follow. We study this sensitivity through opening memory, the keys and values cached at the opening-token position. Matched-text interventions show that exchanging opening keys or values changes output length without changing the visible opening, and that later cache entries can carry these effects under identical token histories. Based on these observations, we introduce Opening-Memory Steering (O-Steer), which steers long reasoning trajectories through a single edit to the memory written by the opening token. O-Steer uses natural opening tokens to construct a compact edit space, complete-continuation feedback to select a shared memory anchor, and a small problem-dependent controller to adapt the edit. At inference time, the frozen model’s opening memory is modified once, after which ordinary decoding proceeds without further intervention. Across 1,919 problems from six math benchmarks, O-Steer reduces mean output length by 40.3% on Qwen3-1.7B and 22.8% on DeepSeek-R1-Distill-Qwen-1.5B, with accuracy decreases of 5.91 and 4.17 percentage points, respectively. These results show that a single opening-memory edit can substantially change the accuracy–length trade-off of long reasoning trajectories.

Primary area:
interpretability and explainable AI
Keywords:
language model reasoning, representation steering, internal-state interventions, KV cache, mechanistic interpretability

Reviews

Among:
Shape:
1234this paper, Top 49%
Figure 1. Mean review rating of the reviewed papers at ICLR 2027. This paper averages 2.33 over 3 reviews, above 51% of the others.
Reviewer_m2HtOpenReview ↗
Rating
2
Confidence
3
Informativeness
2
Presentation
2
Soundness
2
Read review
Paper summary

This paper finds that a reasoning model’s first generated token can change the thousands of tokens that follow, and that output length still changes when only the keys and values cached at that position (the “opening memory”) are exchanged while the visible token is held fixed. The authors then introduce Opening-Memory Steering (O-Steer), which edits the opening token’s KV cache entries once at inference time, using a shared memory anchor and a small problem-dependent controller to adapt the edit to each problem. Across 1,919 problems from six math benchmarks, O-Steer reduces mean output length with accuracy decreases.

Critical strengths

This paper is well written and easy to understand, fun to read from a interesting motivation.

Critical weaknesses
  1. The core claim feels mixed between an interpretability study and an efficient-reasoning method. In my reading, the central contribution is essentially “steering the first token’s memory to control the long reasoning trace.” Given this, the paper would be stronger if it engaged more directly with the recent LLM-reasoning literature, which has grown substantially since EMNLP 2025 / NeurIPS 2025 / ICLR 2026, rather than framing the contribution around “analyzing the role of memory” (as i quote from L453)

Recommend to read:

[1] Training-Free Intermediate-Budget Reasoning via Token-Level Triggers. ACL 2026. https://arxiv.org/abs/2601.07036

[2] Wait, We Don’t Need to “Wait”! Removing Thinking Tokens Improves Reasoning Efficiency. EMNLP 2025 https://arxiv.org/abs/2506.08343

[3] Echoes as Anchors: Probabilistic Costs and Attention Refocusing in LLM Reasoning. ICLR 2026. https://arxiv.org/abs/2602.06600

[4] Demystifying Reasoning Dynamics with Mutual Information: Thinking Tokens are Information Peaks in LLM Reasoning. NeurIPS 2025. https://arxiv.org/abs/2506.02867

  1. The experimental evidence is insufficient.
  • I did not find the mechanistic-interpretability evidence convincing, despite “mechanistic interpretability” appearing in the keywords and “interpretability and explainable AI” being the selected primary area. Section 2 and Appendix A show that exchanging the opening keys or values changes output length, but not how; the authors themselves note that the analysis covers one model and one token pair (L473) and does not identify which memory entries carry the effect (L875–876).

  • The experiments section reports only Table 1 (page 8) for the accuracy–length trade-off, with a single operating point per method. Viewed within the LLM-reasoning literature (see W1), this result is relatively incremental: on DeepSeek-R1-Distill-Qwen-1.5B, SEAL is better on both accuracy and length (L367–368), and on both models the accuracy drop exceeds the fitting tolerance ε = 0.04 (L1240–1241). Extending it with deeper analysis, such as accuracy–length curves at matched budgets and a breakdown of where the saved tokens come from, would substantially strengthen the paper.

Questions for authors
  1. Layout: is there a reason why L478–485(in page9) are left blank? Given the page limit, that space could be used for the additional analysis mentioned in the weaknesses

  2. Do you plan to open-source the problem-dependent controller and code?

7 comments

Thank you for the four references and for asking where the saved tokens come from. We now measure reasoning and final-answer length separately and compare paired solutions to see which checks survive a shorter trace.

AuthorsOpenReview ↗

W1. Framing and recent work

Neither Okay nor To supplies a mathematical step, yet the choice of opening can change solution length by thousands of tokens. We use differences in checks, calculations, and repetitions across openings to construct first-token KV edits. We test them on complete development solutions, then fix one opening per model and one edit per objective.

We discuss all four suggested studies in §6. Mid-Think combines an early reasoning cue with a closing pattern, while NoWait suppresses reflection-associated tokens. Echoes as Anchors examines question restatement, and Demystifying Reasoning Dynamics with Mutual Information links reflection and transition tokens to peaks in representation–answer dependence. In our experiments, reversing a checking edit from negative to positive increased both checking frequency and length on Qwen3-1.7B. On 8B, reversing a repeat edit barely changed repetition (Table 31). Omitting the two Chinese openings changed the 8B repeat–length cosine from −0.206 to +0.681 (§5.3).

AuthorsOpenReview ↗

W2a. What the cache exchange changes

In Qwen3-1.7B, replacing just the values at Alright with those from Yes shortened output by 1,757 tokens on average. The first two output tokens agreed. In the 235 paired draws that reached 128 reasoning tokens under all four conditions, we counted 49 checking-intent expressions with the native cache and 9 after the exchange. Detected calculations rose from 1 to 45. These are freely generated prefixes matched in length (Appendix B.2).

In a separate experiment, we replayed identical text with the Okay and Yep opening caches. Before processing token 128, we set the opening KV to Okay in both conditions. Changing only entries 2–127 from the Okay to the Yep replay shortened the remaining output by 368 tokens on average. The later entries therefore carry part of the opening’s effect. We have yet to identify the responsible heads or layers (§5.4 and Table 16).

Reviewer_ZiqyOpenReview ↗
Rating
3(+1)
Confidence
3
Informativeness
2
Presentation
2
Soundness
2
Read review
Paper summary

The paper studies how much a reasoning model’s trajectory can be controlled by editing only the memory (keys and values) written by its first generated token, which it calls the opening memory. On Qwen3-1.7B with 600 development problems, it uses matched-text interventions. These display the same token but swap in the cached K and/or V from a different opening token (Okay vs. Yep). Swapping keys, values, or both shortens the output by 400, 798, and 948 tokens respectively, and the effect persists in later memory when the opening entry is restored.

Building on this, the paper proposes Opening-Memory Steering (O-Steer):

  • Opening bank and anchor: it forms a bank of 16 memory differences induced by natural opening tokens, builds a “short” and an “accuracy-weighted” proposal, and selects a shared memory anchor by constrained Bayesian optimization on complete-continuation outcomes.

  • Controller: a small affine controller of at most 36 parameters, conditioned on an 8-dimensional PCA feature of the native opening memory, adapts the anchor per problem. It is trained from antithetic Gaussian-perturbation feedback in two batches.

  • Inference: a single additive edit is applied at the opening position, with no further intervention.

On 1,919 problems from six math benchmarks, O-Steer reduces mean length by 40.3% (Qwen3-1.7B) and 22.8% (DeepSeek-R1-Distill-Qwen-1.5B), at accuracy decreases of 5.91 and 4.17 points. It is compared with Fixed Opening, Mean-V, SEAL and ReBalance.

Critical strengths
  1. Careful matched-text interventions that isolate opening keys, values and later memory. The interventions hold fixed the displayed token, the prompt cache, and the already computed second-token logits. The same-history replay and the later-memory exchange (Table 4) show that opening-dependent effects remain active under identical visible text. The distinct K and V effects (−400 and −798 tokens, with intervals excluding zero) are a clean, reusable observation. The authors also decline to over-interpret it, noting that attention mass alone does not explain the effect and that the opening token is not identified as an attention sink.

  2. Unusually honest reporting. I actually really admire that in papers. Well done. The abstract states the accuracy decreases, and the text notes that they exceed the fitting tolerance. It says that SEAL gives a better aggregate point on DeepSeek, that the evaluation suite was consulted during development, and that token reductions are not wall-clock reductions. Matched controls and paired intervals for the SEAL and ReBalance ports are provided (Tables 8–9). A single edit with no decoding-time intervention is also an appealing deployment property.

Critical weaknesses
  1. The results do not establish that O-Steer improves the accuracy–length trade-off over simpler or existing options. Everything is reported at one operating point (ε = 0.04). The accuracy loss is large, with intervals excluding zero: −5.91 [−7.98, −3.88] on Qwen and −4.17 [−6.29, −2.08] on DeepSeek. It exceeds the fitting tolerance. The comparisons are not matched. Fixed Opening is selected with a 2-point accuracy floor (Appendix C.1) while O-Steer uses 4 points, so the claim that O-Steer compresses 27% more than Fixed Opening on Qwen is not like-for-like. On DeepSeek, Fixed Opening is much shorter (2,878 vs. 4,096 tokens) at a different accuracy (49.87% vs. 51.41%), and SEAL has both higher accuracy and shorter outputs than O-Steer (54.64% at 4,005 vs. 51.41% at 4,096). There is no sweep over ε or baseline strength, and no simple budget-forcing or truncation baseline at matched mean length. The learned controller also appears to move along the trade-off curve rather than beyond it. On Qwen, the shared anchor alone gives 64.07% at 4,499 tokens (about −3.0 points and −31% length). The controller adds roughly −620 tokens for another −2.9 points. Accuracy losses are worst on out-of-distribution harder sets (Minerva −9.74 and AMC −8.43 on Qwen), although fitting used only 480 GSM8K/MATH/Omni-MATH problems.

  2. The mechanistic story and the method’s components are under-validated. The Section 2 analysis uses one model (Qwen3-1.7B) and one token pair (Okay/Yep). It never tests whether the opening position is special. There is no comparison with equal-norm edits at other early positions, such as the second token or the last prompt token, or with random K/V perturbations at the opening. The authors acknowledge that this would be needed to identify an attention-sink-like role. The K–V interaction in length is unresolved (+249.84 tokens, interval includes zero). The appendix says the experiments do not identify a unique carrier of the effect. In the method, “problem-dependent” adaptation is not isolated. The final edit has cosine similarity above 0.99 with the shared anchor, and the correction is about 16% of the anchor norm. Appendix B.6 states that the results do not separate problem-dependent adaptation from a learned shared correction. The pipeline has many fitted choices (k ≤ 4, an 8-dimensional PCA, ρ = 0.5, σ = 0.25, η = 0.05, ε = 0.04), and the evaluation suite was consulted during development.

Questions for authors
  1. Can you plot full accuracy–length frontiers with intervals, by sweeping ε for O-Steer and the strength or tolerance of Fixed Opening, SEAL and ReBalance? Can you add a plain budget-forcing or truncation baseline at matched mean length, and a Fixed Opening baseline selected with the same 4-point tolerance?

  2. What do ablations show for (a) the shared anchor plus a learned shared correction only (W = 0), versus the full controller, and (b) equal-norm edits at other early positions and random K/V directions at the opening position?

  3. Since the evaluation suite was consulted during development, can you evaluate on genuinely fresh problems (for example a recent AIME or a held-out benchmark) and on a configuration where the controller is fit only on GSM8K/MATH? Per-benchmark intervals would also help, especially for AIME and AMC (90 and 83 problems, two generations each).

  4. Why does the held-out accuracy drop exceed the fitted tolerance, and how large is the gap between the locally predicted and the realized accuracy change? Is there a way to enforce the constraint on unseen problems?

  5. Do the results hold for larger models (e.g., 7B+) and other domains (code, science QA)? Do other opening-token pairs reproduce the Section 2 findings?

7 comments

Thank you for pointing out the unequal accuracy allowances. We now apply the same selection rule to edited and literal openings and compare the selected settings on the same test problems. Revised O-Steer fixes one opening per model and one first-token KV edit per objective on development. We also isolated the original controller’s problem-dependent term.

AuthorsOpenReview ↗

Q1. Matched comparisons

For Table 22, we chose the shortest edit and the shortest literal opening whose development accuracy was at most 4 percentage points (pp) below natural sampling (Natural). On 240 test problems, both Qwen edits shortened outputs. The R1 edit gained accuracy and used more tokens. Differences are edit minus literal, with paired 95% intervals.

ModelSelected edit / literalAccuracy difference (pp)Mean token difference
R1-1.5BBalanced / First+6.88 [2.71, 11.04]+3,536 [2,994, 4,078]
Qwen3-1.7BLength-first / Yes−2.08 [−5.63, 1.25]−712 [−991, −437]
Qwen3-8BLength-first / I−1.67 [−4.17, 0.83]−545 [−791, −300]

Figure 5 shows three O-Steer settings and four strengths each of SEAL and ReBalance on both smaller models, with pointwise 95% intervals. Qwen3-1.7B Balanced and SEAL 1× both gave 303/480 correct. Balanced saved 243 tokens per response [13, 489]. On that panel, SEAL 2× reached 63.54% at 5,441 tokens and Balanced reached 63.13% at 5,567. A fixed cutoff matched to Balanced’s mean length on development reached 65.42% at 5,472. On both models, the cutoff-minus-Balanced intervals span zero for accuracy and length (§5.2). Table 21 gives the literal choices at 0, 2, 4, and 6 pp. We did not retrain the original controller at each ε\varepsilon.

AuthorsOpenReview ↗

Q2. Shared correction, position, and direction

On 180 additional Qwen3-1.7B problems, the submitted controller gave 65.83% at 3,799 tokens. With only the input-dependent term set to zero (W=0W=0), the shared correction gave 68.33% at 4,252. Full minus shared was −2.50 pp [−7.50, +2.22] and −453 tokens [−1,050, +89] (family-adjusted 95% intervals). The added benefit of adaptation remains uncertain.

Relative to unedited Okay, the same edit saved 1,412 tokens at the first generated token and 260 at the second, but added 106 at the last prompt token. Moving the edit also changes when it first acts. At the opening, two random directions from the opening subspace, matched in K and V norm, changed mean length by +9,501 and −345 tokens (Table 17).

Reviewer_68Z9OpenReview ↗
Rating
2(+1)
Confidence
3
Informativeness
1
Presentation
2
Soundness
2
Read review
Paper summary

The authors study how the key/value cache written by the first generated content token affects subsequent reasoning, using separate key/value swaps and matched-history cache exchanges. In this paper, they introduce O-Steer, which learns a small controller that edits this opening memory once, and evaluate its accuracy–length trade-off on two small reasoning models and 1,919 mathematics problems from six benchmarks.

Critical strengths

Separate key/value swaps hold the displayed opening and already-computed logits fixed, and the later-memory exchange uses identical intervening text. These controls support a causal effect of the manipulated cache state on continuation length without attributing it simply to different visible prefixes.

Critical weaknesses
  • SEAL has a better DeepSeek aggregate point estimate on both metrics (54.64% at 4,005 tokens versus 51.41% at 4,096), while the Qwen comparison does not match accuracy or sweep competitors to the same operating point.
  • Comparing the learned controller with the initial anchor changes both the shared correction and the input-dependent term, while the learned edits remain close to the anchor direction.
Questions for authors

See above.

4 comments

Thank you for taking the time to work through both the cache experiments and the method. You pointed out that SEAL was stronger at the reported R1 setting and that our anchor comparison changed two terms at once. The revision selects one opening per model on development problems and uses differences in reasoning features across openings to build candidate edits. We still intervene only once, at the first token.

AuthorsOpenReview ↗

Q1. Comparison with SEAL and ReBalance

In the submitted R1 result, SEAL was both more accurate and shorter (54.64% at 4,005 tokens vs. 51.41% at 4,096 for O-Steer). We now compare three revised O-Steer settings with four strengths each of SEAL and ReBalance (0.5× to 2×) on the same 240 problems per smaller model. Figure 5 (p. 28) shows the pointwise 95% intervals.

Balanced selects the shortest edit whose development accuracy reaches that of natural sampling. On Qwen3-1.7B, Balanced and development-selected SEAL 1× each produced 303 correct responses out of 480. Balanced saved 243 tokens per response relative to SEAL 1× (paired 95% CI, 13–489 saved). At 2×, SEAL had higher observed accuracy and shorter mean output than Balanced (63.54% at 5,441 tokens vs. 63.13% at 5,567). On R1, Balanced reached 55.63% at 5,027 tokens, close to development-selected ReBalance 2× at 55.21% and 5,052 (Table 24, p. 29).

On the full 1,919-problem suite (Table 2), R1 Balanced was more accurate and longer than SEAL 1× (56.64% at 4,959 vs. 54.64% at 4,005). Qwen3-1.7B Balanced was shorter with slightly lower observed accuracy (67.17% at 5,408 vs. 67.59% at 5,626).

AuthorsOpenReview ↗

Q2. What the input-dependent correction adds

In the submitted method, the mean cosine between the learned edit and the anchor exceeded 0.99 in each K/V block for both models. To isolate the input-dependent term, we kept the anchor, learned shared correction, and projection fixed on Qwen3-1.7B and set only that term to zero (W=0W=0). On 180 additional problems, the full controller gave 65.83% at 3,799 tokens and the shared correction alone gave 68.33% at 4,252. The full controller used 453 fewer tokens on average, and its accuracy was 2.50 percentage points (pp) lower. The family-adjusted 95% intervals for full minus shared were [−1,050, +89] tokens and [−7.50, +2.22] pp. The additional benefit of adapting to each problem remains uncertain.

For revised O-Steer, we compare the selected edit with the same opening left unedited. Across 1,919 problems, Balanced saved 634 tokens per response [545, 721] at okay on Qwen3-1.7B, with an accuracy change of +0.60 pp [−0.34, +1.51]. At So on R1, it saved 841 tokens with an accuracy loss of 1.51 pp (§5.1 and Table 23).

Thank you again for pointing out why the shared correction needed its own control.
Your review helped us identify these missing baselines and directly strengthened our evaluation.
We sincerely appreciate the time and thoughtful feedback you dedicated to improving our paper.

From OpenReview, CC BY 4.0. Authors as signed there; shown unedited.

Related papers

Loading the map…

Discussion (0)

Sign in to comment.