acceptodds
Under review as a conference paper at ICLR 2027

DYNAMIC WRITES, SIMPLE SCHEDULES: AN ORACLE STUDY OF IN-PLACE TEST-TIME TRAINING

Abstract

Test-time training (TTT) adapts a language model as it processes an input, but existing methods typically apply a fixed adaptation configuration throughout the context. We examine whether varying where and when the model writes can outperform the best constant write action. Using a fixed Qwen3-1.7B In-Place TTT checkpoint, we compare the per-example best constant action against a bounded oracle search over chunkwise dynamic trajectories. Across 6,500 examples from 13 RULER-16K tasks, the best trajectories returned by the bounded oracle search increase the mean score from \(78.38\) to \(79.77\), a gain of \(1.39\) percentage points. Among the 1,933 non-ceiling examples, the bounded oracle search improves 170 and ties the remaining 1,763, increasing their mean score from \(27.29\) to \(31.96\) (\(+4.67\) points). Controlled interventions show that successful outcomes depend on both the ordering of write actions and the model state accumulated across chunks. Nevertheless, complex switching is often unnecessary: 62 of 78 improved multi-switch trajectories admit an exactly score-matched one-switch alternative. We observe similar, though smaller, gains on Qwen3-4B and Llama-3.1-8B under their respective 32K-token protocols, improving over the per-example best constant action by \(0.79\) and \(0.36\) points, respectively. These results establish an oracle-reachable advantage for dynamic in-place TTT and suggest that relatively simple switching policies may capture much of this opportunity. It motivates learning policies that select write actions from the evolving context and adaptation state.

Primary area:
foundation or frontier models, including LLMs
Keywords:
Test-Time Training

Reviews

Among:
Shape:
1234this paper, Bottom 10%
Figure 1. Mean review rating of the reviewed papers at ICLR 2027. This paper averages 1.67 over 3 reviews, above 10% of the others.
Reviewer_ixGyOpenReview ↗
Rating
2
Confidence
3
Informativeness
2
Presentation
2
Soundness
2
Read review
Paper summary

The paper provide an oracle study whether varying where and when the model writes can outperform the best constant write action for in-place test-time training. The experiment results show that some dynamic trajectories can outperform the best constant write action. Besides, the paper provides some observations in the update of test-time training, which may provide insights for the method design.

Critical strengths
  1. Studying dynamic strategies for deciding which parameters to update during TTT is an interesting research direction.
Critical weaknesses
  1. The action space is limited. The paper only considers updating layers 0, 6, 12, 18, and 24, either updating a single one of these layers, updating all of them, or updating none of them. The authors do not explore updating arbitrary combinations of these layers. I think that, for a given chunk, the set of parameters that should be updated is unlikely to be restricted to the action space considered in the paper. Moreover, I do not fully understand why there should be an optimal single layer to update for a particular chunk, or what the underlying relationship is between the content of a chunk and the layer that is best to update.
  2. The main contribution consists of exploratory experiments showing that dynamic scheduling can be beneficial, but the paper does not propose any scheduling approach that can operate without access to an oracle, even a simple one. This makes the overall contribution feel somewhat limited to me.
Questions for authors

see weaknesses

3 comments

Response to Reviewer ixGy

Thank you for your review, suggestions, and requests for further clarification. Below, we provide a constant-action experiment over all 32 layer combinations, the functional motivation linking chunk content to write location, and an answer-free write rule evaluated on 6,500 fresh inputs to address each of the concerns raised. We hope these revisions and clarifications strengthen your confidence in our work and support a positive evaluation.

W1(a). Why these seven actions, and what about other layer combinations?

We have evaluated all 32 constant subsets of the five writable layers on 300 examples. Each subset is held constant throughout the input, giving 9,600 full replays. CO selects the highest-scoring constant action for each example.

Constant referenceMean score (%)
OFF36.97
CO over the original 7 actions39.58
CO over all 32 subsets42.34

The full set raises CO by 2.76 percentage points (95% CI [1.49, 4.33]). A combination outside the original seven scores higher on 16 of the 300 examples. In comparison, enumerating seven constant actions on the same 300 examples requires 2,100 full replays, compared with 9,600 for 32 actions: 7,500 additional replays, or about 4.57 times as many candidate evaluations. The expanded set thus yields a modest score gain at a substantially higher evaluation cost.

The writable sites follow Feng et al. (2026), In-Place Test-Time Training [1, Section 9.3], which places MLP write modules every sixth layer. Our checkpoint uses this layout, with five writable sites at {0, 6, 12, 18, 24}. We construct OFF, five single-layer actions and ALL to compare stopping new writes, writing to one layer and writing to all five. OFF retains existing state, the single-layer actions vary write location individually, and ALL tests joint writing.

Functional studies across depth provide further motivation: middle-layer MLPs participate in factual associations [2], and interventions on task representations have depth-dependent effects [3]. We therefore hold the model and update rule fixed, compare writes at five sites spanning depth, and use state interventions to study their effects on later computation. The seven actions place stopping, single-layer writing and joint writing in the same experiment, helping distinguish the contributions of location, timing and accumulated state.

Searching all 32 subsets shifts the focus to finding the highest-scoring configuration. When several layers change together, a combination's score pools layer contributions and their interactions. Explaining those contributions then requires breaking the combination apart and testing its components. We study how write location, timing and accumulated state affect the output, and retain single-layer actions as direct controls while using ALL to test joint writing.

Dynamic expansion additionally selects a subset at every chunk. With KK actions and TT complete chunks,

Nfixed(K)=K,Ndynamic(K,T)=∏t=1TK=KT.N_{\mathrm{fixed}}(K)=K,\qquad N_{\mathrm{dynamic}}(K,T)=\prod_{t=1}^{T}K=K^T.

Five writable layers give 25=322^5=32 subsets, expanding the full dynamic space from 7T7^T to 32T32^T. For T=2T=2, the candidate count rises from 49 to 1,024. Bounded search evaluates part of this space, with 32 action alternatives at each search position instead of seven. Each added-candidate evaluation currently involves full replay and generation, and running this search online would add forward computation. Given our research objective and this computational cost, we completed all 32 constant combinations in this revision and retained the seven-action design for dynamic experiments. The dynamic expansion exceeds the time and compute budget for this revision.

After the action-space definition in Section 2.1, we will add the layout source, the rationale for the seven actions and the candidate-space formula. A new “Action-space expansion” subsection in Appendix B will present the completed 32-combination table above and account for the 2,100 versus 9,600 full replays.

W1(b). How does chunk content relate to the layer to update?

Chunk content affects later computation through the update it produces. The selected layer determines where that update enters the model. Each chunk is first read using the weight state accumulated so far, then writes to the selected layers for use by later chunks. We therefore compare complete trajectories to examine how a chunk's write location and the preceding state affect subsequent outputs.

Two functional studies motivate this comparison. Meng et al. (2022) use causal tracing and ROME to identify a role for middle-layer MLPs in factual associations [2]. Todd et al. (2024) locate task representations in attention heads through function-vector interventions and compare their effects across depths [3]. We build on these findings by comparing write sites across depth with the model and update rule held fixed.

Trajectory comparisons measure score differences due to location and timing. Reversal changes the write schedule, while resetting and deletion remove accumulated writes. These experiments directly test how write schedules and retained state affect subsequent outputs. We will add this functional motivation after the action definitions in Section 2.1. At the start of each intervention in Sections 3, we will specify the write schedule or accumulated state being changed and the comparison conditions held fixed.

W2. Can a practical scheduling method work without an oracle?

We use the frozen Qwen3-1.7B checkpoint with the same chunking. The five writable sites are {0, 6, 12, 18, 24}. Their every-sixth-layer layout follows Feng et al. (2026), In-Place Test-Time Training [1, Section 9.3]. We introduce range and dose checks that control writes using training-document calibration and the current input's update state in a single pass. The rule has been evaluated on 6,500 fresh inputs.

  1. Range check, first chunk. For each of layers 0, 6 and 12, compare the norm of its first proposed update with that layer's 99th-percentile threshold on training documents. With sℓ(x)=∥ΔWℓ,1prop(x)∥s_\ell(x)=\lVert\Delta W^{\mathrm{prop}}_{\ell,1}(x)\rVert, the rule is
\mathrm{OFF}(x)\iff\max_{\ell\in\lbrace 0,6,12\rbrace}\bigl[s_\ell(x)-q_\ell\bigr]>0.$$ Calibration and inference use the same definition of proposed-update norm for each layer. The criterion is strictly greater than that layer's threshold, with equality passing. For example, a norm of 1.05 times the threshold exceeds it by 5% and triggers document-level OFF. The check runs before the first update is committed. If any layer triggers it, every writable layer stays OFF for that input, including layer 24. 2. **Dose check, every chunk.** After the range check passes, layers 0, 6 and 12 each compute the accumulated write that would result from adding the proposed update. Commit it when it stays within that layer's functional radius. Otherwise, skip the update and retain the existing state. The functional radius is calibrated on training documents and bounds the relative change in layer outputs. Layers 0, 6 and 12 make dynamic write decisions through these checks. After the document-level range check passes, layer 24 writes at every complete chunk and layer 18 stays disabled. Layers 18 and 24 use a fixed configuration, while actions at 0, 6 and 12 vary with proposed updates and accumulated state. At inference, write decisions use only training-document calibration and the current input's update state, without labels, reference answers, questions or oracle trajectories. The dose rule was designed on development examples from the original inputs, and some evaluation results were available during rule assembly. The table reports the descriptive evaluation from this round of rule development. | Evaluation | OFF | ALL | Rule | Rule − OFF, pp (95% CI) | Rule − ALL, pp (95% CI) | |---|---:|---:|---:|---|---| | Fresh inputs | 78.91 | 79.62 | 81.02 | +2.11 [1.65, 2.57] | +1.40 [1.04, 1.77] | We will add “Answer-free write rule” after the model and write protocol in Section 2.1. A new “Answer-free scheduling” subsection after the one-switch results in Section 5 will present the rule and table above. A new “Write-rule calibration and execution” subsection in Appendix A will provide the training-document calibration procedure and chunkwise pseudocode.

[1] Feng, G., Luo, S., Hua, K., Zhang, G., He, D., Huang, W., & Cai, T. (2026). In-Place Test-Time Training. arXiv:2604.06169v1. Section 9.3, “Details of Model Configuration”, PDF page 20. Section link: https://arxiv.org/html/2604.06169v1#S9.SS3.

[2] Meng, K., Bau, D., Andonian, A., & Belinkov, Y. (2022). Locating and Editing Factual Associations in GPT. NeurIPS 2022. arXiv:2202.05262. Sections 2.2 and 3.1. Paper: https://arxiv.org/abs/2202.05262.

[3] Todd, E., Li, M. L., Sen Sharma, A., Mueller, A., Wallace, B. C., & Bau, D. (2024). Function Vectors in Large Language Models. ICLR 2024. arXiv:2310.15213. Sections 2.3 and 3.1. Paper: https://arxiv.org/abs/2310.15213.

Reviewer_Hd1zOpenReview ↗
Rating
2
Confidence
3
Informativeness
3
Presentation
2
Soundness
3
Read review
Paper summary

This paper investigates whether dynamically varying test-time training (TTT) write actions across input chunks can outperform the best constant write action under a fixed in-place TTT checkpoint. The authors formulate adaptation as a sequence of actions that determine whether and where temporary parameter updates are applied. They introduce a per-example Constant Oracle (CO), which selects the best constant action, and compare it against a bounded, answer-aware dynamic trajectory search. On 13 RULER tasks with 6,500 examples, the dynamic search improves over the Constant Oracle by 1.39 percentage points for Qwen3-1.7B, with smaller gains on Qwen3-4B and Llama-3.1-8B.

Critical strengths
  1. A well-defined empirical investigation of sequential adaptation utility.

The paper asks a meaningful question about within-example test-time adaptation: whether changing update locations across chunks provides additional utility beyond the strongest constant action. Using a per-example Constant Oracle establishes a strong reference and avoids attributing improvements merely to choosing a better fixed adaptation configuration. The evaluation across three language models and the use of paired confidence intervals provide useful evidence that dynamic write schedules can yield additional improvements under the evaluated conditions.

  1. Careful intervention studies and useful observations about state dependence and switching complexity.

The schedule-reversal, fast-weight-reset, selective-deletion, and prefix-cache interventions provide complementary evidence that continuation depends on how adaptation is organized and on the computational state accumulated through the context. The One-Switch Oracle analysis is also interesting, suggesting that many successful dynamic schedules may admit substantially simpler alternatives.

Critical weaknesses
  1. Limited practical and scientific significance of the oracle gains without an answer-free policy.

The central results rely on selecting trajectories using the final task score, which requires access to the reference answer. Therefore, the method does not demonstrate that a deployable controller can identify beneficial write actions from observable context or adaptation-state features. Moreover, the overall gains are relatively small. For Qwen3-1.7B, only 170 of the 6,500 examples are strictly improved by the bounded dynamic search, yielding an aggregate improvement of 1.39 percentage points. The gains decrease to 0.79 and 0.36 points on the two larger model configurations.

  1. The intervention findings do not fully distinguish meaningful scheduling structure.

The main intervention analyses are conducted on examples already selected because dynamic search improves their final task scores. Consequently, the substantial decreases under schedule reversal, state reset, or selective deletion must be interpreted conditionally on this selected cohort. The equal-budget random-search control also recovers approximately 93% of the observed dynamic-oracle improvement over the Constant Oracle on the improved cohort. This suggests that much of the gain may arise from searching a larger action space with outcome feedback, rather than from a particularly informative trajectory-search strategy.

Questions for authors
  1. Is there evidence that beneficial write actions or switching boundaries can be predicted using only information available during inference, without access to the reference answer? For example, could a simple one-switch policy selected or fitted using separate validation examples retain a meaningful portion of the oracle advantage on held-out examples? Such an experiment would substantially strengthen the practical significance of the central finding.

  2. The equal-budget random-search control recovers most of the Dynamic Oracle gain on the selected improved cohort. Could the authors provide a more representative comparison of structured and random search on the full non-ceiling population, or an independently sampled subset of that population?

  3. How much of the observed intervention sensitivity reflects properties specific to the successful dynamic trajectories, rather than the general fragility of selected successful predictions to state perturbations? Can the authors clarify which conclusions remain after accounting for the magnitude-matched random perturbation control and the fact that interventions are evaluated on oracle-selected successful trajectories?

4 comments

Response to Reviewer Hd1z

Thank you for your review, suggestions, and requests for further clarification. Below, we provide an answer-free write rule evaluated, the random-trajectory mean alongside random and structured search, and paired interventions on successful trajectories from different search methods to address each of the concerns raised. We hope these revisions and clarifications strengthen your confidence in our work and support a positive evaluation.

W1. A practical write policy and the size of the gains

We use the frozen Qwen3-1.7B checkpoint with the same chunking. The five writable sites are {0, 6, 12, 18, 24}. Their every-sixth-layer layout follows Feng et al. (2026), In-Place Test-Time Training [1, Section 9.3]. We introduce range and dose checks that control writes using training-document calibration and the current input's update state in a single pass. The rule has been evaluated on 6,500 fresh inputs.

  1. Range check, first chunk. For each of layers 0, 6 and 12, compare the norm of its first proposed update with that layer's 99th-percentile threshold on training documents. With sℓ(x)=∥ΔWℓ,1prop(x)∥s_\ell(x)=\lVert\Delta W^{\mathrm{prop}}_{\ell,1}(x)\rVert, the rule is
\mathrm{OFF}(x)\iff\max_{\ell\in\lbrace 0,6,12\rbrace}\bigl[s_\ell(x)-q_\ell\bigr]>0.$$ Calibration and inference use the same definition of proposed-update norm for each layer. The criterion is strictly greater than that layer's threshold, with equality passing. For example, a norm of 1.05 times the threshold exceeds it by 5% and triggers document-level OFF. The check runs before the first update is committed. If any layer triggers it, every writable layer stays OFF for that input, including layer 24. 2. Dose check, every chunk. After the range check passes, layers 0, 6 and 12 each compute the accumulated write that would result from adding the proposed update. Commit it when it stays within that layer's functional radius. Otherwise, skip the update and retain the existing state. The functional radius is calibrated on training documents and bounds the relative change in layer outputs. Layers 0, 6 and 12 make dynamic write decisions through these checks. After the document-level range check passes, layer 24 writes at every complete chunk and layer 18 stays disabled. Layers 18 and 24 use a fixed configuration, while actions at 0, 6 and 12 vary with proposed updates and accumulated state. At inference, write decisions use only training-document calibration and the current input's update state, without labels, reference answers, questions or oracle trajectories. The dose rule was designed on development examples from the original inputs, and some evaluation results were available during rule assembly. The table reports the descriptive evaluation from this round of rule development. | Evaluation | OFF | ALL | Rule | Rule − OFF, pp (95% CI) | Rule − ALL, pp (95% CI) | |---|---:|---:|---:|---|---| | Fresh inputs | 78.91 | 79.62 | 81.02 | +2.11 [1.65, 2.57] | +1.40 [1.04, 1.77] | We will add “Answer-free write rule” after the model and write protocol in Section 2.1. A new “Answer-free scheduling” subsection after the one-switch results in Section 5 will present the rule and table above. A new “Write-rule calibration and execution” subsection in Appendix A will provide the training-document calibration procedure and chunkwise pseudocode. The original study measures the gain from dynamic writing over each input's best constant action. CO already selects the highest-scoring constant action for each input. Dynamic trajectories still achieve a gain of 1.39 percentage points over 6,500 examples, with 170 improved examples. This gain comes from varying the write schedule within the input.

W2. Random trajectories, random search and structured search

We computed the random-trajectory mean on the original 170 examples by first averaging all evaluated random trajectories within each example, then averaging those means with equal weight across examples. The table places this mean alongside random-search selection and structured search.

ComparisonMean score (%)
Best constant, CO40.41
Random-trajectory mean42.34
Best of random search89.96
Dynamic (DO-LB)93.53

Dynamic exceeds the random-trajectory mean by 51.19 percentage points (95% CI [49.61, 52.81]), scoring above that mean on all 170 of 170 examples. Executing many candidates and selecting them by reference-answer scores raises random search from a trajectory mean of 42.34 to 89.96. The gap shows that searching for and selecting high-quality trajectories is a key step in achieving high scores.

Appendix B.3 further compares the two searches at the same total evaluation budget. Each uses 62,772 fresh, unique nonconstant trajectory evaluations in total, with the same checkpoint, action space, execution protocol, decoding and scorer. Structured search scores 3.57 percentage points higher (95% CI [1.45, 6.04]), tying on 159 examples and scoring higher on the remaining 11. It finds higher-scoring trajectories at the same total evaluation count. Appendix B.3 will present the random-trajectory mean, random-search selection and structured-search results together, including the 51.19 pp paired difference and its interval. We will reference this table after the search results in Section 2.

Q1. Inference-time signals and a simple one-switch policy

Following your suggestion, we designed and evaluated a simple one-switch rule. Chosen on development examples, it writes layers {0, 12, 24} and adds layer 6 at the last complete chunk. It scores 80.02 on fresh inputs, improving on OFF by 1.12 pp [0.58, 1.65]. The corresponding constant policy {0, 6, 12, 24} scores 79.69, giving a difference of +0.34 pp [−0.05, 0.72]. The additional improvement over the corresponding constant policy is therefore modest.

The second experiment uses the two checks in W1 to choose actions directly from inference-time update statistics. Comparing the first proposed update with its training threshold determines whether writing proceeds for the input. Comparing each layer's accumulated write with its calibrated radius determines whether to commit the current update. The rule computes these statistics and acts at each chunk. On the same fresh inputs, it scores 81.02, compared with 78.91 for OFF and 79.62 for ALL. It improves on OFF by 2.11 pp [1.65, 2.57] and on ALL by 1.40 pp [1.04, 1.77]. A new “Answer-free scheduling” subsection in Section 5 will present both completed experiments: the one-switch rule with its constant reference, and the two-check rule against OFF and ALL.

Q2. Search budget and the random control

Search budgets are matched by the total number of unique candidate-trajectory evaluations. Random trajectories are drawn without replacement from the nonconstant space, and the full budget is executed. Both searches use the same execution and scoring procedure. The 3.57 pp difference compares their results at the same total evaluation count.

The score of 42.34 in W2 uses all evaluated random trajectories on the same 170 examples. Averaging within examples and then across examples gives each example equal weight. The score of 42.34 is the trajectory mean, and 89.96 is the score after random-search selection with CO as a fallback. Appendix B.3 will give the mean definition, total budget and both results together, labelled “Random-trajectory mean” and “Random-search selection”.

Q3. State dependence and general perturbation sensitivity

We added two sets of controls: the same interventions on successful trajectories found by different searches, and score-drop comparisons between targeted state removal and matched random perturbations.

Paired trajectories from different search methods

Under the paper's native writer and execution settings, we paired each DO-LB trajectory with the random-search winner on the same input. We retained pairs with equal original scores and applied the same intervention. The table reports score drops from each trajectory's original score.

InterventionPairsDO-LB trajectory dropRandom-search winner dropDrop difference, pp (95% CI)
Norm-matched random perturbation14634.0631.54+2.52 [−0.28, 5.36]
Reverse15952.0049.09+2.91 [−0.57, 6.60]
Reset15446.1039.91+6.19 [0.82, 11.63]

Successful trajectories from both searches suffer substantial score drops when the write schedule is reversed or accumulated state is cleared. State dependence appears in both sets of successful trajectories, showing that accumulated writes contribute to subsequent computation.

Loss beyond matched random perturbations

We further compared targeted operations with matched random perturbations. In the paper's 157-example Deletion experiment, magnitude-matched random perturbations cost 34.40 pp [31.50, 37.25], while the total loss from targeted deletion is 38.94 pp. The two controls on DO-LB trajectories give the following excess losses.

OperationExcess loss over matched random perturbations, pp (95% CI)
Deletion+4.54 [1.67, 7.35]
Reset+4.12 [1.48, 6.68]

Targeted removal of accumulated writes causes larger score drops than the matched random perturbations, showing that these successful trajectories use the previously written state. We will add the cross-search paired table after the reversal analysis in Section 3 and the excess-loss table after the reset and deletion results in Section 3.2. The corresponding experimental appendices will give the pairing definitions and intervention steps.

Reference

[1] Feng, G., Luo, S., Hua, K., Zhang, G., He, D., Huang, W., & Cai, T. (2026). In-Place Test-Time Training. arXiv:2604.06169v1. Section 9.3, “Details of Model Configuration”, PDF page 20. Section link: https://arxiv.org/html/2604.06169v1#S9.SS3.

Thanks for the very thorough rebuttal and additional experiments. The new answer-free write policy, as well as the state intervention controls, do address some of my concerns. However, I still find that the proposed policy is not compared to a strong constant-action baseline in independent fashion, and Random Search recovers only approximately 93% of the Dynamic Oracle gains on the few examples for which results are provided. While I appreciate these additional results, I will maintain my original rating.

Reviewer_HTD2OpenReview ↗
Rating
1
Confidence
2
Informativeness
1
Presentation
1
Soundness
2
Read review
Paper summary

This paper discovers that the improved performance of test-time training (TTT) for LLMs can be through dynamic write actions including the ordering of write actions and the model state accumulated across chunks. The dynamic write action can be better than the constant write action for In-Place TTT. The experimental results and analysis support the findings of this paper.

Critical strengths

The experiments are well controlled to verify the influence of dynamic write actions, fast weights, and prefix key-value cache on the In-place TTT.

Critical weaknesses
  1. The experiments depend on the oracle trajectory upper bound pre-obtained. The paper provides no mechanism to learn it.
  2. The paper is poorly structured and difficult to follow.
Questions for authors
  1. For the Dynamic Oracle Lower Bound (DO-LB), what impact would different action spaces have?
  2. What does one action switch mean in the one-switch oracle (OSO)?
  3. How does an LLM learn the policy without the oracle ground-truth write actions?
5 comments

Response to Reviewer HTD2

Thank you for your review, suggestions, and requests for further clarification. Below, we provide a clarification of the oracle analysis, a constant-action experiment over all 32 layer combinations, a worked one-switch example, and an answer-free write rule evaluated to address each of the concerns raised. We will also consolidate the definitions of CO, DO-LB and one-switch schedules in the revision. We hope these revisions and clarifications strengthen your confidence in our work and support a positive evaluation.

W1. Oracle analysis and how a policy is obtained

DO-LB records the dynamic score achieved through search and actual execution. For each input, CO is the best score among the seven constant actions, and D∗D^{\ast} is the best score over all dynamic trajectories in the same action space. Bounded search retains all constant candidates, so

CO(x)≤DO-LB(x)≤D∗(x).\mathrm{CO}(x) \leq \mathrm{DO\text{-}LB}(x) \leq D^{\ast}(x).

DO-LB is an achieved lower bound on the optimal dynamic score. Each candidate trajectory is executed and produces an output before reference-answer scoring and selection. The forward computation uses the input and the state accumulated chunk by chunk. Reference answers enter only the post-generation evaluation.

With the checkpoint, update rule and action space held fixed, comparing dynamic trajectories with CO tests the gain from varying writes within an input. State interventions test how accumulated writes affect later computation. The one-switch experiment tests how far successful trajectories can be simplified.

We also completed an experiment with a rule calibrated on training documents that selects actions directly at inference. Q3 gives its procedure and results.

W2. Structure and readability

Thank you for raising this point. Our paper is a mechanistic study, not a methods paper. It holds the checkpoint, update rule and action space fixed, first establishes an effect, and then uses controlled interventions to identify what the effect depends on. This follows the organization of the mechanistic studies we build on. Meng et al. [2] first localize factual recall with causal tracing and then test the localized MLPs through targeted edits. Todd et al. [3] first identify function vectors and then test them through interventions at different depths. Our paper follows the same sequence. Section 2 establishes the score gap between dynamic trajectories and the best constant action. Section 3 intervenes on the write schedule and the carried fast weights through Reverse, Reset and Selective Deletion. Section 4 matches the boundary fast weights and replaces only the prefix cache, an intervention in the spirit of activation patching. Section 5 asks how much switching complexity is needed to reproduce the observed scores.

Because the comment does not point to a specific passage, we were not certain which part was hardest to follow. We would be grateful if you could indicate the section, definition or figure you had in mind, and we will revise it directly. In the meantime, we re-read the paper from the perspective of a first-time reader and identified four places where readers are likely to lose track.

1.Oracle terms are introduced in different places and named inconsistently. CO and DO-LB are defined in Section 2.1, while OSO first appears in Section 5.1. The selected dynamic trajectory is called “DO” in Figure 1, “DO-LB” in Table 1 and “Dynamic Oracle” in Sections 3 to 5. We will add one definition table at the end of Section 2.1 that gives each term’s candidate set, selection rule and purpose, and we will use one name for each quantity throughout. For example, the table will state that CO executes the seven constant actions and keeps the best score. DO-LB keeps the best score from a bounded search that retains all constant candidates, so CO ≤ DO-LB. 2.Each analysis uses a different cohort. Section 2 reports all 6,500 examples and the 1,933 non-ceiling examples, Reverse and OSO use the 170 improved examples, Reset and the prefix-cache analysis use 165, and Selective Deletion uses 157. The lineage is recorded in Table 4 of Appendix C, but the main text first points to it only at the end of Section 3.2. We will move this table into Section 2.2 so that readers see each cohort before the analyses that use it. 3.Key formulas lack a worked instance. The one-switch family in Eq. (8) and the cache branches in Eq. (7) are stated symbolically, and Figure 1 shows only abstract A/B patterns. We will add a concrete instance after each formula. For Eq. (8), over six chunks, L6 → L6 → L6 → ALL → ALL → ALL writes only to layer 6 for the first three chunks and to all five writable layers for the last three. The state written in the first segment carries into the second. For Eq. (7), we will add one sentence stating that both branches enter the suffix with the same fast weights and differ only in which history produced the prefix cache. 4.Scope statements interrupt the results. Sections 3.2, 4.2 and 5.2 each end with several sentences on what the result does not establish. These statements are necessary, but they break the line of the argument. We will open each results subsection with a one-sentence main finding and collect the scope statements in a short closing paragraph.

Revision location. The definition table will appear at the end of Section 2.1, and the cohort table will appear in Section 2.2. Worked instances will follow Eqs. (7) and (8). Sections 3.2, 4.2 and 5.2 will open with the main finding and close with the scope statements.

Q1. How does the action space affect DO-LB?

We have evaluated all 32 constant subsets of the five writable layers on 300 examples. Each subset is held constant throughout the input, giving 9,600 full replays. CO selects the highest-scoring constant action for each example.

Constant referenceMean score (%)
OFF36.97
CO over the original 7 actions39.58
CO over all 32 subsets42.34

The full set raises CO by 2.76 percentage points (95% CI [1.49, 4.33]). A combination outside the original seven scores higher on 16 of the 300 examples. In comparison, enumerating seven constant actions on the same 300 examples requires 2,100 full replays, compared with 9,600 for 32 actions: 7,500 additional replays, or about 4.57 times as many candidate evaluations. The expanded set thus yields a modest score gain at a substantially higher evaluation cost.

The writable sites follow Feng et al. (2026), In-Place Test-Time Training [1, Section 9.3], which places MLP write modules every sixth layer. Our checkpoint uses this layout, with five writable sites at {0, 6, 12, 18, 24}. We construct OFF, five single-layer actions and ALL to compare stopping new writes, writing to one layer and writing to all five. OFF retains existing state, the single-layer actions vary write location individually, and ALL tests joint writing.

Functional studies across depth provide further motivation: middle-layer MLPs participate in factual associations [2], and interventions on task representations have depth-dependent effects [3]. We therefore hold the model and update rule fixed, compare writes at five sites spanning depth, and use state interventions to study their effects on later computation. The seven actions place stopping, single-layer writing and joint writing in the same experiment, helping distinguish the contributions of location, timing and accumulated state.

Searching all 32 subsets shifts the focus to finding the highest-scoring configuration. When several layers change together, a combination's score pools layer contributions and their interactions. Explaining those contributions then requires breaking the combination apart and testing its components. We study how write location, timing and accumulated state affect the output, and retain single-layer actions as direct controls while using ALL to test joint writing.

Dynamic expansion additionally selects a subset at every chunk. With KK actions and TT complete chunks,

Nfixed(K)=K,Ndynamic(K,T)=∏t=1TK=KT.N_{\mathrm{fixed}}(K)=K,\qquad N_{\mathrm{dynamic}}(K,T)=\prod_{t=1}^{T}K=K^T.

Five writable layers give 25=322^5=32 subsets, expanding the full dynamic space from 7T7^T to 32T32^T. For T=2T=2, the candidate count rises from 49 to 1,024. Bounded search evaluates part of this space, with 32 action alternatives at each search position instead of seven. Each added-candidate evaluation currently involves full replay and generation, and running this search online would add forward computation. Given our research objective and this computational cost, we completed all 32 constant combinations in this revision and retained the seven-action design for dynamic experiments. The dynamic expansion exceeds the time and compute budget for this revision.

After the action-space definition in Section 2.1, we will add the layout source, the rationale for the seven actions and the candidate-space formula. A new “Action-space expansion” subsection in Appendix B will present the completed 32-combination table above and account for the 2,100 versus 9,600 full replays.

From OpenReview, CC BY 4.0. Authors as signed there; shown unedited.

Related papers

Loading the map…

Discussion (0)

Sign in to comment.