acceptodds
Under review as a conference paper at ICLR 2027

Computation, Not Memory: Chain-of-Thought and State Tracking in LLM Agents

Abstract

Language-model agents often reason before they act, and the value of monitoring that reasoning depends on how it relates to their actions. We ask how an agent's actions depend on its reasoning when the correct action depends on a changing state. In a small file-system task, background events announce which file's lock has been toggled but not its new value. The agent therefore has to reconstruct the current locks from its history before it can move files safely. With Qwen3-8B and Qwen3.5-9B, we separate two roles of reasoning: generating it before each action, and retaining earlier reasoning in the prompt. Most of the reduction in lock-violating commands comes from reasoning written at the moment of action. Retaining earlier reasoning adds little to rule-following, although it helps one model complete the task. The benefit depends on what the reasoning says. Filler text, reasoning written for another history and an accurate table of the state do not reproduce it. When we give the agent reasoning written for a history that differs in one lock value, the agent acts on that reasoning even against its own history. Without reasoning, linear probes recover the state from the model's activations about as well as a text classifier recovers it from the transcript. However, decoding accuracy does not predict which interventions change the agent's actions. In this setting, the benefit of reasoning for rule-following comes mainly from generating it at each step. The state that can be decoded without reasoning is a limited guide to how the agent acts.

Primary area:
interpretability and explainable AI
Keywords:
mechanistic interpretability, large-language models, agents, chain-of-thought

Reviews

Among:
Shape:
1234this paper, Top 49%
Figure 1. Mean review rating of the reviewed papers at ICLR 2027. This paper averages 2.33 over 3 reviews, above 51% of the others.
Reviewer_3zNNOpenReview ↗
Rating
2
Confidence
3
Informativeness
1
Presentation
2
Soundness
2
Read review
Paper summary

This paper studies the role of CoT in the state tracking task of LLM Agent, and mainly distinguishes the contribution of Computation and Memory. The authors build a file-locking state tracking environment, and through comparing different CoT generation and retention conditions, they find that current generated CoT can significantly reduce the violation rate, while the extra benefit from retaining historical CoT is limited. In addition through experiments such as Reasoning Swap, Linear Probing and Activation Patching, it analyze the relationship between CoT content, internal state representation and Agent behavior, and find that CoT content can affect subsequent actions, but the decodability of state information does not necessarily reflect its causal effect on behavior.

Critical strengths
  1. The authors employ Linear Probing and Activation Patching to examine the relationship between state representations and model behavior from various perspectives, providing a comprehensive analysis.

  2. The experimental setup is well-designed; the authors evaluate different model scales, various conditions for retaining information during inference, and longer task sequences, while also reporting confidence intervals.

Critical weaknesses

The primary issue with this paper lies in its experimental design. The paper’s central claim is that computation is more valuable than memory; however, the experimental design compares only direct/cmd, think/cmd, think/body, and think/full, omitting the direct/full condition. This is surprising, as the current experiments only demonstrate that when a model engages in "re-thinking," the original Chain-of-Thought (CoT) may offer little additional value. Conversely, when the model does not re-think, the original CoT could still be valuable and lead to lower violation rates. Consequently, the existing experiments are insufficient to support the core conclusion that computation is more significant than memory.

Furthermore, although the authors report high rates of generation truncation in the appendix (42.7% in some experiments and 28.7% in others), this issue is not adequately addressed in the main text. Given that the paper’s central conclusion relies on CoT generation, such a high degree of truncation may compromise the comparability of different experimental conditions. The authors should explicitly discuss the impact of this truncation and provide a robustness analysis to account for it.

Finally, the claim that the specific content of a CoT influences a model’s subsequent behavior offers limited novelty. While the authors demonstrate—through intervention experiments—that an erroneous CoT can indeed affect an agent’s actions, this phenomenon is not particularly surprising. The paper fails to clearly articulate what new, mechanistic insights these findings provide beyond what is already established in existing literature.

Questions for authors
  1. Why were the high generation truncation rates not sufficiently discussed in the main paper? Could they affect the main experimental conclusions?

  2. Why was the direct/full condition not evaluated? This seems to be a natural and important baseline for distinguishing the effects of current reasoning generation and historical reasoning memory.

3 comments

Thank you for your time and constructive feedback. Your missing-condition point led to the most informative new experiment.

"Why was the direct/full condition not evaluated?"

A direct rollout writes no reasoning to retain, so we built these conditions from saved decisions. We took 400 decisions per model from reasoning rollouts, kept observations and commands fixed, and retained earlier replies as cmd, body or full. Then we generated the next command directly or after new reasoning (Section 4, Table 1). Violations per 100 decisions:

Model, modecmdbodyfull
Qwen3-8B, direct5.251.501.25
Qwen3-8B, think0.500.250.00
Qwen3.5-9B, direct9.751.001.25
Qwen3.5-9B, think1.000.750.75

You were right: retained reasoning helps a lot when the agent does not reason (direct/full versus direct/cmd). Its benefit is much smaller once the agent reasons. We now state the claim this way: computation at the current decision gives most of the benefit for an agent that reasons at each step, and retention substitutes only in part. The two routes also differ: direct/full avoids violations partly by moving less (24 against 67 safe log-file moves for Qwen3.5-9B; Appendix E). At 16 turns, full retention also helps an agent that reasons (2.8 points [0.6, 5.1]; Appendix D).

Section 5 shows why retention cannot replace current reasoning. After a new toggle makes a correct note wrong, an agent with notes but no current reasoning responds little. Current reasoning makes the action follow the event (difference in signed response 0.26 [0.16, 0.35] and 0.21 [0.14, 0.28]; Figure 3).

"Why were the high generation truncation rates not sufficiently discussed in the main paper? Could they affect the main experimental conclusions?"

They do not change the conclusions. Section 3 now defines forced closure: when reasoning reaches its token budget, we close the thought block and ask for the command. In the main runs, it affects 2.0 to 6.9% of reasoning turns, and excluding those turns leaves the generation effect unchanged (Appendix C). The 42.7% and 28.7% rates came from secondary runs with a smaller budget. We reran the goal-file, 16-turn and scale runs with the main-run budget. The 28.7% run duplicated the main run's seeds with less budget, so we removed it. Appendix R lists the budget, the share of turns at the budget, maximum tokens and GPU time for every run.

"The paper fails to clearly articulate what new, mechanistic insights these findings provide beyond what is already established in existing literature."

We agree that changing an answer by changing its reasoning is not itself new. Our contribution is to connect reasoning interventions to a state whose current value can be checked independently.

  • Retention versus new events. Retained reasoning reduces violations of an agent that does not reason, but current reasoning gives a larger response when a new event changes the state (Sections 4 and 5).
  • The action uses the supplied trace despite a conflicting history. In Qwen3-8B's toggled pairs, reasoning from the unlocked version produces 39 moves of the currently locked file, against 4 with its own reasoning, and the history changes the move in at most two pairs (Section 7).
  • State-related information versus the current state. Strong patch recovery remains on pairs where both direct outputs favor the outdated value, and state answers and actions depend on different prompt positions (Section 8).

Together, these results connect the behavioral benefit of current reasoning, its influence on action selection, and the limits of using state readouts to explain behavior. They do not locate where the state is first computed.

Thank you again for your time and feedback, which led directly to the new experiments. We hope these answers resolve your questions about the missing condition, truncation and novelty.

Thank you for the additional experiments and clarifications. They largely address my original concerns about the experimental setup.

My remaining concern is about the paper's central contribution. Since the full observation and action history is always available, the experiments seem to compare generating new reasoning with retaining previous reasoning, rather than computation and memory more broadly. The finding that current reasoning is more effective when the state changes seems somewhat expected under this setup. While the new experiments make this distinction clearer, I am still unsure what fundamentally new insight they provide into how CoT works, beyond demonstrating its behavioral benefits in this task. I would have expected a clearer explanation of what makes current reasoning effective and what broader understanding of CoT in agent decision-making we can draw from these findings.

Based on the current situation, I tend to keep the score unchanged.

Thank you for the follow-up, and we are glad that the new experiments address your concerns about the setup. We agree that we compare current and retained reasoning while the full history stays in the prompt, and Sections 1 and 9 state this scope. A retained note can go out of date after a state change, and Section 5 tests how the agent responds when this happens. However, we think our results show more than the behavioral benefit of CoT. Most of our experiments change what the agent reads or writes, and they show which content the action depends on and where current reasoning fails. Three findings in particular are not predicted by the setup.

After a state change, retained notes help only when the agent reasons over them. Without current reasoning, retained notes do make the agent follow an outdated value. After a toggle locks a file that its note calls unlocked, retaining the notes raises Qwen3-8B's unsafe moves from 10 to 31 of 75. With current reasoning, the same notes instead raise its response to the toggle by 0.14 [0.04, 0.24] (Section 5). Memory and computation therefore work together here: the notes are useful as input to reasoning at the decision.

Current reasoning fails at a specific step, and labels help only when the agent reasons. In all 21 remaining unsafe moves of Qwen3-8B with reasoning, the same file had been toggled earlier, and the trace concluded that it was unlocked. When we label each announcement as a separate event, unsafe moves with reasoning fall by half (26 to 13 of 75, with both wordings rerun in the same session), while the direct agent shows no clear change (Section 5; Appendix I.2). The traces that switch to a safe command list both toggles and conclude that the file is locked again. The reasoning thus updates the state by combining the events in the history, and its remaining errors are consistent with difficulty telling repeated events apart.

Reasoning helps even when no update is needed. At files whose lock never changed, reasoning lowers violations from 12.8% to at most 0.4% of attempts for Qwen3-8B and from 10.1% to at most 1.7% for Qwen3.5-9B (Appendix C.5). A correct state table before the command does not reproduce this benefit (Section 6). Editing only the lock statements in a finished trace shifts the model's answer about the lock but changes the move in one pair, while swapping the whole trace redirects the action (Section 7; Appendix H). As in our reply to reviewer jvFL, current reasoning therefore does more than track the state. It connects the state to the commands the task allows, and the action follows the trace's conclusions.

For CoT in agent decision-making, these results point to two roles of current reasoning: combining events into the current state, and applying the task rules to that state before the action. Retained reasoning supports these steps but does not replace them. Because the action follows the trace's conclusions, a trace should be evaluated by how those conclusions affect the next command, and not only by whether the state it reports is correct.

We hope these points clarify the contribution of the paper, and we would be grateful if you would take them into account in your final assessment. Thank you again for your time and for the detailed feedback throughout the discussion.

Reviewer_EHctOpenReview ↗
Rating
3
Confidence
2
Informativeness
3
Presentation
3
Soundness
3
Read review
Paper summary

This paper investigates whether CoT helps LLM through additional computation or by retaining previous reasoning as memory. This is an interesting study and well organized.

Critical strengths

The paper studies an interesting and important question. The experimental design is clear and well motivated, with controlled comparisons of reasoning generation and persistence.

Critical weaknesses
  1. The thinking and non-thinking settings use very different token budgets, so how can we tell whether the improvement comes from CoT itself or simply from more computation?

  2. The task has only 8 turns, and the full history is always available. Would previous CoT become more useful in longer tasks where the agent needs to remember past information?

  3. In the CoT swapping experiments, both the reasoning and action plans are changed. Is it possible to identify which part actually influences the agent's actions?

  4. The task involves only four files and simple state changes. Would the same conclusions hold for more realistic agent tasks and other model families?

Questions for authors

see weakness

2 comments

Thank you for the positive assessment and the specific questions.

"how can we tell whether the improvement comes from CoT itself or simply from more computation?"

We do not isolate the effect of additional computation from the effect of generating meaningful reasoning, and Section 9 states this limitation. Our controls address a narrower question: whether arbitrary added text or explicit state information reproduces the benefit.

  • Filler with as many tokens as the model's own reasoning, and reasoning from another history, do not reproduce the reduction in violations per attempt (Section 6, Table 2).
  • An accurate state table gives only a partial reduction per attempt and no reduction per prompt.
  • For Qwen3-8B, doubling the reasoning budget leaves the benefit similar (Appendix F).

These results support a role for the content of the reasoning, while a compute-matched comparison remains open.

"Would previous CoT become more useful in longer tasks where the agent needs to remember past information?"

Yes. We ran all four Qwen3-8B conditions for 16 turns with seven toggles and the main-run budget (Section 4; Appendix D, Table 7):

ConditionViolations per eligible attempt
direct/cmd28.5%
think/cmd4.6%
think/body4.8%
think/full1.7%

Full retention lowers violations by 2.8 percentage points [0.6, 5.1], against 0.5 at eight turns, so earlier reasoning becomes more useful in longer episodes even when the agent reasons. Excluding force-closed turns leaves this unchanged. All observations still remain in the prompt, so memory after old observations are removed is left for future work.

"Is it possible to identify which part actually influences the agent's actions?"

We answer this partially. When we edit only the explicit lock statements in the model's own reasoning, the state answer shifts but the move changes in only one pair (Appendix H). The edit leaves the trace's conclusions and plan in place, so the action appears to rely on that content rather than on single state statements. In retained notes, sentences with action words reduce violations more than sentences that only state a lock value, clearly so for Qwen3.5-9B (Appendix G, Table 10). One author manually checked these word-based groups against the meaning of 40 sentences and agreed on 36.

"Would the same conclusions hold for more realistic agent tasks and other model families?"

We chose a small environment so that the true state and every action can be checked, and histories can be edited with known ground truth. Within this setting, the generation benefit holds at every Qwen3 size from 0.6B to 14B (Appendix D, Figure 11) and in two models with different attention architectures. Both models come from one developer, and Section 9 lists transfer to open-ended tasks and other families as a limitation.

Thank you again for your time and insightful feedback. We hope these answers address your questions.

Thank you for the clarifications and additional experiments. I maintain my score

Reviewer_jvFLOpenReview ↗
Rating
2
Confidence
3
Informativeness
2
Presentation
3
Soundness
2
Read review
Paper summary

The paper attempts to explain the role of CoT reasoning in a state-tracking task, focusing on the distinction between generating reasoning at each step and retaining previous reasoning in the context. The authors introduce a simple file-system environment in which an agent must track changing lock states to decide which files can be safely moved. They find that generating CoT substantially improves rule-following, while retaining previous reasoning has a much smaller effect. The authors also show that the agent's actions are strongly influenced by the content of its CoT, even when it contradicts the actual history. Finally, they use probing and activation patching to study how the model represents and uses state information without CoT.

Critical strengths
  • The task is simple and well-controlled, and it therefore allows to directly study the effect of CoT on actions.

  • The experiments seem thorough, with several controls and interventions on the reasoning traces.

  • I appreciated the counterfactual reasoning swaps, which show that the model's actions can follow its CoT even when it contradicts the actual history.

Critical weaknesses

I'm not sure what new insights about the role of CoT I gained from reading the paper:

  • The paper doesn't explain why CoT helps: The authors show that generating CoT improves performance, but don't investigate what computation is responsible for this improvement. Does CoT help the model track the correct state, or does it help the model act on information it already has? The fact the CoT blocks improves reasoning is neither new nore surprising.

  • The comparison between generating and retaining CoT seems problematic to me: Qwen models discard previous reasoning blocks by default, so it's unclear whether these models were ever trained to make use of earlier reasoning. The limited benefit of retaining CoT might stem from the training setup, rather than show something about CoT itself.

  • The probing results are limited. The distinction between decodability and causal relevance is well established, including in previous work (eg thello). The finding that mean-difference directions have stronger causal effects than linear probe directions has also been reported before. I'm not sure what additional insight these experiments provide.

  • The counterfactual experiments don't actually study changing states(!). In Sections 6–7, the histories differ in a single initial lock value, which is never subsequently toggled. These experiments therefore examine how the model uses a fact provided in the context, rather than how it tracks changes in state. This is an important limitation given the paper's main motivation.

Questions for authors

Please see weaknesses

3 comments

Thank you for your time and valuable feedback. Your questions about tracking versus acting, and about static counterfactuals, shaped the revision.

"The paper doesn't explain why CoT helps ... Does CoT help the model track the correct state, or does it help the model act on information it already has?"

Both, and we separate them in the revision.

  • Tracking. Without reasoning, Qwen3-8B's answer about a lock that has been toggled since it was written has AUROC 0.28: it favors the old value. Its own reasoning raises this to 0.77 (Section 8, Table 4). When a new toggle makes a retained note wrong, current reasoning makes the action follow the event (Section 5). Labelling each event's position halves the remaining unsafe moves of Qwen3-8B when it reasons, consistent with difficulty in telling repeated announcements apart (Appendix I.2, Table 17). Current reasoning therefore improves the updating of the state.
  • Acting. In 106 of 112 direct Qwen3-8B log-file violations, the target was last explicitly observed as locked and had not been toggled since. An accurate state table also fails to reproduce the benefit of reasoning (Sections 6 and 8). An unsafe action therefore need not come from an unobserved or outdated lock value.

Our contribution is not that CoT helps. It is where the benefit comes from (reasoning at the decision rather than retained notes) and how it affects the action (it makes the action respond to new events, and the action follows what it says).

"Qwen models discard previous reasoning blocks by default, so it's unclear whether these models were ever trained to make use of earlier reasoning."

We agree that retained thought blocks are unfamiliar to the models. The paper's body condition, the template default, keeps only the earlier replies, and in rollouts it adds little beyond cmd when the agent reasons. The new test shows that this is not because the models cannot use earlier text. Without current reasoning, body retention cuts violations from 5.25 to 1.50 and from 9.75 to 1.00 per 100 decisions, and adding the thought blocks (full) changes little (Section 4, Table 1). The models use retained replies well. Retention adds little in the eight-turn rollouts because the agent already reasons at each step; without current reasoning, the same notes help a lot.

"The probing results are limited. The distinction between decodability and causal relevance is well established ..."

We agree that decodability alone does not establish causal relevance, and the mean-difference comparison is now only in Appendix M. Section 8 instead asks whether state readouts and causal dependencies reflect the current state after an update. A decodable state or a strong causal dependence does not by itself establish that the action uses the current state. Qwen3-8B's direct state-answer AUROC falls from 0.99 when the last explicit value is still current to 0.28 after a toggle. Its probes also become less reliable (0.47 to 0.69), although they do not uniformly favor the outdated value. On toggled pairs where both direct outputs favor the outdated value, patching still recovers 0.81 and 0.86 of the action contrast in the two models. The action depends on history positions, while the state answer depends on question positions (Appendix P). This is why the update and swap experiments are needed alongside the readouts.

"The counterfactual experiments don't actually study changing states(!)."

Thank you; we added this test. In the new pairs, the two versions differ in a file's opening lock, and the history then toggles that file once (Section 7, Table 3; Appendix H.1). Without reasoning, Qwen3-8B follows the outdated opening value: it moves the currently locked file in 17 pairs and the currently unlocked file in 4. With its own reasoning, these become 4 and 37. With reasoning written for the unlocked version, it moves the locked file in 39 pairs, and the history changes the move in at most two pairs. The action therefore depends strongly on the supplied reasoning, even when the correct state must be reconstructed from a toggle. Qwen3.5-9B's reasoning also removes most unsafe moves (98 to 7), but swaps have little effect on it, and we report this difference.

Thank you again for your time and detailed feedback. We hope these answers and the new experiments address your concerns.

The new experiments help establish that current reasoning improves both state updating and action selection. However, I am still unsure what they tell us about the computations responsible for these improvements. In particular, the observation that most direct-action violations occur even when the correct lock value is explicitly available suggests that state tracking is not the dominant source of errors. Can the authors clarify what conclusions we can draw about why CoT helps, beyond establishing that its content influences subsequent actions?

Thank you for this question. It helps us state the main takeaway more clearly. We agree with your reading that most direct violations happen even when the correct lock value is available, so recovering the state cannot be the whole explanation. Our results support the interpretation that current reasoning connects state information to the constraints on the next action, and that the action then depends on the resulting trace. Below we summarize the evidence:

Having the state does not ensure using it. A correct state table placed right before the command lowers the violation rate per attempt, but the model still often moves files listed as locked in the table (Section 6, Table 2). So an explicit or correctly reported lock value does not mean that command generation uses it appropriately. Our matched patching experiment also reinforces this distinction. It shows that on identical history prefixes, the state answer and the action score depend on different positions (Section 8; Appendix O, Figure 25).

Reasoning substantially closes this gap, even where no update is needed. At files whose lock has not changed since the first observation, reasoning lowers violations from 12.8% and 10.1% of attempts to at most 0.4% and 1.7% (Appendix C.5, Figure 9). Before the first toggle is announced, the direct agent never moves a locked file. The agent starts moving these never-toggled locked files only after toggles of other files are announced. This suggests that announcements about other files make the direct agent more likely to move a file whose lock has not changed. With reasoning, the remaining violations are concentrated at files that were toggled at some point. In the Qwen3-8B update cases that we inspected, the remaining failures came from wrong conclusions about the current lock (Appendix I.1).

Actions appear to follow the trace's conclusions. By only editing the explicit lock statements in the model's own reasoning, the state answer shifts but it changes the move in only one pair, while swapping the whole trace makes Qwen3-8B move the locked file in 43 of 114 pairs (Section 7; Appendix H). The edits leave the trace's conclusions about what to do unchanged, and the action appears to follow these conclusions. Our ablation experiment on retained notes also points the same way. We found that sentences that combine a file's state and an action reduce direct violations more than sentences that only state the lock (Qwen3.5-9B: 8 against 24 violations in 400 decisions; exploratory, Appendix G, Table 10). This improvement does not come with a better state readout (AUROC 0.81 for mixed sentences against 0.84 for state-only sentences).

Together, the results support two contributions of current reasoning: it improves state updates after events (Sections 5 and 8), and it helps translate available state information into rule-consistent actions. These interventions narrow down the explanation for the benefit of CoT, although they do not identify its internal implementation. We will add this synthesis to the discussion (Section 9).

Thank you again for engaging so closely with our work.

Comments on the paper

2 comments
AuthorsOpenReview ↗

Summary of Revisions

We are grateful to the reviewers for their careful reading and helpful comments. We answer each point under the corresponding review and have uploaded a revised manuscript. Below, we summarize the changes that concern several reviewers.

Retention without current reasoning and in longer episodes (3zNN, jvFL, EHct). We added the missing direct/body and direct/full conditions by completing 400 saved decisions per model with different retained text, either directly or after new reasoning (Section 4, Table 1; Appendix E). Without current reasoning, retained reply notes cut violations from 5.25 to 1.50 (Qwen3-8B) and from 9.75 to 1.00 (Qwen3.5-9B) per 100 decisions; with current reasoning, retention adds little. We also ran all four retention conditions at 16 turns, where full retention lowers violations by 2.8 percentage points even when the agent reasons (Appendix D).

Changing states (jvFL, EHct, 3zNN). We added a test in which a new toggle makes a correct retained note wrong. With retained notes, current reasoning raises the action's response to the new event by 0.26 and 0.21; it also helps without notes (0.21 and 0.16; Section 5, Figure 3). Labelling each event's position halves the remaining unsafe moves of Qwen3-8B when it reasons, with no clear effect without reasoning (Appendix I.2). New counterfactual pairs toggle the file after its lock is stated; with swapped reasoning, Qwen3-8B moves the currently locked file in 39 of 122 pairs, against 4 with its own reasoning (Section 7, Table 3; Appendix H.1).

Readouts of the state and the direct agent (jvFL, 3zNN). Section 8 now asks whether state readouts reflect the current state after an update. Without reasoning, Qwen3-8B's state answer has AUROC 0.99 when the written value is current and 0.28 after a toggle, and refitted probes also become less reliable (Table 4; Appendix K). Matched patching on toggled pairs shows that a strong dependence can be on an outdated value (Appendix P). The known comparison of probe and mean-difference directions moved to Appendix M.

Comparable reasoning budgets (3zNN, EHct). Section 3 now defines forced closure and reports how often it occurs; excluding force-closed turns leaves the effects unchanged (Appendices C and D). Secondary runs with a smaller budget were rerun with the main-run budget, or removed when they duplicated the main run. Appendix R lists the budget, maximum tokens and GPU time for every run.

Clearer narrative and scope (all reviewers). The abstract, introduction and discussion now connect three questions: when retained reasoning can substitute for current reasoning, how new state changes affect actions, and what the reasoning and state readouts reveal about those actions. Generating reasoning gives most of the reduction in violations in our rollouts, and retention helps without current reasoning and in longer episodes. Current reasoning improves responses to new events, and swapping the supplied reasoning redirects later actions. The readout and patching results show why a decodable state or a strong causal dependence alone does not establish that an action uses the current state. We state the scope of the main claim and its limitations explicitly (Sections 1 and 9).

Taken together, these changes respond to the main concerns raised in the reviews and make the scope of our claims more explicit. The replies under each review give the details and the supporting numbers. We would be glad to answer any remaining questions.

AuthorsOpenReview ↗

Summary of the discussion

We thank the reviewers for their time. For convenience, we briefly summarize where the discussion stands.

Experimental setup. The reviewers asked for the missing retention conditions, longer episodes, matched reasoning budgets and counterfactual pairs with a state change. We added all of these (Sections 4, 5 and 7; Appendices D and R). Reviewer 3zNN writes that the new experiments "largely address my original concerns about the experimental setup", and reviewer jvFL writes that they "help establish that current reasoning improves both state updating and action selection".

Remaining question: why CoT helps. Both follow-up comments ask what the results show beyond the behavioral benefit of reasoning. Our replies point to interventions already in the paper:

  • Current reasoning updates the state by combining events. After a new toggle, Qwen3-8B's remaining unsafe moves all involve a file that had been toggled before, and labelling each event as separate halves these moves only when the agent reasons (Section 5; Appendix I.2).
  • Knowing the state is not enough to act safely. A correct state table placed before the command does not reproduce the benefit of reasoning (Section 6).
  • Reasoning helps even when no update is needed. It almost removes violations at files whose lock never changed, so its benefit is not limited to tracking changes (Appendix C.5).
  • The action follows the trace's conclusions. Editing only the lock statements in a trace changes the move in one pair, while swapping the whole trace redirects it (Section 7; Appendix H).

Together, these results point to two roles of current reasoning: combining events into the current state, and applying the task rules to that state before the action. As promised in our reply to reviewer jvFL, the revised Section 9 now includes this synthesis (paragraph "Why current reasoning helps").

Thank you again for a constructive discussion.

From OpenReview, CC BY 4.0. Authors as signed there; shown unedited.

Related papers

Loading the map…

Discussion (0)

Sign in to comment.