acceptodds
Under review as a conference paper at ICLR 2027

Targeted Retrieval, Compact Representations: How CoT Reasoning Improves Long-Context Counting

Abstract

Large language models (LLMs) have been rapidly improving in long-context tasks, powered by Chain-of-Thought (CoT) reasoning. However, the internal mechanisms underlying this improvement remain unclear. We investigate these mechanisms through a needle-in-a-haystack (NIAH) counting task, where an LLM is asked to count the number of records dispersed in a long text. Across twelve model comparison groups, Thinking (or reasoning) improves counting accuracy over Non-thinking, with pronounced gains at larger counts. This motivates our mechanistic analysis, which identifies two contrasting mechanisms: (i) broad retrieval, where Non-thinking models broadly attend to multiple needles; (ii) targeted retrieval, where Thinking models use enumeration in CoT traces to successively retrieve needles. Targeted retrieval concentrates attention on individual needles and is accompanied by more compact internal representations. Moreover, causal intervention analysis suggests that Thinking models use the CoT trace to maintain and update an internal counter as needles are successively retrieved, even without explicit numbering. In small controlled experiments, both retrieval mechanisms and counter states emerge under standard autoregressive training. Together, our results connect long-context retrieval with representation geometry of counting, supporting a state-tracking account of CoT reasoning.

Primary area:
interpretability and explainable AI
Keywords:
CoT reasoning, representation geometry, mechanistic interpretability, needle in a haystack, long-context retrieval, training dynamics

Reviews

Among:
Shape:
1234this paper, Bottom 27%
Figure 1. Mean review rating of the reviewed papers at ICLR 2027. This paper averages 2.00 over 3 reviews, above 27% of the others.
Reviewer_dNxdOpenReview ↗
Rating
2
Confidence
3
Informativeness
2
Presentation
3
Soundness
2
Read review
Paper summary

The paper uses a needle-in-a-haystack counting task to investigate why chain-of-thought improves long-context reasoning. City-score records are inserted into Paul Graham essays and the model must report how many there are. Across twelve model groups, thinking mode is far more accurate than non-thinking mode, and the gap widens as the number of records grows. Mechanistic analyses of Qwen3-8B and Gemma-4-E4B identify two contrasting strategies. Without thinking, a set of heads at the answer token attends to all records simultaneously and the count information they gather is noisy. With thinking, the model enumerates the records one at a time, a head attends to the next record at each step, and the hidden states along the enumeration separate cleanly by count. Head ablation and activation patching support both accounts, most notably in an experiment where transplanting the state of a later item to an earlier position causes the model to skip ahead and continue enumerating correctly. A small transformer trained from scratch reproduces the two retrieval patterns.

Critical strengths
  1. The empirical work is thorough. The behavioral comparison covers twelve model groups with thirty paired seeds per condition, retrieval heads are selected on one set of seeds and evaluated on another, ablations are compared with layer-matched random heads, and patching is performed in both directions. The state-transplant experiment in Figure 6C is particularly persuasive. After a single intervention the model goes on to list the correct subsequent records, which shows that the state is used by the model and not merely decodable from it. The appendix on instructed enumeration with thinking disabled is also a useful contribution for researchers working with closed models.
  2. The counting task offers a controlled setting that brings together long-context retrieval, the geometry of numerical representations and state tracking in chain-of-thought. The account in which per-step retrieval errors compound over many steps is simple and agrees well with the degradation of thinking mode when many records are combined with long contexts. The release of code and an interactive visualization further strengthens the paper.
Critical weaknesses
  1. The main comparison of representational compactness is confounded, and the claim of an internal counter goes somewhat beyond the evidence. Figure 1C contrasts prompt states in non-thinking mode at one layer with trace states in thinking mode at another, and the thinking traces carry explicit numbering of the form 2. City, 3. City. The trace format and the data split were chosen after inspecting the data, and the appendix itself describes the comparison as exploratory. When the index is written in the context, it is unsurprising that states at the kth item separate by k, so the contrast between 98 and 46 percent says little about whether chain-of-thought makes representations more compact. The authors acknowledge that numbering and position may contribute, yet the abstract still presents the result as a central finding. The patching experiment raises a similar issue. The transplanted state encodes both the count and the identity of the record just retrieved, so continuing with the next record is equally consistent with a pointer to the last retrieved position, in the spirit of the iteration heads described by Cabannes et al., without any counter being involved. The non-thinking analysis already shows that an accurate linear readout can coexist with failed steering, and the same caution should apply to the thinking analysis.
  2. Novelty and scope are limited. That chain-of-thought enumeration retrieves records sequentially by attending to the next one is fairly intuitive and closely related to prior work on retrieval heads and iteration heads. The records follow a rigid template and stand out lexically from the surrounding essays, so it is not clear that the conclusions extend to realistic long-context reasoning, where the relevant information is not marked by fixed phrasing. The mechanistic analysis is restricted to two models at around ten thousand tokens, and the causal tests rely on ten seeds or a few dozen prompt pairs. The synthetic experiment uses a single training seed, and the thinking model is trained on traces that list the targets explicitly, so the large accuracy gap there is to be expected.
Questions for authors
  1. Could the comparison in Figure 1C be repeated with unnumbered traces, such as the natural Qwen traces analyzed in Section 4.2, at matched layers, and reported in the main text? If the separation persists, my first concern would largely be resolved.
  2. Is it possible to dissociate count from content in the patching experiment, for instance by transplanting the kth-item state from a prompt with different cities and checking whether the model continues with record k+1 of the target prompt? Such a test would distinguish a counter from a positional pointer.
  3. At the final answer, is the count read from an internal counter state or simply from the last number written in the trace?
2 comments

Response to Reviewer dNxd

We thank the reviewer for recognizing the multi-step intervention evidence. We address the representation controls, causal interpretation, and contribution below.

Questions

Q1: Unnumbered traces at matched layers.

  • Count separation remains clear when we do match layers and control trace grammar. At the same layer, L19 (the layer selected by NCC), unnumbered comma-separated records (CITY with SCORE, …) and quoted records on separate lines ("In the 2024 city score audit, CITY received a score of SCORE.") achieve NCC balanced accuracy of 85% and 97%, respectively, compared with 54% for Non-thinking. The interactive supplement (https://anonymous.4open.science/r/CoT-NIAH-Counting-rebuttal/index.html) shows both formats and the pooled unnumbered traces without grammar filtering across all layers. We will include these comparisons in the main text of the revised manuscript. Click “JS off” to enable the interactive 3D figures.

  • Note that different computational mechanisms might not be implemented at the same layer. Non-thinking aggregates source-record information for direct answering, whereas Thinking retrieves records successively and tracks progress in its trace.


Q2: Separating a counter from a record carrier or a positional pointer.

  • Thanks for pointing out this confound. We tested the proposed cross-city transfer. Transplanting states between different cities introduces a record-content mismatch: in 22/30 transfers, the model rechecks or departs from enumeration, which complicates the interpretation of unsuccessful transfers. Prior work also has only identified item-by-item counter updates, and isolating a content-independent counter from retrieval position and record identity in LLMs remains challenging.

  • We therefore use multi-content counting to functionally distinguish accumulated count from current record identity and retrieval position. Prompts contain city and flower records, and the model counts only cities. Source and target traces share the same current record and remaining records but differ in accumulated city count. An illustrative example:

    Source traceTarget trace
    Current recordRecord 6: BerlinRecord 6: Berlin
    Cities counted so far42
    Remaining recordsTokyo, Lima...Tokyo, Lima...
    Expected counts without patching5, 63, 4

    Successful transfer: the target still retrieves Tokyo and Lima, but counts them as 5 and 6, rather than 3 and 4.

    Across 30 cross-prompt pairs, endpoint-aligned item-span patching at L22, using the main-text patching procedure, changes the first subsequent target count in 19/30 cases, versus 0/30 for self and norm-matched random patches. This shows that transferred trace-state information affects counting beyond a current-record pointer alone.


Q3: Reading an internal counter versus copying the last numeral.

  • Answer-state patching remains effective in traces without indices. Following the main-text protocol, we patch answer states between prompts with different true counts and correct baseline answers for both source and target. We use naturally generated Qwen3-8B traces without explicit item numbering, stop reasoning after complete enumeration, and then query the answer. Across 40 new prompt pairs, with the layer and patch boundary fixed before confirmation, final-layer patching immediately before the answer yields source-count matching in 33/40 cases (82.5%), versus 0/40 for norm-matched random interventions; self patches preserve the target's original count in 40/40. This extends the main-text answer-state patching result to traces without explicit item indices.

Weaknesses

W1: Confounded compactness comparison and an overly strong counter interpretation.

  • The evidence extends beyond numbered traces and comparisons across different layers. Our response to Q1 provides unnumbered representation results and a same-layer comparison; Q2 tests the effect of transferred counting progress while matching current record identity and retrieval position; Q3 extends answer-state patching to unnumbered traces. Together, these support functionally used trace states that track progress. We will distinguish these findings from the stronger hypothesis of a content-independent counter and make trace selection, token positions, and layer selection explicit in the main text.

W2: Limited novelty, task scope, and mechanistic coverage.

  • Beyond retrieval heads and iteration heads. Retrieval heads copy information from individual needles, and iteration heads (Cabannes et al.) were identified in small transformers trained on synthetic iterative tasks. We study naturally generated CoT in pretrained LLMs, contrast broad and targeted retrieval, connect them to distinct counting-error laws, and show that trace states carry accumulated count beyond a current-record pointer (our response to Q2).

  • Template and scope. The fixed record template provides the exact retrieval targets and progress labels needed for causal tests. Our new TableBench experiment extends the analysis to counting over real tables, where records are table rows rather than template sentences: Thinking achieves 90% accuracy compared with 20% for Non-thinking, and patching shifts subsequent counts as predicted in 8/10 trials, versus 0/10 for self and random patches.

  • Synthetic experiment. Listing the targets is what defines enumeration CoT; the experiment aims to show that targeted and broad retrieval mechanisms emerge under standard autoregressive training.

  • Please see the Novelty claim and Agentic tasks sections of our General Response for details.


We hope our responses address your concerns and encourage a more favorable assessment of our paper.


Reviewer_sR7NOpenReview ↗
Rating
2
Confidence
3
Informativeness
2
Presentation
1
Soundness
2(+1)
Read review
Paper summary

This paper investigates how CoT improves long-context counting by comparing retrieval mechanisms and internal representations in Thinking and Non-thinking modes. The authors find that direct answering relies more on broad retrieval, whereas enumeration-based CoT exhibits targeted retrieval of successive records and more separable representations of counting progress. Cross-model evaluations, attention-head ablations, activation patching, and small-model training experiments provide evidence supporting these mechanisms.

Critical strengths
  1. Broad empirical evaluation. The paper compares Thinking and Non-thinking across multiple model families, context lengths, and target counts, systematically characterizing the benefits and limitations of CoT for counting.

  2. Complementary mechanistic evidence. The study combines attention analysis, representation probing, head ablation, and activation patching, supplemented by small-model training experiments, providing useful evidence about enumeration-based CoT.

Critical weaknesses
  1. The central finding is not surprising. A substantial portion of the paper is devoted to showing that chain-of-thought reasoning outperforms non-thinking reasoning, a conclusion that is already well established in the literature and, for the setting studied here, largely expected. In addition, the task itself is relatively simple, so the observed advantage of explicit reasoning is somewhat trivial by construction. These points together limit the contribution, as it remains unclear whether the conclusions would still hold on more challenging tasks where the benefit of chain-of-thought reasoning is less obvious and more informative.

  2. Evaluation relies on two controlled data constructions centered on retrieval and counting, with limited evidence from realistic or more complex reasoning tasks. Since the benefit of chain-of-thought reasoning is known to be most pronounced on complex tasks, whereas on these simple retrieval and counting settings it is comparatively marginal, I would suggest extending the study to more complex and practically meaningful tasks, where explicit reasoning matters more and the findings would be considerably more informative. Detailed mechanistic analysis focuses on two smaller models. Validation on larger models under challenging conditions, including both successful and failed cases, is needed.

Questions for authors
  1. Does the broad-versus-targeted attention contrast persist when both modes are compared at final-answer positions using shared metrics?

  2. Do the proposed mechanisms also appear in non-enumerative CoT, such as grouped counting?

  3. Do these mechanisms generalize to larger models and more complex reasoning tasks, and how do they differ between successful and failed cases?

3 comments

Response to Reviewer sR7N

We thank the reviewer for the questions about matched comparisons, reasoning formats, and generalization, and address each point below.

Questions

Q1: Shared attention metrics at final-answer positions.

  • Our comparison aligns computational function rather than token position. Non-thinking retrieves source records at the answer query, whereas Thinking retrieves them during enumeration. In Thinking, attention at the final-answer position cannot distinguish retrieving records from reading out count states, so comparing the two modes at this position would not provide a matched comparison of retrieval mechanisms. We therefore compare the stages at which each mode retrieves source records.

Q2: Non-enumerative CoT, such as grouped counting.

  • Count separation persists without explicit item numbering. Variation in trace grammar can obscure this separation, so we additionally examine two unnumbered formats: comma-separated records (CITY with SCORE, …) and quoted records on separate lines ("In the 2024 city score audit, CITY received a score of SCORE."). The interactive supplement (https://anonymous.4open.science/r/CoT-NIAH-Counting-rebuttal/index.html) shows the representation geometry of the pooled unnumbered traces and both formats across all layers. Click “JS off” to enable the interactive 3D figures.

  • Grouped counting is not part of the setting analyzed here: the traces analyzed in our paper retrieve and count individual records, without an explicit grouping step. We would appreciate clarification on the grouped-counting procedure intended in your question.


Q3: Larger models, harder tasks, and successful versus failed reasoning.

  • In our initial experiments, 32B-scale models also showed similar patterns. However, their substantially higher inference cost led us to focus the mechanistic analysis on 8B-scale models. Our behavioral evaluation covers 12 comparison groups, including Qwen3-32B and Gemma-4-31B, with contexts up to 100k tokens.

  • A well-defined task does not make its mechanistic analysis straightforward. NIAH counting couples retrieval with state tracking, while naturally generated CoT varies in wording, structure, and reasoning trajectory. Parsing these traces and aligning computational states for causal interventions are substantive experimental challenges. Additional kkth-record retrieval and category-counting experiments extend the retrieval analysis.

  • We additionally evaluate two realistic counting questions from TableBench, involving city population density and mountain elevation. The task definitions, behavioral results, and mechanistic analyses are summarized in the Agentic tasks section of our General Response.

  • Failed cases. In failed Thinking trajectories, the records omitted from the enumeration indeed receive low attention from the targeted retrieval heads, consistent with the per-step failure account in the Novelty claim section of our General Response. A matched mechanistic comparison of successful and failed trajectories in larger models remains an extension for future work.


Weaknesses

W1: The expected accuracy advantage provides limited insight.

  • The accuracy gap motivates, rather than constitutes, our contribution. Our goal is not to show that Thinking outperforms Non-thinking, but to explain how it does so and where it fails. Across 12 comparison groups, the two modes show distinct error patterns: Non-thinking errors are consistent with noise that accumulates as the count grows, whereas Thinking errors are consistent with small per-step failures that compound over enumeration (leave-one-count-out R2=0.828R^2=0.828 and R2=0.924R^2=0.924). We will foreground these error laws in Section 2 in the revision.

  • The mechanisms explain these error patterns. Non-thinking relies on broad retrieval, and adjacent counts become less distinguishable as NN grows; Thinking relies on targeted retrieval, and its trace states causally steer subsequent retrieval (24/30 Qwen and 21/30 Gemma transplant trials, versus 0/30 and 2/30 for self patches).

  • The benefit of CoT is not trivial by construction. Thinking accuracy still degrades at very long contexts, as predicted by the per-step failure account, so the task probes when explicit reasoning succeeds and when it breaks down.

  • Please see the Novelty claim section of our General Response for the derivations and mechanistic evidence.


W2: Simple controlled tasks and insufficient validation in larger models.

  • Controlled tasks make mechanistic tests possible. NIAH counting provides exact retrieval targets and progress labels at every step, which are needed to identify retrieval heads, align trace states, and run causal interventions. Such labels are rarely available in more complex reasoning tasks.

  • Evidence on a realistic task. On TableBench counting questions over real tables, Thinking achieves 90% accuracy compared with 20% for Non-thinking, and the representation and patching results are consistent with the main text (8/10 trials, versus 0/10 for self and random patches).

  • Larger models and failed cases. The larger-model evaluation is summarized in our response to Q3: our behavioral evaluation includes Qwen3-32B and Gemma-4-31B with contexts up to 100k tokens, and in failed Thinking trajectories, the records omitted from the enumeration receive low attention from the targeted retrieval heads.

  • Please see the Agentic tasks section of our General Response for the benchmark context and the full TableBench results.


Presentation

  • We will improve the presentation as outlined in our General Response: highlighting the empirical counting-error laws in Section 2, connecting the mechanistic analyses in Sections 3 and 4 to these laws, and presenting counting as a testbed for agentic tasks with the new TableBench experiment.

We hope our responses address your concerns and encourage a more favorable assessment of our paper.


Thank you for the response. Your clarifications addressed some of my technical questions, so I raised my Soundness score to 2. However, I still think the problem setting and perspective are too simple. The main findings on counting are not surprising, so I will keep my rating at 2.

Reviewer_taCkOpenReview ↗
Rating
2
Confidence
2
Informativeness
2
Presentation
2
Soundness
2
Read review
Paper summary

This paper investigates how CoT reasoning improves long-context counting in LLMs. Through a needle-in-a-haystack counting task, the authors compare Thinking and Non-thinking modes across multiple model families. The paper identifies two different retrieval patterns: broad retrieval in Non-thinking models and targeted, sequential retrieval in Thinking models. The authors further analyze the internal representations of counting progress and conduct causal interventions to examine the role of these representations in subsequent retrieval and prediction.

Critical strengths
  1. Interesting research question. Understanding how CoT reasoning affects internal retrieval and state-tracking mechanisms is an important research direction. I appreciate the authors' effort to investigate these mechanisms.

  2. Comprehensive empirical analysis. The paper presents extensive experiments across multiple model families, supported by attention visualization, representation analysis, causal interventions, and controlled synthetic experiments. The experiments are generally well organized and provide detailed observations of model behavior.

Critical weaknesses
  1. Limited novelty of the findings. Although the paper conducts several interesting investigations, I find most of its key observations rather intuitive. For example, the superior counting performance of Thinking models is largely expected, since explicitly enumerating relevant records naturally helps avoid omissions and counting errors. Similarly, the observed attention patterns are not particularly surprising: models generating an answer directly may attend to multiple relevant records, whereas models enumerating records step by step are expected to attend to individual records sequentially. These patterns could reasonably be anticipated without conducting the experiments. While the paper empirically verifies these intuitions and provides some causal evidence, I do not find that it establishes sufficiently novel or unexpected insights into the mechanisms of CoT reasoning.

  2. Small scope and insufficient generalizability. The paper focuses primarily on a relatively narrow counting task. I understand that the authors intend to use counting as a controlled setting to investigate more general mechanisms of CoT reasoning. However, the paper does not convincingly demonstrate how the identified mechanisms generalize to more complex reasoning tasks. It remains unclear whether the proposed distinctions between broad and targeted retrieval, or the observed properties of internal counting representations, can explain CoT reasoning in substantially different tasks, such as mathematical reasoning, multi-step logical reasoning, or planning. Without broader validation or a stronger conceptual connection to general reasoning mechanisms, the implications of these findings for the broader ML community remain limited.

Questions for authors

I do not have more questions.

1 comment

Response to Reviewer taCk

We thank the reviewer for recognizing the research question and the complementary empirical analyses, and address the concerns below.

Weaknesses

W1: The findings are intuitive and their novelty is limited.

  • Intuition suggests that CoT helps, but not how it helps. CoT could improve counting simply by externalizing records in the trace, without internal states that guide retrieval. Our interventions show that trace states are functionally used.

  • The two modes fail in quantitatively different ways. Non-thinking errors are consistent with noise that accumulates as the count grows, whereas Thinking errors are consistent with small per-step failures that compound over enumeration. These two error laws fit the cross-model median accuracy curves with R2=0.828R^2=0.828 and R2=0.924R^2=0.924 under leave-one-count-out validation, and our mechanistic analyses of broad and targeted retrieval explain where each error pattern comes from. These quantitative and causal results go beyond the qualitative intuition.

  • Please see the Novelty claim section of our General Response for the derivations and mechanistic evidence.


W2: Narrow scope and insufficient generalizability.

  • Counting isolates two building blocks of complex reasoning. Mathematical reasoning, multi-step logic, and planning all require retrieving relevant information and carrying intermediate states across steps. Counting exercises both capabilities while providing exact retrieval targets and progress labels at every step, which are rarely available in more complex tasks and are what make causal tests of the mechanism tractable.

  • The mechanisms extend to counting in real tables. In our new TableBench experiment, Thinking achieves 90% accuracy compared with 20% for Non-thinking, Thinking states achieve at least 20 percentage points higher NCC accuracy, and patching shifts subsequent counts as predicted in 8/10 trials, versus 0/10 for self and random patches. Extending this mechanistic analysis to mathematical reasoning and planning is left for future work.

  • Please see the Agentic tasks section of our General Response for the benchmark context and the full TableBench results.


We hope our responses address your concerns and encourage a more favorable assessment of our paper.


Comments on the paper

3 comments
AuthorsOpenReview ↗

General Response on Novelty claim

General Response (1/2)

Novelty claim

  • While the benefits of reasoning are intuitive, the mechanism of state tracking through CoT remains much less understood, even in the counting task (where the model needs to maintain a counter). To our knowledge, our work provides the first comprehensive study connecting empirical counting-error laws to retrieval and state tracking within CoT traces.

  • Further, our empirical-law appendix (Appendix C.3, pp. 19–21; Figures 9–10) provides a quantitative account of counting errors, motivating the mechanistic investigation. We will present this framework more explicitly in Section 2 in the revision.

  • Non-thinking: noise accumulation from broad retrieval. We hypothesize that broad attention in long contexts introduces noise, which accumulates as the count increases. Let NN denote the number of target records and LL the context length. A simple mathematical derivation is as follows: suppose the count estimate ZZ, expressed in count units, is given by Z=N+σ(N,L)GZ=N+\sigma(N,L)G, where G∼N(0,1)G\sim\mathcal N(0,1) is standard Gaussian noise, and σ(N,L)=Nσ(1,L)\sigma(N,L)=N\sigma(1,L) is its standard deviation under a count-scaled approximation; σ(1,L)>0\sigma(1,L)>0 is fitted separately at each length. Rounding succeeds when ∣Z−N∣<1/2|Z-N|<1/2, giving the exact-count success probability pNT(N,L)=2Φ (12Nσ(1,L))−1,p_{\mathrm{NT}}(N,L)= 2\Phi\ \left(\frac{1}{2N\sigma(1,L)}\right)-1, where Φ\Phi is the standard normal cumulative distribution function. Under this model, increasing NN increases the effective noise variance N2σ(1,L)2N^2\sigma(1,L)^2 and reduces counting accuracy. Across 12 comparison groups, 14 counts, and 8 lengths, leave-one-count-out validation of the cross-model median accuracy curves achieves R2=0.828R^2=0.828, showing that this explanation is plausible.

  • Our Section 3 provides complementary mechanistic evidence. Non-thinking exhibits broad attention over multiple records, and head interventions establish the functional contribution of this retrieval pattern. Representation analysis further shows that adjacent counts become less distinguishable with higher NN.

  • Thinking: compounding failure across successive enumeration steps. In contrast, Thinking models exhibit a different error pattern, because step-by-step targeted retrieval decomposes counting into successive operations that do not directly inflate the noise variance. A mathematical argument is as follows. Let φL\varphi_L denote the effective failure probability of one required step at length LL, and let α\alpha denote a count-independent baseline failure probability. Assuming NN independent required steps, the exact-count success probability is pT(N,L)=(1−α)(1−φL)N.p_{\mathrm T}(N,L)= (1-\alpha)(1-\varphi_L)^N. For NφL≪1N\varphi_L\ll1, the error probability satisfies 1−pT(N,L)≈α+(1−α)NφL.1-p_{\mathrm T}(N,L) \approx\alpha+(1-\alpha)N\varphi_L. Thus the interaction NφLN\varphi_L describes how small per-step failures accumulate over enumeration. Our account proposes that CoT suppresses counting noise through targeted enumeration, while remaining vulnerable to failures at very long contexts. Under the same leave-one-count-out protocol, this model achieves R2=0.924R^2=0.924 on the cross-model median accuracy curves.

  • Our Section 4 connects this account to representations and causal interventions. Thinking exhibits targeted retrieval and distinguishable representations of counting progress, including in unnumbered traces. More importantly, transplanting a later trace state into an earlier item redirects the next retrieval in 24/30 Qwen and 21/30 Gemma trials, compared with 0/30 and 2/30 for self patches. These interventions demonstrate that trace states functionally influence subsequent retrieval, supporting our account of how targeted retrieval and state tracking help “denoise” long-context counting.


AuthorsOpenReview ↗

General Response on Task Scope and Presentation

General Response (2/2)

Agentic tasks

  • Our novelty lies in identifying and causally testing how retrieval and state tracking interact within CoT; both are important capabilities in agentic tasks. NIAH counting provides exact retrieval targets, progress labels, and answers, enabling controlled tests of how models locate evidence and carry progress across successive operations.

  • A well-defined task does not make its mechanistic analysis straightforward. Naturally generated CoT varies in syntax, ordering, numbering, and repeated or revised reasoning. Parsing these traces and aligning comparable computational stages are substantive experimental challenges. Our work pioneers a controlled approach to addressing these challenges, providing a foundation for mechanistic studies of more complex reasoning.

    Counting is a recurring component of agentic reasoning benchmarks:

  • BIG-bench [1]: object counting.

  • BIG-Bench Hard [2]: object counting.

  • DROP [3]: counting over passages.

  • WikiTableQuestions [4]: count aggregation over real tables.

    Our new experiment shows that our mechanistic analysis is applicable to a realistic agentic task in TableBench [5]. For tractable mechanistic analysis with Qwen3-8B, we select two counting questions from TableBench, asking the model to (i) count cities or municipalities with a population density above 3,000 people/km² in a 13-row table, and (ii) count mountains in Papua New Guinea with an elevation of at least 3,000 meters in a 30-row table. Both require retrieving numerical attributes, applying the question’s threshold, and counting qualifying records.

    Following our main-text approach, we find that Thinking mode achieves 90% accuracy on 30 samples, compared with 20% for Non-thinking mode. Thinking states achieve at least 20 percentage points higher NCC accuracy than Non-thinking states, and patching shifts both subsequent counts as predicted in 8/10 trials, versus 0/10 for self and random patches. Retrieval-head patterns and head-ablation effects are also similar to those in the main text. These results support count-related representations and causal effects in realistic table reasoning. We will provide the detailed analyses and controls in the revision.

References

[1] Srivastava et al. (2022). BIG-bench. (https://arxiv.org/abs/2206.04615)

[2] Suzgun et al. (2022). BIG-Bench Hard. (https://arxiv.org/abs/2210.09261)

[3] Dua et al. (2019). DROP. (https://arxiv.org/abs/1903.00161)

[4] Pasupat and Liang (2015). WikiTableQuestions. (https://arxiv.org/abs/1508.00305)

[5] Wu et al. (2024). TableBench. (https://arxiv.org/abs/2408.09174)

Presentation

In the revision, we will improve the presentation as follows:

  • Framing. We will emphasize the significance of our analysis as a first mechanistic study toward understanding state tracking through CoT.
  • Empirical laws. We will highlight the empirical counting-error laws in Section 2.
  • Connecting mechanisms to the laws. We will connect the analyses in Sections 3 and 4 to these laws, showing how the mechanistic account supports the quantitative error characterization.
  • Counting as a testbed. We will present counting as a testbed for agentic tasks, including our new TableBench experiment.
AuthorsOpenReview ↗

Summary for Chairs and Reviewers

Summary for Chairs and Reviewers

Dear Senior Area Chairs, Area Chairs and Reviewers,

Thank you for your careful evaluation. We briefly summarize how our rebuttal addresses the main concerns, complementing our general response.


Our Central Contribution

Our paper connects empirical counting-error laws to long-context retrieval and state tracking within CoT. The quantitative results motivate our representation analyses and causal interventions in a controlled, well-defined testbed.


How We Addressed the Main Concerns

  • Novelty. While the benefits of reasoning are intuitive, the mechanism of state tracking through CoT remains much less understood, even in the counting task (where the model needs to maintain a counter). To our knowledge, our work provides the first comprehensive study connecting empirical counting-error laws to retrieval and state tracking within CoT traces. Please see our General Response (1/2) for the quantitative framework and supporting results.
  • Representation controls and causal interpretation. Additional same-layer comparisons, unnumbered traces, and category-counting interventions address the numbering and positional-pointer concerns. Please see our response to reviewer dNxd, Q1–Q3, for the results and their interpretation.
  • Task scope. Two additional TableBench case studies test retrieval and counting in realistic table reasoning, extending the mechanistic analysis beyond NIAH counting. Please see our General Response (2/2) for details.

We hope this summary clarifies the contribution and the additional evidence provided in our responses. We respectfully ask that these results be considered in the final assessment and welcome any remaining questions.

Best regards,

The Authors

From OpenReview, CC BY 4.0. Authors as signed there; shown unedited.

Related papers

Loading the map…

Discussion (0)

Sign in to comment.