- Rating
- 2
- Confidence
- 3
- Informativeness
- 3
- Presentation
- 2
- Soundness
- 2
Read review
Paper summary
The paper investigates "task modeling" (the capacity to maintain a global, unwritten aggregate across context) via a minimal probe (exact counting of repeated tokens ()). The authors introduce Counting Capacity (CC), estimated via a BIC-gated, two-regime Bernoulli change-point estimator (). Evaluating 205 language models across 296 configurations, they report that models fail abruptly at a median of 50 items (maximum 627). Mechanistic interventions across six open-weight models (linear probes, decode-point clamps, mid-depth activation shifts, and boundary-window transplants) indicate that count representations saturate and are not read directly from early linear directions. The paper further connects CC to agentic benchmarks (METR task horizon, grid execution, and state tracking), concluding that CC reflects a distinct, domain-general latent capacity.
Critical strengths
- The BIC-gated change-point Bernoulli likelihood () is a major methodological improvement over naive first-error metrics, properly handling isolated early slips (observed in 284/296 runs).
- Comprehensive evaluation spanning 205 models (296 configurations) with rigorous tracking of bounds (256 mistake-resolved, 39 budget-bounded), full prompt disclosure, and low replication cost ($\approx$$200).
- Commendable candor in reporting replicate run-to-run variance (Table 1), instruction-following failures (3,887/45,287 deviations in App. D), and reasoning-trace failure modes (App. G).
- The probing suite extends beyond passive linear probes to include causal tests: decode-point clamps with random controls, chimera decompositions, mid-depth item shifts, and boundary transplants with sham conditions.
Critical weaknesses
- Construct Validity: Surface-Form Sensitivity vs. Latent "Task Modeling" (W1). The central claim that homogeneous counting probes a domain-general capacity is directly challenged by the paper's own data:
- Delimiter sensitivity: Changing the separator from a comma to a section sign drops CC fourfold (89 to 19).
- Heterogeneous collapse: On mixed sequences, accuracy drops to 10–48% at counts well below the homogeneous CC threshold.
- Known failure mode: Repeated-token regimes trigger well-documented attention sink instabilities and representational collapse. Rather than isolating abstract task modeling, CC appears to measure model stability under long runs of identical tokens.
- Structured errors cluster heavily around offsets of and , alongside round-number attractors. This strongly suggests digit-chunking failures in tokenization and multi-digit emission rather than representational loss. The authors do not evaluate standard formatting controls (e.g., words, zero-padded strings, or modular arithmetic like count mod 10).
- The claim of a shared capacity across counting, grid execution, and state tracking rests on single, unreplicated runs across only two models with divergent behaviors: grid load degrades Sonnet's grid accuracy without shifting its CC, whereas pointer chains degrade Sonnet's CC by 29% but leave Gemini unaffected.
- The METR correlation was identified after screening 66 benchmarks without multiple-comparison adjustments. In a partial correlation controlling for general capability (ECI, ), the incremental out-of-sample gain is with a 95% confidence interval containing zero (), heavily leveraged by a few high-end models.
- Nominal 95% profile intervals derived from single scans severely understate run-to-run uncertainty. In replicate testing (Table 1), 100% of Claude Sonnet 4.6 replicate boundaries (370–424) fall outside its catalogue interval (), indicating that catalogue intervals do not support fine-grained model comparisons.
- The paper omits citation of Dai & Fan (arXiv:2605.02028, May 2026), which predates this submission by 4.5 months. Dai & Fan uses the identical prompt string, stimulus structure, and "Counting Capacity (CC)" terminology across 126 models with mechanistic probing. The authors must explicitly articulate their incremental contribution and reconcile large metric discrepancies (e.g., Claude Opus 4 at 2,176 vs. 580/627 here).
Questions for authors
- How does your methodology and causal probing suite differentiate from this preprint? Why are the reported CC values so divergent on shared models? Dai & Fan (arXiv:2605.02028, May 2026)
- In a two-way mixed-effects model across diverse encodings, what percentage of CC variance is attributable to the model versus the surface format?
- Does the boundary or error clustering shift when prompting for outputs in written words, zero-padded integers, or under modular arithmetic ()?
- Can the dual-task experiments in Table 14 be replicated across runs and extended to open-weight models with a non-executive control task?
- How do you reconcile the claim that the internal readout "keeps advancing" with the flat post-boundary probe slopes observed in four of the six models?
2 comments
W3 / Q4. Replicating the dual-task experiment
The replication changes our claim. The row-rule drops we reported (36% and 28%) do not replicate, and a second task lowers CC beyond its controls for one load in each model.
We scanned six conditions five times each in both models (60 scans; Appendix H, Table 17), adding a second control the model has to read: a passage as long as the grid load with a four-option question. Ratios are to the mean of five solo scans.
| Load | Gemini 3.7 Flash | Claude Sonnet 5 |
|---|---|---|
| solo CC, mean ± SD | 357 ± 42 | 95.0 ± 4.6 |
| control, matched length | 0.91 | 0.93 |
| control, reading | 1.03 | 0.90 |
| pointer chain | 0.93 | 0.67 (62 to 64 in all five scans) |
| row rule, 60% | 1.08 | 0.92 |
| grid rule, 60% | 0.81 (p = 0.03 vs solo, 0.24 vs control) | 0.89, grid answers 65% correct |
Your reading of the single scans was right. The two row-rule scans fell where an occasional scan of each model collapses early. The loads that hold are the pointer chain in Sonnet, in every scan, and the grid in Gemini, less securely. We removed the dual-task claim from the abstract, rewrote the paragraph in Section 3 and Fig. 3C around the replicates, and now say in Sections 5 and 6 that the evidence for a shared capacity is limited to one load per model and does not separate a shared resource from other dual-task interference.
We also ran the conditions on three open-weight models. That run does not test the question and we report it as such: answering immediately, the models solve the rule and grid loads in 0 to 7% of probes, and in two of the three the controls alone lower CC by 22 to 48%.
W4. METR and multiple comparisons
The METR partial correlation survives adjustment for the whole screen, and we agree that the gain in prediction is not established. We computed the partial correlation given ECI for every benchmark in the screen that shares enough models. METR's is the largest positive one (0.74; the next are 0.51 on ten models and 0.46 on fifty). Its unadjusted p is 4.9e-5, which stays below 0.004 after Bonferroni adjustment for every benchmark screened. The bootstrap puts the out-of-sample gain at 0.06 with an interval of -0.24 to 0.30. Section 5 now reports both numbers and says the relation rests on 24 models and should be tested on more.
W5. Profile intervals and run-to-run variation
Agreed. The new scans make the point more strongly than Table 1 did, because the Table 1 rescans start at 0.8 of CC and cannot see an early collapse. Five solo scans from short lengths give a standard deviation of 12% of CC for Gemini 3.7 Flash (290 to 394) and 5% for Claude Sonnet 5, and a single scan can fall 20 to 30% from the others. Section 3 and Appendix A.2 now say so and state that we do not interpret differences of that size between two models; the catalogue spans two orders of magnitude.
Q5. "Keeps advancing" against the flat slopes
The wording was too strong. Readout slope between 1.0 and 1.2 times CC (the stable-regime slope is 1 by construction):
| Model | first probed layer | second | third | last |
|---|---|---|---|---|
| gemma-4-26b | 0.33 | 0.29 | 0.24 | -0.59 |
| gpt-oss-120b | 0.32 | -0.09 | -0.27 | -0.78 |
| gpt-oss-20b | 0.06 | 0.29 | 0.13 | -0.33 |
| llama-3.3-70b | 0.36 | -0.67 | -0.70 | -1.00 |
| muse-glimmer-30b | 0.33 | 0.19 | -0.52 | -1.69 |
| qwen3.6-35b | 0.51 | 0.37 | 0.15 | -0.60 |
At the first probed layer the readout still rises, at a third to a half of its earlier slope in five models and 0.06 in the sixth. At the last layer it falls in all six. Section 4, the Fig. 4D caption and Appendix I.1 now give these numbers in place of "keeps advancing".
Thank you for the detailed review. We ran the three experiments you asked for and revised the paper. Some of the results cut against claims in the submitted version, and we have changed those claims.
W6 / Q1. arXiv:2605.02028 and the different CC values
We have written to the AC about this preprint under the ICLR policy on arXiv versions. On the technical question, the two numbers measure different things. The preprint calls a length stable when the mean absolute error over 16 sampled targets is below 5% of the length. Our CC requires the exact count to be the model's majority answer.
For Claude Opus 4 with reasoning off, exact counting stops being the majority at 580. From 581 to 988 (the end of the scan) the reply is exact in 45% of probes, but 84% of the errors are exactly one short, the mean relative error is 0.08%, and no reply is more than 3.4% off. A 5% tolerance passes that whole range; an exact-majority rule does not. The 627 in our catalogue is the same model with reasoning on.
Relative to that preprint, the submission covers 205 models (against 126), replaces the tolerance rule with the change-point estimator you describe in Strength 1, and adds the METR, execution-ladder, state-tracking, concurrent-load and transplant analyses.
W1 / Q2. How much of CC is the model and how much is the surface form
Across 16 API models from seven vendors and all eleven encodings (176 scans; new Table 2, Appendix A.6), the model accounts for 77% of the variance in log CC and the encoding for 6%. Among six open-weight models with similar capacities, the two shares are about equal.
The API panel's median capacities run from 12 to 295; the open-weight models lie within a factor of three of each other. One encoding can also move one model a long way: the section sign cuts CC to a third or less in five of the 16 API models and leaves it within 6% in three. The revised text says that CC is a property of a model under an encoding.
Accuracy on mixed sequences is lower than on homogeneous ones at the same count. That argues against instability under long runs of identical tokens as the cause of the limit, since removing the runs makes counting harder. The first ladder task (counting a target digit among distractors) and object tracking over natural sentences have no such runs either, and both scale with CC, the second between vendor families. We agree that none of this proves a domain-general capacity. The abstract, introduction and discussion now state task modeling as a hypothesis and say that we do not show counting to be necessary or sufficient for agentic work.
W2 / Q3. Words, zero-padded digits, and count mod 10
The boundary does not shift consistently with the output format. The hundred-item errors do not shift either in the one model that makes them, but that test is weaker than we would like, for the reason given below.
We ran all three formats on nine configurations (Appendix A.7, Table 3). Two Claude configurations did not follow the formats and are excluded: one wrote its counting out in prose until the output cap, the other stated the integer first and then converted it. In the seven that remain, CC with words is 0.80 to 1.28 of its bare-integer value (median 1.00). With the last digit alone it is 0.81 to 1.03 in four configurations and higher in the two Gemini models, and one small open-weight model fails the format itself.
Only Gemini 3.7 Flash makes hundred-item errors in this set. Its reply is exactly 100 short in 32 of 47 errors past the boundary with integers, 31 of 62 with words ("one hundred ninety-seven" for 297), and 42 of 58 with one digit per token. Asked for the last digit alone, it is right in 233 of 240 probes up to 479. The hundred is lost and the units are kept, whatever the format.
This model reasons before it answers (about 1,800 reasoning tokens per probe), so the format only changes how an already-formed count is written. The result rules out the writing of the final answer as the source of the error and leaves open a digit-level slip inside the reasoning text. The configurations that answer immediately were scanned to at most 300 items and make no hundred-item errors. The nearest evidence from them is an error of ten in Gemma 4 26B, which is as common with words (14 of 42 errors) as with integers (11 of 49). The paper states this scope, and the test on an immediate-answer model with a capacity of several hundred has not been run.