- Rating
- 3(+2)
- Confidence
- 2
- Informativeness
- 1
- Presentation
- 1
- Soundness
- 2
Read review
Paper summary
This paper looks at arithmetic tasks such as judging even/odd from adding 4 numbers. They focus on Qwen for the main analyses and activation based analyses. They use a method to define when the model has settled on an answer: i.e. log odds of the yes/no answer exceeds 2, and the model doesn't change its preference after this point. They show that this usually occurs at around the time the model writes the total of the sum, but tokens before it writes its final yes/no answer. They also show that hidden state probe (though not specifically at the commitment point) decode the yes/no answer near the final answer with high AUROC. They show that activation patching at points is sufficient to change the log odds in favour of the donor answer. They show a modest advantage for early stopping based on their method compared to a fixed prefix baseline. Their conclusions surround when commitment occurs in these kind of arithmetic procedures.
Critical strengths
S1: The authors define a procedure for detecting early commitment during a reasoning trace S2: The authors use a combination of analyses including hidden state probes and activation patching
Critical weaknesses
W1: I don't find the question the authors address sufficiently important or the contribution significant in terms of understanding what it means when the model has committed to an answer W2: the activation based analyses are not specifically related to the commitment point, so are somewhat disconnected from that part of the paper. W3: The paper relies on a single Qwen model for the activation based analyses and for the main coverage W4: Minor but I found the paper difficult to follow in terms of the method and needed to refer to the appendix.
Questions for authors
Q1: Why is it surprising that the model commits to the yes/no answer at a point when it has emitted the total answer of the sum? Why is this meaningful in terms of understanding the computations that the model is performing? Q2: Why aren't the activation based analyses specifically targeted at the commitment point? Why do the activation based analyses help us understand what is going on at the time of commitment? Q3: Why was only Qwen used for the activation based analyses? Q4: Can the authors please enhance the readability of the main text?
3 comments
Q4 / W4. Readability of the main text.
We rewrote the abstract, the introduction, Sections 5.2, 5.4 and 5.5, the discussion and the conclusion in plainer language. The introduction now states the two measurement problems in words and ends with a list of contributions. Section 3 has a worked example that follows Figure 1 token by token: which strings are scored, why an early agreement at token 32 is not commitment, and where commitment falls. The method can now be followed without the appendix. A new Figure 3 shows the main result for three models in one panel, and the conclusion states three findings, each tied to one results section.
A note on the summary.
In case it helps the discussion, the hidden-state readouts are not evaluated by AUROC against the correct answer. They predict the model's own log odds, and we report the Pearson correlation with that quantity (lines 347 to 349 of the submission, Figure 4a in the revision). This matters for Q2, because a readout of the model's own preference can be compared directly with the effect of editing the state, which is what the new Section 5.4 does.
Dear Reviewer faU9,
Thank you for reading the response and the revision, and for updating your assessment. If any part of the revised main text is still hard to follow, please tell us which one and we will revise it.
One result arrived after our reply and concerns your Q1. On 200 new questions, with the rule fixed before any response was generated, SmolLM3 again prefers the correct answer when it writes the total and loses it at the parity word. 39 responses print a correct total and end with a wrong verdict, and the preference is correct at the total in 38 of them. Returning the preference at the total gives 190/200 against 159/200 (Appendix X).
Thank you for the direct questions. They point at real gaps in the submission, and the revision changes the paper on each of them. Changes are in blue in the revised PDF.
Q1 / W1. Why is it surprising that the model commits when it has written the total, and why is this meaningful?
It would be unsurprising if it were guaranteed. It is not, and the cases where it fails are the informative ones. The revision adds three observations.
First, writing the correct total does not settle the answer. SmolLM3-3B writes a correct total in 242 responses, and 44 of them end with the wrong yes/no verdict. A reader of the text sees a correct number followed by a wrong answer. The trajectory shows what happened. In 43 of the 44, the model preferred the correct answer when it finished the total, and the preference flipped at one later token, a wrong parity word such as "parity = odd" after "total = 144" (Section 5.5, Figure 3b, Table 57).
Second, whether commitment coincides with the number depends on how the output is formatted. For the same questions in two formats it coincides in 31/39 responses in one format and 1/39 in the other. In a 120B-parameter model the preference turns more than 100 tokens before the final answer, inside the reasoning channel, when the number is first written there (Section 5.2).
Third, the alternative that early-answer studies worry about, that the model knows its answer from the start and the reasoning is decoration, is not what happens here. One token before the number is written, the preference is correct in 56%, 50% and 59% of responses in the three models, which is near chance. At the number it is correct in 99.6% and 97.5% for the two small models, and in 100% one token later for the large one (Figure 3a). When we edit the hidden state so that a different number is written, the verdict follows the new number.
On why this matters, we see two reasons. It says what an early-answer lead contains. In the parity tasks, for responses whose preference changes, the lead before the answer word consists of the four or five tokens written after the number that already decides the answer, and not of a stretch of hidden computation. It is also useful. Returning SmolLM3's preference at the number instead of its final verdict raises held-out accuracy from 77/96 to 91/96, and from 55/64 to 63/64 on a second set that was used for nothing else, while generating 18% fewer tokens.
Q2 / W2. Why are the activation analyses not targeted at the commitment point, and what do they tell us about it?
You are right about the submission. Its patching was done near the answer and not at commitment. The revision adds interventions at the commitment token and at neighbouring tokens, in two model families (Section 5.4, Appendices Q and W).
We move the hidden state at that token toward the state of a response with the opposite answer, while keeping linear predictions of the input numbers and of text progress fixed. The control is a random edit of the same size with the same component along the linear answer readout. Applied at the commitment token, the first edit changes the final answer in 5/32 responses for Qwen3-4B and 5/20 for SmolLM3. The random edit changes 0/32 and 0/20, and leaves every continuation identical to the unedited response token for token.
This tells us two things about commitment. In Qwen, the answer is easy to redirect one token before commitment (20/32 responses), harder at commitment (5/32), and almost fixed one token after (1/32). The edited text explains why. Before the number is complete, the edit changes which number is written, and the verdict follows it. Once the number is on the page, the same edit rarely changes the answer. The observational finding, that preference settles at the number, therefore has a causal counterpart: the written number carries the decision. We also learn that the linear readout predicts the preference without being the lever that sets it, since moving the state only along the readout direction never changes the preferred answer.
We do not want to overstate this. The cohorts are small: each model alone gives p = 0.0625 at the commitment token, and 0.002 when the two are pooled. SmolLM3 does not show the same drop across commitment (3/20, 5/20, 4/20), and edit sizes differ somewhat between positions. The paper states these limits.
Q3 / W3. Why was only Qwen used for the activation analyses?
That was a limitation of the submission. SmolLM3-3B, from a different family, now has its own readouts, transfer tests and commitment-relative interventions, reported next to Qwen3-4B in Sections 5.3 to 5.5. The timing measurement now also covers MMLU-Pro and GPT-OSS-120B, in addition to the three further models and three multiple-choice benchmarks that were already in the submission (lines 207 to 210).