acceptodds
Under review as a conference paper at ICLR 2027

Helpful When Right, Controlling When Wrong: Causal Dependence on External Intermediate States in Small Language Models

Abstract

Step-level feedback is usually judged by whether it improves a final answer, but this hides whether a model recomputes or simply inherits a supplied intermediate state. We isolate these mechanisms with controlled interventions on arithmetic and symbolic reasoning traces. Each trace has a unique first error and deterministic continuation, so we can change only the external state while measuring final repair, local adoption, and exact downstream propagation. Across Qwen3 and Gemma 3 models from 270M to 4B parameters, correct states sometimes produce large gains. On symbolic logic, they improve repair over localization alone by 58.0 percentage points for Qwen3-1.7B and 18.7 points for Gemma 3 1B. Capable models also propagate false states. Qwen3-4B adopts matched wrong feedback on 51.3% of arithmetic items and 56.7% of logic items. A pre-specified seven-level arithmetic intervention reveals a sharp peak at the true state, alongside persistent family-specific susceptibility to counterfactual states. The results support three descriptive regimes: capacity limited, feedback enabled, and autonomous but feedback susceptible. External step-level feedback can therefore be both helpful and behaviorally controlling.

Primary area:
foundation or frontier models, including LLMs
Keywords:
Small Language Models, Reasoning, Step-level Feedback, Causal Intervention, Intermediate States, Feedback Susceptibility

Reviews

Among:
Shape:
1234this paper, Bottom 2%
Figure 1. Mean review rating of the reviewed papers at ICLR 2027. This paper averages 1.33 over 3 reviews, above 2% of the others.
Reviewer_5QBKOpenReview ↗
Rating
1
Confidence
3
Informativeness
2
Presentation
1
Soundness
1
Read review
Paper summary

This paper investigates what happens when reasoning in small language models is corrected at the first observed mistake in the trace. Reasoning is corrected using mainly two types of signals: localization (telling the model where the mistake happens) and providing the corrected state. The authors observe different responses from models of different sizes.

Critical strengths

Investigating how LLMs respond to corrective messages may, in the long term, inform the design of better training protocols for reasoning LLMs.

Critical weaknesses
  • LLMs were overused in the writing of this paper, yielding text that was hard to follow in many critical parts.
  • It is unclear if the findings observed generalize beyond the simple deterministic scenarios proposed.
  • It is unclear what exactly do these synthetic problems look like.
  • The findings may be over stated. They do not seem to be as consistent as stated.
Questions for authors

LLM usage was abused in this paper, yielding text that is unnecessary, confusing, and which considerably impacts the readability of the paper. Here are a few examples:

  • "A matched message containing 32 can." How exactly does "32" help? This was not mentioned before and is not explained at all.
  • I am not sure the mathematical symbol ⊭ is used correctly in lines 112 to 116. This symbol means “does not satisfy” or “does not semantically entail.” But it seems to be used in a positive sense in these lines (if I understand them correctly, which may not be true...). For example, the definition of Adopt should be that the correct state is adopted, not that the adoption property is not satisfied. In any case, the whole math in these lines is completely unnecessary, as it is never used again. The plain language definitions in Section 3.5 are much better.
  • "The trade-off is reduced ecological breadth," - what does this mean in English?
  • "The confirmatory set contains... " - This is the first time I hear of "confirmatory" sets. Is this dev or test? The difference matters.
  • "Semantically related feedback forms were not interchangeable." - what does this mean?
  • "The stable observation is the tension itself" - what does this mean?

The authors state that "We therefore study compact transition systems in which every intermediate state is known." But do the observed findings generalize beyond this simpler scenario? It is unclear. This limitation reduces the usefulness of the stated observations.

"Our controlled tasks make each transition deterministic." - are you sure the process is completely deterministic? Many mathematical operations are commutative, which produces many equally valid alternatives for a reasoning trace over a mathematical problem. The authors should include examples of problems and reasoning traces and discuss their deterministic nature more.

"Table 1 summarizes Qwen3 repair. Localization-only performance increases sharply with capacity, from near zero at 0.6B and 1.7B to 74.3% at 4B. Correct-state feedback has its largest effect at 1.7B: a 63.0-point paired gain. At 4B, however, localization alone outperforms structured correct feedback by 14.3 points." - Why is this happening? the "Correct-state" feedback should also improve with model size, no? In fact, it does for Gemma, just not for Qwen. Further, the "wrong state" feedback is also inconsistent for Qwen: its responses is largest at 4B for Arithmetic problems, but not for Logic, where it is much larger for the intermediate model (1.7B).

Reviewer_P6npOpenReview ↗
Rating
1
Confidence
3
Informativeness
2
Presentation
2
Soundness
2
Read review
Paper summary

The paper studies how small language models use external feedback when revising a reasoning trace. Controlled arithmetic and symbolic tasks allow the authors to distinguish correcting the final answer, adopting a supplied intermediate state, and propagating that state through the remaining computation. Across two model families, correct feedback can substantially improve performance, while incorrect feedback can redirect models that are otherwise capable of repair. Signed state interventions provide a particularly clear demonstration of this behavioral dependence.

Critical strengths
  • S1. The experimental design addresses a useful question that final-answer accuracy misses. Deterministic continuations make it possible to tell whether an incorrect external state actually drives the subsequent solution.

  • S2. The matched correct/incorrect interventions and signed-offset experiment provide convincing evidence that the content of feedback matters. Replication across arithmetic, symbolic reasoning, and two model families strengthens this finding.

  • S3. The study is careful about separating prompt exploration from confirmatory evaluation and reports paired uncertainty. The distinction between feedback benefit and feedback susceptibility is a useful contribution to evaluating verifier-guided systems.

Critical weaknesses
  • W1. The reported “correct-feedback advantage” does not cleanly isolate the benefit of state correctness from the effect of the state-supplying feedback interface. The primary benefit metric compares structured correct-state feedback against localization alone, although these conditions differ not only in state correctness but also in the amount and form of information supplied. This matters empirically: on the symbolic task, Qwen3-1.7B achieves 14.7% repair under localization, 60.0% under matched wrong-state feedback, and 72.7% under correct-state feedback. Thus, the reported +58.0-point correct-versus-localization gain should not be interpreted as a 58-point causal benefit of receiving the correct intermediate state. The matched correct-versus-wrong comparison is better suited to isolating sensitivity to state content. The paper would be stronger if it separated these two effects explicitly and tempered the corresponding interpretation of the “feedback-enabled” regime.

  • W2. The behavioral source of the large repair gains remains incompletely characterized. This is especially striking for Qwen3-1.7B on symbolic reasoning, where wrong-state feedback yields 60.0% final-answer repair despite only 1.3% adoption of the supplied wrong state. Because Repair is defined only by the final answer, this result does not establish that the targeted intermediate state and subsequent reasoning chain were corrected coherently. A trace-level analysis separating coherent recovery, partial correction, and intermediate inconsistency would clarify what behavior actually produces these gains. In particular, non-adoption should not by itself be interpreted as evidence that the model explicitly verified and rejected the supplied state.

Questions for authors
  • Q1. Among Qwen3-1.7B's symbolic responses with a correct final answer under wrong feedback, how often are the targeted state and the subsequent reasoning chain also correct? Could the authors distinguish coherent recovery from intermediate inconsistencies and explain how these behaviors account for the improvement over localization alone?

  • Q2. Do the main Qwen–Gemma differences persist when the same intermediate states are presented as tentative suggestions rather than authoritative corrections? A focused comparison would help establish which conclusions are properties of the models and which depend on the feedback interface.

  • Q3. Why does correct-state feedback reduce repair accuracy for Qwen3-4B relative to localization alone? What do the failed revisions suggest about when an otherwise capable model is disrupted by additional guidance?

  • Q4. Which reported comparisons directly support the proposed strategy of localizing an error first and supplying a replacement state only when verifier confidence is high or autonomous repair is weak? Which parts of this recommendation remain hypotheses, particularly because the experiments do not evaluate a calibrated verifier or an adaptive feedback policy?

  • Q5. Can the authors provide even a small-scale validation on a more natural reasoning setting?

Reviewer_QY6AOpenReview ↗
Rating
2
Confidence
3
Informativeness
1
Presentation
3
Soundness
3
Read review
Paper summary

The paper shows that pointing out a mistake to a model often lets the model fix it, but giving it the correct fix is even more helpful.

Critical strengths

The programmatic generation of reasoning traces is clever.

The three conditions of localization, correct, and matched wrong are very clear.

Paper is clearly written.

Critical weaknesses

Qwen has strange behavior where the "Correct - Localization" peaks with the 1.7B model. This is mysterious, and the paper does not explain or comment on it.

Table 3 is not very interesting. How are we supposed to interpret the changes seen in the different non-zero regimes? Table 3 is presented in a much clearer way later on in Table 4.

Ultimately, this paper fails to show anything especially interesting beyond the fairly obvious result that giving a clear correction is better than giving an unclear correction.

Questions for authors

I suggest that you follow this paper up with additional interventions. The result you have found is clear but not especially interesting. I unfortunately do not have clear suggestions, but I suggest that you either:

  • use a wider range of models
  • investigate different types of error
  • investigate more ways of flagging the error to the LLM. For example, what if only a vague hint is given?

From OpenReview, CC BY 4.0. Authors as signed there; shown unedited.

Related papers

Loading the map…

Discussion (0)

Sign in to comment.