acceptodds
Under review as a conference paper at ICLR 2027

How Instruct Models Inherit Reasoning

Abstract

Instruct models increasingly read prompts written by other agents, as systems delegate simpler tasks to cheaper models. These instructions are seldom verified, but they are often the backbone of complex agentic operations. Nobody needs to verify them, as long as two assumptions are true: a model good enough at the task will not be misled by errors in the input, and if it is misled, its output will show it. We find that neither holds. Using BIG-Bench-Hard tasks rebuilt as generators, across twelve difficulty tiers and five models, we show that instruct models treat worked solutions as the answer, and therefore errors in the solution are propagated in the large majority of generations; for the hardest tiers the same holds when the error is planted mid-derivation, where no answer is stated anywhere. The rate at which errors are propagated does not fall as the model gets better at the task, and it leaves no trace in the answer. We study the copying mechanism using activation patching, and show the provided value is transported to the position that emits the answer. Hence, errors in the computation an agent passes on are not repairable by a stronger model or by reading its answers. We also study strategy prompts, in which an agent gives a task-specific overview of how the task is to be solved. A correct strategy buys about 40% of what a full solution buys, at a fraction of its cost. A strategy for the wrong task is ignored by some models and derails others, and a plausible but wrong strategy is followed at rates set by the model rather than by capability. Switching the same weights to thinking mode removes much of the copying, but less so on the hardest problems.

Primary area:
interpretability and explainable AI
Keywords:
instruct models, provided reasoning, LLM

Reviews

Among:
Shape:
1234this paper, Top 17%
Figure 1. Mean review rating of the reviewed papers at ICLR 2027. This paper averages 2.75 over 4 reviews, above 83% of the others.
Reviewer_zWr6OpenReview ↗
Rating
3(+1)
Confidence
2
Informativeness
2
Presentation
1
Soundness
2
Read review
Paper summary

This paper studies the setting of a larger model decomposing tasks for smaller models. In particular, the work is interested in whether smaller, instruction-tuned models can complete tasks given strategies and partial work from larger models and whether they can detect errors inserted into these strategies or partial work. They find that most instruction-tuned models are able to follow the follow provided strategies and continue partially complete work, but they are generally not able to identify and correct errors. Finally, the work uses interpretability techniques to illustrate the source of these issues.

Critical strengths
  • As models are increasingly used as parts of multi-agent systems, questions around how and whether to delegate work to weaker models leads to interesting research problems like the ones studied in this work.
  • The work studies a number of different task settings along with a number of different models, illustrating that their findings are not model-specific.
  • Finally, the finding that in the cases where models report errors (lines 350-352), they re-derive answers is neat.
Critical weaknesses
  • A critical weakness of the work was the choice to focus on instruction models rather than reasoning models. For the reasoning-intensive tasks that are studied in this work, reasoning models are more appropriate to study compared to instruction-tuned models. It makes more sense to provide reasoning models with reasoning strategies or partial reasoning traces and see if they can detect and correct errors. The one reasoning model studied performs well, but the discussion seems like it is cut short.
  • Additionally, while it is appreciated that open-weight models are the focus of this work, if possible it would be good to include frontier models in the analysis as well. These are models that would likely be used by practitioners, so it is worth having a sense of how they perform.
  • Finally, while mentioned by the authors, putting the partial solution into the assistant message is likely not the most ecologically valid design choice—-during training, the assistant message is only generated by the model, so injecting something into it does not necessarily give a natural response.
Questions for authors

not questions, just flagging some typos:

  • line 70: “at at”
  • line 129: “trough” → “through”
  • line 198/199: “you” is used in the first sentence but “we” is used in the second.
  • line 201: “which” should be capitalized
Reviewer_mdMXOpenReview ↗
Rating
3(+1)
Confidence
2
Informativeness
2
Presentation
3
Soundness
2
Read review
Paper summary

The paper investigates how instruction models use prompts written by other models, a case which occurs in multi-agent pipelines. Experiments across three generated BIG-Bench-Hard task families, twelve difficulty tiers, and multiple open-weight models show that correct content can improve accuracy, while corrupted solutions frequently cause models to propagate errors without explicitly acknowledging them, even on tasks they otherwise solve accurately. Misleading strategies produce failures that vary across models and tasks. Authors show that enabling thinking mode can reduce error adoption, although to the lesser extent on harder problems. The main contribution is a controlled comparison of general strategies and problem-specific solutions, systematic measurements of error propagation, and analyses of how provided values influence model predictions. The authors validate their findings through comparisons with correct partial solutions, errors introduced at intermediate steps, The authors also check the reliability of automatically extracting answers from model responses and examine how provided values are represented inside the models and influence their predictions using activation patching and probing.

Critical strengths

The experimental design provides a useful understanding of how models use provided reasoning and propagate its errors. Programmatically generated tasks allow precise manipulation of intermediate states and comparisons of correct and misleading strategies with correct and corrupted worked solutions across models and levels of problem difficulty. Comparisons with correct partial solutions help distinguish failures caused by provided errors from difficulties completing the remaining computation. Although prior work has established that models can propagate reasoning errors, this systematic comparison provides additional insight into how provided general strategies and worked solutions can help or mislead them.

Critical weaknesses

The experimental setup is limited to only one way of providing reasoning from model AA to model BB: the provided text is placed at the start of BB's response. The authors themselves acknowledge this limitation, see Section 8. However, AA's reasoning can be provided as a separate chat message isolated by special tokens. In this case, the model BB might treat AA's reasoning differently by attributing it to another source, which can drastically affect the error propagation process. Without comparing these two setups, the broader applicability of the findings to real-life multi-agent pipelines remains unclear.

Questions for authors
  1. See Critical Weaknesses. Could you provide a comparison of the aforementioned two setups?
  2. See Appendix A.2. It says that different sampling settings (temperature and top-pp) were used for Qwen3-14B with and without thinking. Meanwhile this setup follows the recommendation provided by the model's authors, it doesn't isolate the effect of enable the thinking mode alone. I would recommend conducting an additional set of experiments in which thinking mode is enabled, but the sampling settings are kept the same as in when thinking mode is disabled.
  3. Could you compare models of different sizes within the same family on identical tasks and corruptions? For example, this would help test whether the planted value has a similar influence on accuracy across model sizes, or whether larger models more often override it.
  4. The introduction would benefit from a more explicit highlighting of novelty relative to prior work. In particular, the authors should highlight more explicitly which new conclusions follow from their findings beyond the established observation that models can propagate errors from provided reasoning.
Reviewer_F7EsOpenReview ↗
Rating
3(+1)
Confidence
2
Informativeness
2
Presentation
3
Soundness
2
Read review
Paper summary

The paper shows how models tend to inherit reasoning errors when given an hint about a solution or a strategy from another model. They show that they tend to inherit errors and that this happens both silently and even when the models are able to solve the problem without the hint.

Critical strengths
  • Well written and easy to read
  • Good range of models covered (although more recent and frontier are missing)
Critical weaknesses
  • The experiments mainly use models that are already a couple of years old, modern and frontier models are missing.
  • The paper motivates the question by citing the agentic setting, but why is that not tested in the main experiments?
  • The experimental settings does not include classic math and reasoning benchmarks (like GSM8K, MATH, MMLU PRO, others...) that could be very useful in establishing how models inherit reasoning errors in more complex settings.
Questions for authors
  • Could the autors show how results generalize to modern models like qwen3.5 or the latest Mistral models and frontier closed ones?

  • How do the results change if you put in the prompt to the model the possibility of the hint being incorrect and how much the specific chat format contributes to error adoption?

  • How do the findings extend to classic math / reasoning (and if time permitting agentic) benchmarks?

Reviewer_VJanOpenReview ↗
Rating
2(+1)
Confidence
3
Informativeness
2
Presentation
3
Soundness
2
Read review
Paper summary

This paper generates three types of BBH-style tasks across 12 difficulty levels and asks 5 instruct models to continue from given reasoning assistant prefills. The authors provide correct or incorrect strategies, intermediate steps, and final states. Models benefit from correct information but often follow errors without pointing them out. Errors introduced in the middle of a solution also affect the remaining reasoning. Activation patching experiments suggest that state information is carried to the answering position through multiple components. Switching the same model to thinking mode substantially reduces error adoption.

Critical strengths
  • The well-controlled experimental setup allows for more rigorous analysis and makes the findings more reliable.
  • The main experiments and ablations are comprehensive, and the interpretability analysis provides deeper insights into the observed behavior.
Critical weaknesses
  • The main experiments only use assistant prefill, asking models to continue from a given reasoning prefix. As the authors acknowledge in the discussion, this may encourage models to stay consistent with the preceding text rather than question it. However, the paper does not compare this with placing the same content in user/tool messages or explicitly asking models to check and correct the given solution. These comparisons would help distinguish whether models lack the tendency to check reasoning on their own or cannot correct it even when asked. They would also help clarify how well the findings apply to real agent workflows.

  • The controlled synthetic tasks make it easier to introduce precise errors and study internal mechanisms. However, the experiments cover only three task types, with no validation on real user queries. It is therefore unclear whether the findings generalize to more open-ended, practical settings.

  • For non-thinking models, high error adoption may be related to how they were trained. Training examples involving checking, reflection, and error correction have strong thinking-style characteristics, so they are more likely to be used for training thinking behavior, with fewer such examples used for non-thinking models. In that case, not checking the given reasoning may simply reflect the behavior encouraged by the training setup, rather than a general inability to handle reasoning inherited from another model. The clear improvement after enabling thinking may also support this explanation, but the paper does not distinguish between these possible causes.

Questions for authors

None.

1 comment

Thanks for your detailed responses. I have raised my score.

From OpenReview, CC BY 4.0. Authors as signed there; shown unedited.

Related papers

Loading the map…

Discussion (0)

Sign in to comment.