acceptodds
Withdrawn from ICLR 2027

Systematic Reuse of Shared Reasoning in Language Models

Withdrawn from ICLR 2027, so left out of the paper lists and the map.

Abstract

Large language models often repeat the same intermediate reasoning across different queries. Prior work often reuses information between problems that are similar in topic or wording. We instead define relatedness by whether problems require the same intermediate reasoning—even when the queries might not be similar in wording or topic—and reuse that shared reasoning directly. Across four math benchmarks and MuSiQue, we find that shared reasoning structure is much more informative than surface semantic similarity. When two queries share an intermediate reasoning step, having the model solve that step once and reuse its answer for other queries improves accuracy by 7.2 percentage points while reducing total tokens used by 9.4%. We also study whether we can automatically identify the intermediate reasoning that different queries share. Although models can generate and match candidate reasoning steps, the generated steps often change the meaning of the original reasoning, causing reuse to fail even when the match is correct. Our results suggest that efficient multi-query inference should focus on identifying and preserving shared intermediate reasoning, with reliable automatic discovery remaining the key challenge.

Primary area:
foundation or frontier models, including LLMs
Keywords:
shared reasoning, test-time compute, multi-query inference, language models

Reviews

Among:
Shape:
1234this paper, Bottom 10%
Figure 1. Mean review rating of the reviewed papers at ICLR 2027. This paper averages 1.67 over 3 reviews, above 10% of the others.
Reviewer_ruBROpenReview ↗
Rating
2
Confidence
3
Informativeness
2
Presentation
3
Soundness
2
Read review
Paper summary

This paper studies whether intermediate reasoning shared across queries can be identified and reused to improve inference efficiency. It distinguishes semantic similarity from shared reasoning structure and investigates structural relationships across four math benchmarks and MuSiQue. On MuSiQue, assuming that shared sub-questions are known, reusing model-generated intermediate answers improves accuracy by 7.2 percentage points relative to token-matched irrelevant-information controls and reduces total token usage by 9.4%. The paper also evaluates cross-query compute allocation and an automatic decomposition-and-matching pipeline. The tested allocation methods do not yield reliable improvements, while the two evaluated automatic-discovery configurations substantially reduce accuracy. Additional analyses examine failures in preserving the meaning of generated sub-questions.

Critical strengths

1、The paper investigates a useful distinction between semantic similarity and reusable intermediate structure. The analyses in Section 4.1 and Figure 2 show that structural relationships can carry information not captured by surface similarity. This distinction is relevant to designing cross-query reasoning and caching systems, where similar-looking queries need not share reusable answers.

2、The controlled reuse experiment provides meaningful positive evidence, accompanied by informative negative results. The MuSiQue experiments include model-generated intermediate answers, token-matched irrelevant-information controls, and filtering of intermediate answers that directly match the final answer (Section 3.3; Tables 1 and 15). Table 18 reports a 7.0-percentage-point improvement over the no-reuse baseline, with a 95% confidence interval of [4.0, 10.0]. The paper also reports substantial failures of automatic discovery rather than presenting the known-structure result as a successful automatic system. Together, these results establish a useful opportunity and expose a practical obstacle.

Critical weaknesses

1、The positive experiment does not sufficiently isolate the contribution of cross-query reuse from that of explicit decomposition. The current controls establish that relevant intermediate answers help more than irrelevant text, but they do not compare reuse against independently solving the same supplied sub-questions for each query. Such a baseline would separate the accuracy benefit of decomposition from the computational benefit of sharing intermediate work. This matters particularly because Section 2.1 identifies LAG as a closely related sub-question caching approach, yet there is no direct comparison in the same setting. I therefore find the incremental contribution of cross-query reuse insufficiently quantified, although the known-structure result itself is useful.

2、The explanation of automatic-discovery failure is plausible but not sufficiently isolated experimentally. The 83.9% versus 12.5% accuracy gap in Table 13 is substantial, and Table 14 demonstrates problematic generated sub-questions. However, examples involving “that county” or “the country identified in question 2” also raise the possibility that missing context or unresolved dependencies contribute to the failure. It is unclear how generated questions are aligned to benchmark identities and whether both question versions receive comparable context. Holding matches fixed and repairing entity references and dependencies would help distinguish semantic drift from failures to make sub-questions independently answerable. Since this diagnosis is a central contribution, the missing control materially limits the conclusions that can be drawn from the negative results.

Questions for authors

1、Requested comparison: With the same known sub-question structure, how does cross-query reuse compare with independently solving the required sub-questions for each query, without sharing answers? Reporting both accuracy and total token usage would clarify how much of the benefit comes from decomposition and how much comes from reuse.

2、Clarification and diagnostic experiment: How are generated sub-questions mapped to benchmark identities in the 472 “correctly matched” instances, and what context is provided when answering each version? Does repairing entity references and supplying required preceding answers, while keeping the matches fixed, substantially recover sub-question accuracy or downstream performance?

3、Evaluation clarification: How are the 40 MuSiQue groups constructed, and how are the 462 reported query pairs selected from the 1,800 possible within-group pairs? Please also clarify whether confidence intervals account for within-group dependence and provide the token-cost breakdown underlying the reported 9.4% saving, including the treatment of input tokens.

Reviewer_n3hPOpenReview ↗
Rating
1
Confidence
3
Informativeness
1
Presentation
1
Soundness
1
Read review
Paper summary

This paper focuses on whether models can reuse answers from shared sub-questions when answering multiple questions to save computational resources and improve accuracy. The authors first identify common sub-questions across multiple questions and then directly add the answers to these sub-questions to the original prompt for each question to help the model's problem-solving. For finding common sub-questions, the authors tried two methods: one was to directly use the gold sub-question from the existing MultiHop QA dataset, and the other was to let the model break down the questions and match them based on semantic similarity. The results showed that only using the gold sub-question resulted in improved performance and reduced token usage. The authors also tried allocating the token budget for each question based on information from other questions, but did not obtain better empirical results than a simple baseline.

Critical strengths

None

Critical weaknesses

I think the biggest weakness of this paper is its failure to propose any consistently working methods. The only positive experimental result relies on using the gold sub-question. This is not possible to obtain in actual testing time. Another approach (let the model decompose the problem itself) actually led to a performance drop. I also feel the experiments in the paper are confusing. The authors haven't provided a clear and unified definition of what constitutes sharing across different tasks. Another weakness is the lack of a clear and logical main thread. During the reading, the paper abruptly shifts from sub-question reuse to the dynamic allocation of token usage. So I believe the writing logic and overall experimental design of this paper require significant improvement.

Questions for authors

Since there are no gold sub-questions annotations during testing and the model cannot decompose sub-questions itself, how should we use sub-question sharing in testing?

Reviewer_a2ESOpenReview ↗
Rating
2
Confidence
2
Informativeness
2
Presentation
2
Soundness
2
Read review
Paper summary

This paper distinguishes semantic similarity between queries, e.g. "Why is everyone using AI to write papers?" vs "Everyone seems to be using AI to write papers, right?" which have low semantic similarity but require giving the same answer. They ask if that step could be solved once and then the model just reuses the answer for both to save tokens and raise accuracy. They show that math problems have a very similar success rates when changing the numbers while changing the wording lowers success rate.

Critical strengths

Evaluation is careful by counting every token spent and they split cleanly into train and test splits.

Also clear explanation of why sub-question reuse fails, with the generated sub-questions loosing their meaning outside the original question

Critical weaknesses

The accuracy gain seems to come from it being given the intermediate answer, not from reusing them, so a control would be to have the same answer given to each question separately without sharing or the sub questions given without their answers. Also the model gets 97% on MATH so success rate is already at ceiling so same-topic pairs cant really show a correlation

As far as I can see most of the results are from one model and one run, while only the MuSiQue questions are reused

Questions for authors

How are the placeholders in MuSiQue sub-questions resolved? Gold or model answers?

Its also unclear how the 40 groups of 10 were built? Is it random?

From OpenReview, CC BY 4.0. Authors as signed there; shown unedited.

Related papers

Discussion (0)

Sign in to comment.