- Rating
- 2
- Confidence
- 3
- Informativeness
- 2
- Presentation
- 3
- Soundness
- 2
Read review
Paper summary
This paper studies whether intermediate reasoning shared across queries can be identified and reused to improve inference efficiency. It distinguishes semantic similarity from shared reasoning structure and investigates structural relationships across four math benchmarks and MuSiQue. On MuSiQue, assuming that shared sub-questions are known, reusing model-generated intermediate answers improves accuracy by 7.2 percentage points relative to token-matched irrelevant-information controls and reduces total token usage by 9.4%. The paper also evaluates cross-query compute allocation and an automatic decomposition-and-matching pipeline. The tested allocation methods do not yield reliable improvements, while the two evaluated automatic-discovery configurations substantially reduce accuracy. Additional analyses examine failures in preserving the meaning of generated sub-questions.
Critical strengths
1、The paper investigates a useful distinction between semantic similarity and reusable intermediate structure. The analyses in Section 4.1 and Figure 2 show that structural relationships can carry information not captured by surface similarity. This distinction is relevant to designing cross-query reasoning and caching systems, where similar-looking queries need not share reusable answers.
2、The controlled reuse experiment provides meaningful positive evidence, accompanied by informative negative results. The MuSiQue experiments include model-generated intermediate answers, token-matched irrelevant-information controls, and filtering of intermediate answers that directly match the final answer (Section 3.3; Tables 1 and 15). Table 18 reports a 7.0-percentage-point improvement over the no-reuse baseline, with a 95% confidence interval of [4.0, 10.0]. The paper also reports substantial failures of automatic discovery rather than presenting the known-structure result as a successful automatic system. Together, these results establish a useful opportunity and expose a practical obstacle.
Critical weaknesses
1、The positive experiment does not sufficiently isolate the contribution of cross-query reuse from that of explicit decomposition. The current controls establish that relevant intermediate answers help more than irrelevant text, but they do not compare reuse against independently solving the same supplied sub-questions for each query. Such a baseline would separate the accuracy benefit of decomposition from the computational benefit of sharing intermediate work. This matters particularly because Section 2.1 identifies LAG as a closely related sub-question caching approach, yet there is no direct comparison in the same setting. I therefore find the incremental contribution of cross-query reuse insufficiently quantified, although the known-structure result itself is useful.
2、The explanation of automatic-discovery failure is plausible but not sufficiently isolated experimentally. The 83.9% versus 12.5% accuracy gap in Table 13 is substantial, and Table 14 demonstrates problematic generated sub-questions. However, examples involving “that county” or “the country identified in question 2” also raise the possibility that missing context or unresolved dependencies contribute to the failure. It is unclear how generated questions are aligned to benchmark identities and whether both question versions receive comparable context. Holding matches fixed and repairing entity references and dependencies would help distinguish semantic drift from failures to make sub-questions independently answerable. Since this diagnosis is a central contribution, the missing control materially limits the conclusions that can be drawn from the negative results.
Questions for authors
1、Requested comparison: With the same known sub-question structure, how does cross-query reuse compare with independently solving the required sub-questions for each query, without sharing answers? Reporting both accuracy and total token usage would clarify how much of the benefit comes from decomposition and how much comes from reuse.
2、Clarification and diagnostic experiment: How are generated sub-questions mapped to benchmark identities in the 472 “correctly matched” instances, and what context is provided when answering each version? Does repairing entity references and supplying required preceding answers, while keeping the matches fixed, substantially recover sub-question accuracy or downstream performance?
3、Evaluation clarification: How are the 40 MuSiQue groups constructed, and how are the 462 reported query pairs selected from the 1,800 possible within-group pairs? Please also clarify whether confidence intervals account for within-group dependence and provide the token-cost breakdown underlying the reported 9.4% saving, including the treatment of input tokens.