Systematic Reuse of Shared Reasoning in Language Models

Large language models often repeat the same intermediate reasoning across different queries. Prior work often reuses information between problems that are similar in topic or wording. We instead define relatedness by whether problems require the same intermediate reasoning—even when the queries might not be similar in wording or topic—and reuse that shared reasoning directly. Across four math benchmarks and MuSiQue, we find that shared reasoning structure is much more informative than surface semantic similarity. When two queries share an intermediate reasoning step, having the model solve that step once and reuse its answer for other queries improves accuracy by 7.2 percentage points while reducing total tokens used by 9.4%. We also study whether we can automatically identify the intermediate reasoning that different queries share. Although models can generate and match candidate reasoning steps, the generated steps often change the meaning of the original reasoning, causing reuse to fail even when the match is correct. Our results suggest that efficient multi-query inference should focus on identifying and preserving shared intermediate reasoning, with reliable automatic discovery remaining the key challenge.