acceptodds
Under review as a conference paper at ICLR 2027

Think beyond Instances: Abstraction as Modeling for Mathematical Reasoning

Abstract

Mathematical reasoning is a fundamental capability of large language models (LLMs), requiring multi-step deduction and precise symbolic manipulation. Existing training-based and inference-time approaches improve reasoning performance but often rely on instance-specific solution traces, overlooking reusable computational structures shared across related problems and thereby limiting reasoning generalization. To address this limitation, we propose Think beyond Instances, a process-level reasoning abstraction framework that transforms instance-specific solutions into reusable logic-card chains. Our framework integrates multi-expert reasoning, logic-preserving reasoning abstraction that disentangles reusable logic from instance-specific realizations at each step, and chain-level validation to ensure global reasoning consistency before applying the validated chain to solve the original query. Experiments on six mathematical reasoning benchmarks demonstrate consistent improvements over competitive baselines. On Qwen3.5-9B, our framework achieves an average accuracy of 54.72%, outperforming the strongest baseline by 3.69 percentage points. Further analyses validate the effectiveness, transferability, and scalability of our framework. Our code is publicly available at an anonymous link https://anonymous.4open.science/r/Think-beyond-Instances-123-3E7C.

Primary area:
foundation or frontier models, including LLMs
Keywords:
Mathematical Reasoning, Abstraction Reasoning, LLM

Reviews

Among:
Shape:
1234this paper, Top 30%
Figure 1. Mean review rating of the reviewed papers at ICLR 2027. This paper averages 2.67 over 3 reviews, above 70% of the others.
Reviewer_4pirOpenReview ↗
Rating
2
Confidence
3
Informativeness
2
Presentation
3
Soundness
2
Read review
Paper summary

This paper proposes a training-free framework for mathematical reasoning that integrates Multi-Expert Reasoning, Logic-Preserving Reasoning Abstraction and Chain-level Validation. The framework aims to extract reusable reasoning structures from instance-specific solutions. Experiments on six mathematical reasoning benchmarks demonstrate that the proposed method outperforms several baselines.

Critical strengths
  1. The paper presents a approach to abstracting reusable reasoning structures from instance-specific mathematical solutions, potentially improving generalization across related problems.
Critical weaknesses
  1. The proposed method appears to be incremental, with somewhat limited methodological novelty. Several key components, including multi-expert reasoning, logic-preserving query mutation, reasoning abstraction and chain-level validation, are closely related to existing methods (like AdaR, AoT, RLAD).
  2. The proposed method is specifically designed for mathematical reasoning. However, most of the baselines in Table 1 are general-purpose reasoning optimization methods (LaTRO, GRPO, RAFT++, GPG, InTRO, IC-DAPO). Although they are evaluated on mathematical benchmarks, their underlying methodologies are not specifically designed for mathematical tasks. I recommend including more methods specifically developed for mathematical reasoning, such as those discussed in the Mathematical Reasoning with LLMs part of Related Work, regardless of whether they are training-free or training-based. Additionally, it would be better to evaluate the framework using current proprietary LLMs, such as GPT-5.6 and Claude Opus, as backbone models. Because current experiments show that larger Qwen models benefit less than smaller ones, it is necessary to investigate whether this method works on SOTA models.
Questions for authors
  1. I wonder whether Logic-Preserving Query Mutation truly preserves the underlying logic. Similar accuracy does not necessarily imply equivalent underlying reasoning structures, so the current experiment shown in Table 4 is insufficient to validate the assumption of logic preservation.
2 comments

Q1: The proposed method appears to be incremental, with somewhat limited methodological novelty. Several key components, including multi-expert reasoning, logic-preserving query mutation, reasoning abstraction and chain-level validation, are closely related to existing methods (like AdaR, AoT, RLAD).

R1: Thank you for the opportunity to clarify our novelty. Our key contribution is mutation-tested, step-level abstraction: we select ALE–CG reasoning patterns by testing whether they apply to both the original query and its mutated variant.

Unlike AdaR’s use of variants for RLVR training, our test runs at inference time without parameter updates. Unlike AoT’s abstraction-level reasoning, we explicitly test individual ALE–CG candidates on a mutated instance. Unlike RLAD’s jointly trained abstraction and solution policies, we construct and validate problem-specific abstractions at inference time and audit the resulting chain. The additional comparison with AbstRaL and RLAD below complements these methodological distinctions. Due to time constraints, we will extend it to the remaining benchmarks in the next revision.

MethodMATH500MinervaAMC23AIME25
Qwen3.5-4B
AbstRaL79.226.568.618.3
RLAD80.428.669.619.9
Ours83.231.470.520.6
Qwen3.5-9B
AbstRaL82.431.774.022.3
RLAD84.733.475.724.1
Ours86.335.676.024.4

Q2: The proposed method is specifically designed for mathematical reasoning. However, most of the baselines in Table 1 are general-purpose reasoning optimization methods (LaTRO, GRPO, RAFT++, GPG, InTRO, IC-DAPO). Although they are evaluated on mathematical benchmarks, their underlying methodologies are not specifically designed for mathematical tasks. I recommend including more methods specifically developed for mathematical reasoning, such as those discussed in the Mathematical Reasoning with LLMs part of Related Work, regardless of whether they are training-free or training-based. Additionally, it would be better to evaluate the framework using current proprietary LLMs, such as GPT-5.6 and Claude Opus, as backbone models. Because current experiments show that larger Qwen models benefit less than smaller ones, it is necessary to investigate whether this method works on SOTA models.

R2: Thank you for the suggestions. We add AbstRaL and RLAD, two abstraction-based mathematical-reasoning methods from ICLR 2026, alongside IC-DAPO, our strongest original baseline. Our method outperforms all three on all four benchmarks with both Qwen3.5-4B and Qwen3.5-9B (first table).

We also evaluate Qwen3.5-27B, Qwen3.5-122B-A10B, Qwen3.5-397B-A17B, and GPT5.6-Sol, each with and without our framework. Improvements persist across all four benchmarks for every backbone (second table), showing benefits beyond small models despite generally narrowing gains.

MethodMATH500MinervaAMC23AIME25
Qwen3.5-4B
IC-DAPO78.725.867.417.4
AbstRaL79.226.568.618.3
RLAD80.428.669.619.9
Ours83.231.470.520.6
Qwen3.5-9B
IC-DAPO81.630.772.920.7
AbstRaL82.431.774.022.3
RLAD84.733.475.724.1
Ours86.335.676.024.4
ModelSettingMATH500MinervaAMC23AIME25
Qwen3.5-27BBaseline89.039.581.043.0
+ Ours92.245.485.348.5
Qwen3.5-122B-A10BBaseline94.050.588.066.0
+ Ours96.454.490.768.6
Qwen3.5-397B-A17BBaseline95.855.391.474.6
+ Ours97.658.593.177.3
GPT5.6-Sol†Baseline96.772.095.090.0
+ Ours97.974.896.292.1

Questions For Authors

Q3: I wonder whether Logic-Preserving Query Mutation truly preserves the underlying logic. Similar accuracy does not necessarily imply equivalent underlying reasoning structures, so the current experiment shown in Table 4 is insufficient to validate the assumption of logic preservation.

R3: Thank you for highlighting this limitation. We add an LLM-assisted review of 270 original–mutated pairs before validator filtering using GPT-6.1 Sol. The review finds that 251 preserve the core reasoning logic; ten contain incomplete variable substitutions, eight contain notation or context inconsistencies, and one remains unresolved. These direct checks support that our mutation procedure preserves the core reasoning logic in most cases.

Thank you very much for your positive feedback and for acknowledging that your concerns have been fully addressed. We truly appreciate your time and careful consideration.

Since the main concerns are now resolved through our rebuttal and clarifications, we would sincerely appreciate it if you could kindly consider raising your score accordingly. This would mean a great deal to us.

Reviewer_MK8eOpenReview ↗
Rating
3(+1)
Confidence
3
Informativeness
2
Presentation
3
Soundness
2
Read review
Paper summary

This paper addresses a problem in mathematical reasoning: solution trajectories generated for individual problems may depend excessively on the specific numerical values and entities in those problems, without explicitly extracting transferable computational structures.

The authors propose an inference-time framework. First, three role-based experts generate solutions that are integrated into a preliminary answer. The framework then constructs problem variants intended to preserve the computational logic, represents each reasoning step as an “Abstract Logical Expression (ALE)–Concrete Grounding (CG)” card, and selects and revises these cards according to their applicability to the variants. Finally, it audits the complete card chain, applies it to the original problem, and reconciles the resulting answer with the expert answers.

The authors report performance improvements across six mathematical benchmarks and multiple models. For example, Qwen3.5-9B achieves an average accuracy of 54.72%, exceeding the strongest baseline in the table by 3.69 percentage points. Component ablations, comparisons with Self-Consistency, cross-model transfer experiments, and model-size analyses provide further evidence of the framework’s practical value. However, these results do not sufficiently distinguish between two explanations: acquiring reusable abstractions and improving accuracy through additional generation, revision, and transfer of solution information.

Critical strengths

1.The method has a reasonable motivation and receives support from several empirical analyses. Evaluating candidate reasoning steps on problems with altered instance-specific information is a meaningful way to assess reasoning structures. In Table 2, removing problem mutation, stepwise card construction, or validation mechanisms reduces performance, indicating that the full framework’s gains cannot be attributed entirely to the initial expert voting.

2.The experimental effort is substantial, and the authors provide useful analyses of some limitations. Six benchmarks, multiple backbone models, component ablations, comparisons with Self-Consistency, and the analyses of expert configurations and computational costs in Appendices C–E form a reasonably comprehensive experimental evaluation. The authors disclose the additional computational costs and report diminishing gains on larger models. These results help assess the method’s applicability fairly.

Critical weaknesses

1.Whether the framework remains valuable as the backbone model’s reasoning capability improves has not been sufficiently established. The main experiments primarily evaluate models with 4B–9B parameters, although Table 6 also includes 14B and 32B models. The scaling experiments reveal an important limitation: the average improvement decreases from 11.76 percentage points at 4B to 4.49 percentage points at 32B, with no improvement on AIME25 at 32B. This raises the possibility that the framework’s benefits primarily reflect compensation for the limitations of weaker backbone models. Its value as a broadly effective reasoning mechanism therefore remains unclear. Research on smaller models is valuable in its own right, but the paper needs to define its intended scope more clearly. If the authors aim to propose a broadly applicable reasoning method, they should evaluate a representative, substantially stronger reasoning model to determine whether abstraction and validation continue to provide meaningful gains. If the work primarily targets smaller models under resource constraints, experiments under corresponding computational constraints should establish its value in that setting.

2.The improvements on the evaluated benchmarks do not yet establish a compelling trade-off between the benefits and costs of this complex framework. The six mathematical benchmarks show that the evaluated backbone models still have substantial room for improvement. However, the paper has not established that the framework addresses reasoning difficulties that remain challenging for stronger models. The current results therefore provide stronger support for improving the particular models evaluated in the paper than for advancing mathematical reasoning capabilities more broadly. The evidence for practical usefulness is also incomplete. Table 3 compares the framework with Self-Consistency under the same number of LLM calls, which is a useful control. However, individual calls may differ substantially in input length, output length, and computational cost. In Table 8, the full framework with Qwen3-8B uses 23.83K tokens and takes 16.98 seconds per problem, compared with 15.83K tokens and 8.90 seconds for AoT. Comparisons with simpler revision strategies and stronger models under matched token budgets or inference latency are needed to determine whether the additional procedural complexity is justified. The paper should also distinguish the contribution of abstract representations from that of ordinary solution guidance, clarifying the source of the performance gains.

Questions for authors
  1. How does the framework perform on stronger reasoning models? The experiments mainly focus on 4B–9B models, with the largest being only 32B. Table 6 shows diminishing gains as model size increases, from 11.76% at 4B to 4.49% at 32B, with no improvement on AIME25. This raises concerns about whether the framework provides generalizable reasoning improvements or mainly compensates for weaker models. Could the authors evaluate substantially stronger reasoning models to demonstrate its broader applicability?

  2. How does the accuracy–cost trade-off compare with simpler alternatives? Under matched total input and output token budgets or actual inference latency, how does the framework compare with Self-Consistency, iterative refinement, and directly using a stronger model? Please explain how the budget accounting includes problem mutation, card selection, validation, and retries.

  3. For which kinds of new problems are the logic chains “reusable”? Does reuse refer to new instances with the same computational structure, or does it also support problems with different structures? For new problems that did not participate in card construction or selection, how does the framework determine whether an existing logic chain applies and whether its steps need to be adapted or combined? If experts must solve each new problem afresh and construct a new logic chain, please clarify the specific meaning of “reuse.” For problems on which none of the initial experts develops a viable solution approach, is there evidence that subsequent abstraction and validation can discover a new, effective solution structure?

  4. What is the distinct contribution of abstraction beyond concrete solution guidance? Could the authors compare the full ALE–CG representation with ALE-only guidance, CG-only guidance, and conventional chain-of-thought guidance generated by the same teacher model, while controlling for computational budget and guidance length where feasible? In particular, the cross-model transfer results in Table 5 may reflect the benefits of receiving concrete solution information from a stronger model. Which results demonstrate that the abstract representation itself provides additional value?

  5. How reliable are the mutated problems used to select reasoning cards? Beyond the LLM-based validator, could the authors provide independent checks, such as a manual audit or executable verification where applicable, to assess whether accepted variants are well-posed and preserve the intended computational structure? Please also clarify how reference answers for the mutated problems in Table 4 are obtained and verified. How frequently do invalid or structurally inconsistent variants pass validation, and how do such errors affect card selection and final accuracy?

4 comments

Q1: How does the framework perform on stronger reasoning models? The experiments mainly focus on 4B–9B models, with the largest being only 32B. Table 6 shows diminishing gains as model size increases, from 11.76% at 4B to 4.49% at 32B, with no improvement on AIME25. This raises concerns about whether the framework provides generalizable reasoning improvements or mainly compensates for weaker models. Could the authors evaluate substantially stronger reasoning models to demonstrate its broader applicability?

R1: Thank you for raising this question. We extend our evaluation to stronger backbones: Qwen3.5-27B, Qwen3.5-122B-A10B, Qwen3.5-397B-A17B, and GPT5.6-Sol. Each model is evaluated with and without our framework to isolate its contribution.

As shown below, our method improves accuracy on all four benchmarks for every backbone. On AIME25, accuracy rises from 43.0% to 48.5%, 66.0% to 68.6%, 74.6% to 77.3%, and 90.0% to 92.1%, respectively. These results show that the framework’s benefits extend beyond small backbones. Due to time constraints, we will extend it to the remaining benchmarks in the next revision.

ModelSettingMATH500MinervaAMC23AIME25
Qwen3.5-27BBaseline89.039.581.043.0
+ Ours92.245.485.348.5
Qwen3.5-122B-A10BBaseline94.050.588.066.0
+ Ours96.454.490.768.6
Qwen3.5-397B-A17BBaseline95.855.391.474.6
+ Ours97.658.593.177.3
GPT5.6-Sol†Baseline96.772.095.090.0
+ Ours97.974.896.292.1

Q2: How does the accuracy–cost trade-off compare with simpler alternatives? Under matched total input and output token budgets or actual inference latency, how does the framework compare with Self-Consistency, iterative refinement, and directly using a stronger model? Please explain how the budget accounting includes problem mutation, card selection, validation, and retries.

R2: Thank you for highlighting the accuracy–cost trade-off. Table 3 compares our framework with Self-Consistency under matched LLM-call budgets, showing consistent improvements across all benchmarks and backbones. Since calls can differ in token length and latency, we additionally compare with IC-DAPO under separately matched token, latency, and call budgets. Our method outperforms IC-DAPO on all four benchmarks with both Qwen3.5-4B and Qwen3.5-9B across all three settings. We also report absolute budgets and each pipeline stage’s share of latency, tokens, and calls in the 2nd table.

BudgetMethodMATH500MinervaAMC23AIME25
Qwen3.5-4B
TokensIC-DAPO80.527.868.818.8
Ours83.231.470.520.6
LatencyIC-DAPO81.128.869.419.4
Ours83.231.470.520.6
CallsIC-DAPO81.729.569.820.3
Ours83.231.470.520.6
Qwen3.5-9B
TokensIC-DAPO83.933.174.222.2
Ours86.335.676.024.4
LatencyIC-DAPO84.633.874.923.2
Ours86.335.676.024.4
CallsIC-DAPO85.034.575.823.9
Ours86.335.676.024.4
StageLatency share (%)Token share (%)LLM-call share (%)
Qwen3.5-4B
Multi-expert reasoning28.830.825.5
Logic-preserving mutation12.511.614.6
Step-wise card construction43.142.843.8
Chain-level validation/reasoning15.614.816.1
Total100.0100.0100.0
Qwen3.5-9B
Multi-expert reasoning27.432.127.0
Logic-preserving mutation10.210.115.9
Step-wise card construction47.345.441.2
Chain-level validation/reasoning15.112.415.9
Total100.0100.0100.0

Q3: For which kinds of new problems are the logic chains “reusable”? For problems on which none of the initial experts develops a viable solution approach, is there evidence that subsequent abstraction and validation can discover a new, effective solution structure?

R3: Thank you for the question, and we apologize for the ambiguity. “Reuse” refers to applying an abstract reasoning pattern to the original problem and its mutations sharing the same computational structure. Each new problem is processed independently; we do not retrieve or adapt existing chains for structurally different problems.

As illustrated in line 196–201, the initial experts need not reach a correct consensus. When they disagree, the abstraction and validation pipeline generates an additional answer. This answer is selected if it matches any expert; otherwise, we use the expert majority if available, or the pipeline answer if all three experts disagree. This process can recover from initial prediction errors, as supported by the improvement over expert voting in Table 2.

Q4: What is the distinct contribution of abstraction beyond concrete solution guidance? Could the authors compare the full ALE–CG representation with ALE-only guidance, CG-only guidance, and conventional chain-of-thought guidance generated by the same teacher model, while controlling for computational budget and guidance length where feasible?

R4: Thank you for suggesting this comparison. Under matched token and latency budgets, we compare full ALE–CG guidance with ALE-only, CG-only, and abstract CoT guidance for the same reasoning process. Full ALE–CG achieves the highest accuracy on all four benchmarks with both Qwen3.5-4B and Qwen3.5-9B, supporting the benefit of combining abstract operations with concrete mathematical grounding. Absolute resource measurements are reported alongside the results.

The gains from cross-model transfer in Table 5 may partly reflect a stronger source model. However, our same-backbone experiments in Table 1 show gains without a stronger teacher, and this representation ablation isolates the additional value of structured ALE–CG guidance.

RepresentationMATH500Min. MathAMC23AIME25
Qwen3.5-4B
Full ALE–CG83.231.470.520.6
ALE only80.928.467.918.7
CG only81.328.768.519.0
Abstract CoT79.427.366.817.8
Qwen3.5-9B
Full ALE–CG86.335.676.024.4
ALE only83.732.073.421.0
CG only84.232.474.121.9
Abstract CoT82.630.872.119.5

Q5: How reliable are the mutated problems used to select reasoning cards? Beyond the LLM-based validator, could the authors provide independent checks, such as a manual audit or executable verification where applicable, to assess whether accepted variants are well-posed and preserve the intended computational structure?

R5: Thank you for raising this concern. Our LLM-assisted review using GPT-6.1 Sol finds that 251 of 270 candidate mutations preserve the core reasoning logic before validator filtering; 18 contain substitution or notation inconsistencies, and one remains unresolved. These direct checks support that our mutation procedure preserves the core reasoning logic in most cases.

Regarding the evaluation in Table 4, answer correctness on mutated queries is assessed by the same backbone acting as an LLM judge.

The tables report erroneous acceptance (Invalid Accepted), card changes after mutation correction (Card Selection Changed), final errors for incorrectly accepted mutations (Final Wrong Among Invalid), and accuracy gains after correction (Accuracy Impact). Erroneous acceptance rates are 10.8–19.0% for Qwen3.5-4B and 8.3–10.0% for Qwen3.5-9B; correction improves full-test-set accuracy by 1.2–1.8 and 0.7–1.4 percentage points, respectively. All values are percentages except Accuracy Impact (pp). These results highlight the value of detecting and correcting invalid mutations.

MATH500Min. MathAMC23AIME25
Qwen3.5-4B + Ours
Validator acceptance (%)92879083
Invalid Accepted ↓ (%)12.210.819.013.3
Card Selection Changed (%)46.747.641.451.7
Final Wrong Among Invalid ↓ (%)60.071.462.573.3
Accuracy Impact ↑ (pp)+1.2+1.8+1.4+1.6
MATH500Min. MathAMC23AIME25
Qwen3.5-9B + Ours
Validator acceptance (%)95899390
Invalid Accepted ↓ (%)8.38.610.09.9
Card Selection Changed (%)44.438.942.240.6
Final Wrong Among Invalid ↓ (%)55.672.256.370.3
Accuracy Impact ↑ (pp)+0.9+1.4+0.7+1.3

Thank you very much for your positive feedback and for acknowledging that your concerns have been fully addressed. We truly appreciate your time and careful consideration.

Since the main concerns are now resolved through our rebuttal and clarifications, we would sincerely appreciate it if you could kindly consider raising your score accordingly. This would mean a great deal to us.

The authors have addressed my concerns, and I am willing to raise my score.

Reviewer_X4VYOpenReview ↗
Rating
3
Confidence
3
Informativeness
2
Presentation
3
Soundness
3
Read review
Paper summary

This paper proposes Think Beyond Instances, a framework that extracts reusable logic abstractions from problem instances to improve LLMs' reasoning abilities. The framework consists of three parts. The Multi-Expert Reasoning step samples parallel reasoning traces from three experts, each targeting a different aspect. The Logic-Preserving Abstraction step extracts the underlying abstraction using variants and generates ALE-CG logic cards through iterative refinement. The Chain–level Validation and Reasoning step validates the generated logic-card chain to solve the original query, then predicts the final answer alongside expert predictions from the first step. Evaluation is conducted across a range of mathematical reasoning benchmarks and compared against RL-based, prompt-based, and abstraction-based baselines. Extensive ablation is conducted to isolate the effect of each component in the framework, as well as budget efficiency, robustness, cross-model transferability, and scalability.

Critical strengths
  1. The writing and language of the paper are exceptionally clear and easy to understand. The organization is also well-structured, with the illustration serving as a running example that clarifies the framework's three components.
  2. The idea of extracting the underlying reusable abstraction that goes beyond instances is well–motivated. The pipeline design with logic-card construction and validation is interesting.
  3. The evaluation is extensive, covering a wide selection of mathematical reasoning benchmarks and relevant baselines (prompt-based, RL-based, and abstraction–based). The results convincingly demonstrate the framework's effectiveness.
  4. The ablation that isolates each component of the framework justifies the necessity of the design.
  5. The additional analyses are useful to show the efficiency and robustness of the framework, including comparisons with self-consistency on increased inference compute, robustness under query mutation, cross-model transferability of logic-card claims, and scalability with model scales.
Critical weaknesses
  1. [Major] The motivation of the paper claims to extract reusable abstractions across related problems (“beyond instances”), but the framework applies per instance; therefore, this seems to be misaligned. The generated logic cards apply only per instance, rather than as a universal set shared across problems (e.g., within a benchmark).
  2. [Medium] The framework consists of seven models (experts, mutation, validation, etc.), several sampling passes, and refinement iterations, leading to significant costs as a major limitation. Although it is compared with Self–Consistency (SC) as a baseline under the same inference budget, SC may not be a fair comparison, as it is an older method with only parallel sampling and no additional iterative refinement or collaborative expert pipelines. The paper mentions Clustering and Experience-based Verification [Liang et al., 2026]. Would this be a stronger baseline?
  3. [Medium] Evaluation only covers mathematical reasoning benchmarks. Broader coverage of other reasoning benchmarks, such as logic and proof or algorithmic, may greatly improve the utility of the proposed pipeline.
  4. [Minor] The generation of the logic cards requires a set number of “K”. How is this number chosen, and is there any ablation on the appropriate choice?
  5. [Minor] The baselines (lines 264-266) discussion may benefit from better classification (which ones are RL-based, prompt-based, abstraction-based) than mere listing.
Questions for authors
  1. The three experts target different reasoning preferences (algebraic, strategic, and geometric reasoning). Are these preferences injected through prompts? Also, wouldn’t the preference depend on the question type? For example, if I am solving an algebraic question, the geometric reasoning expert may be less effective. Why not use three experts, all with algebraic preferences?
  2. What do ALE and CG stand for in the logic card? The abbreviations do not seem to be explained.
5 comments

Q1: [Major] The motivation of the paper claims to extract reusable abstractions across related problems (“beyond instances”), but the framework applies per instance; therefore, this seems to be misaligned. The generated logic cards apply only per instance, rather than as a universal set shared across problems (e.g., within a benchmark).

R1: We apologize for the ambiguity in “reusable.” Our framework operates on each problem independently; it does not extract abstractions from one problem for reuse on arbitrary new problems. Our intended meaning of reusability is different, referring to applying the same abstract reasoning pattern to the original problem and its mutations, which change instance-specific details, such as numerical values, while retaining the intended reasoning structure. Testing abstractions across these variants helps filter out instance-specific reasoning errors, improving reasoning reliability and answer accuracy on the original problem. We will clarify this distinction in the revised manuscript to avoid overstating the scope of reusability.

Q2: [Medium] The framework consists of seven models (experts, mutation, validation, etc.), several sampling passes, and refinement iterations, leading to significant costs as a major limitation. Although it is compared with Self–Consistency (SC) as a baseline under the same inference budget, SC may not be a fair comparison, as it is an older method with only parallel sampling and no additional iterative refinement or collaborative expert pipelines. The paper mentions Clustering and Experience-based Verification [Liang et al., 2026]. Would this be a stronger baseline?

R2: Thank you for the suggestion. We considered Clustering and Experience-based Verification (Liang et al., 2026); however, its official code was not publicly available, preventing a faithful reproduction. We therefore add two recent, directly relevant baselines, AbstRaL (Gao et al., ICLR 2026) and RLAD (Qu et al., ICLR 2026). Our method outperforms both on all four mathematical benchmarks with both backbones, as shown in the first table.

References: AbstRaL (Gao et al., ICLR 2026), RLAD (Qu et al., ICLR 2026)

MethodMATH500MinervaAMC23AIME25
Qwen3.5-4B
AbstRaL79.226.568.618.3
RLAD80.428.669.619.9
Ours83.231.470.520.6
Qwen3.5-9B
AbstRaL82.431.774.022.3
RLAD84.733.475.724.1
Ours86.335.676.024.4

Q3: [Medium] Evaluation only covers mathematical reasoning benchmarks. Broader coverage of other reasoning benchmarks, such as logic and proof or algorithmic, may greatly improve the utility of the proposed pipeline.

R3: Following your suggestion, we compare our framework with the base models and IC-DAPO (our strongest baseline) on LiveCodeBench (algorithmic reasoning), ProofWriter (logical deduction), and HumanEval (code generation). As shown below, our method outperforms both baselines on all three benchmarks with Qwen3.5-4B and Qwen3.5-9B, demonstrating benefits beyond mathematical reasoning.

MethodsLiveCodeBenchProofWriterHumanEval
Qwen3.5-4B
Base26.881.886.5
IC-DAPO30.284.589.1
Ours32.086.090.2
Qwen3.5-9B
Base35.287.090.0
IC-DAPO40.389.792.1
Ours42.291.193.0

Q4: [Minor] The generation of the logic cards requires a set number of “K”. How is this number chosen, and is there any ablation on the appropriate choice?

R4: Thank you for the question. We evaluate K ∈ {1, 2, 3, 4, 5} with Qwen3.5-4B and Qwen3.5-9B on MATH500, Minerva Math, AMC23, and AIME25. Performance improves from K = 1 to 4, with K = 4 achieving the highest accuracy across all four benchmarks for both backbones; increasing K to 5 slightly reduces accuracy. We therefore use K = 4 in our main experiments. Due to time constraints, we will extend it to the remaining benchmarks in the next revision.

KMATH500MinervaAMC23AIME25
Qwen3.5-4B
180.027.667.217.8
281.529.268.819.0
382.530.769.820.1
483.231.470.520.6
582.830.970.120.3
Qwen3.5-9B
183.231.572.821.3
284.533.374.122.5
385.334.875.223.7
486.335.676.024.4
585.835.175.623.9

Q5: [Minor] The baselines (lines 264-266) discussion may benefit from better classification (which ones are RL-based, prompt-based, abstraction-based) than mere listing.

R5: Thank you for the suggestion. We will group baselines by their primary methodology: abstraction-based methods (AoT, AbstRaL, RLAD), RL-based methods (AdaR, GRPO, RAFT++, GPG, InTRO, IC-DAPO), and other training-based methods (LaTRO). We will revise the baseline descriptions in Section 3.1 accordingly.

Q6: The three experts target different reasoning preferences (algebraic, strategic, and geometric reasoning). Are these preferences injected through prompts? Also, wouldn’t the preference depend on the question type? For example, if I am solving an algebraic question, the geometric reasoning expert may be less effective. Why not use three experts, all with algebraic preferences?

R6: The algebraic, strategic, and geometric preferences are specified through prompts applied to the same backbone (Appendix B). We agree that their suitability depends on the question type. However, these prompts encourage different perspectives rather than restrict each expert to a single problem category, so even an algebraic problem may benefit from complementary approaches.

To test the reviewer’s proposal, we let an LLM select a preference for each question and use three experts with that same preference. As shown below, our heterogeneous experts outperform this adaptive homogeneous setting on all four benchmarks with both Qwen3.5-4B and Qwen3.5-9B, supporting the value of retaining diverse reasoning perspectives.

ConfigurationMATH500Min. MathAMC23AIME25
Qwen3.5-4B
Diverse Experts (Ours)83.231.470.520.6
Same-Type Experts82.129.369.519.1
Qwen3.5-9B
Diverse Experts (Ours)86.335.676.024.4
Same-Type Experts85.233.874.822.9

The expert type in Same-Type Experts is selected by an LLM for each question.

Q7: What do ALE and CG stand for in the logic card? The abbreviations do not seem to be explained.

R7: ALE denotes Abstract Logic Expression and CG denotes Concrete Grounding, introduced in Section 1 (Lines 68–69) and explained in Section 2.3 (Lines 168–173). ALE states the general reasoning operation in words, while CG represents how that operation is instantiated in mathematical expressions or calculations. Their pairing connects the general pattern to an actual derivation. We will make their full names and distinct roles more prominent at first use.

Thank you very much for your positive feedback and for acknowledging that your concerns have been fully addressed. We truly appreciate your time and careful consideration.

Since the main concerns are now resolved through our rebuttal and clarifications, we would sincerely appreciate it if you could kindly consider raising your score accordingly. This would mean a great deal to us.

Comments on the paper

1 comment
Area_Chair_cbctOpenReview ↗

Reminder: Please Review the Author Response and Reconsider Your Final Decision

Dear Reviewers,

This is a gentle reminder to review the authors’ response to your comments. After considering their clarifications and rebuttals, I kindly ask you to reassess your review and update your final recommendation if warranted.

Thank you for your time and thoughtful consideration.

Best regards,

Area Chair

From OpenReview, CC BY 4.0. Authors as signed there; shown unedited.

Related papers

Loading the map…

Discussion (0)

Sign in to comment.