- Rating
- 2
- Confidence
- 4
- Informativeness
- 2
- Presentation
- 2
- Soundness
- 2
Read review
Paper summary
This paper proposes TraCE, a training-free framework for improving the reasoning efficiency of large reasoning models. The framework extracts reusable efficiency rules from successful reasoning trajectories, iteratively refines these rules through execution feedback, and consolidates them into domain-level rule repositories. During inference, TraCE employs embedding-based domain routing and MMR-based rule selection to construct instance-specific reasoning strategies without updating model parameters. The authors evaluate TraCE on DeepSeek-V4-Pro and GLM-5.3-Flash across mathematical reasoning, scientific reasoning, and agent planning benchmarks. The results show substantial reductions in output tokens while generally maintaining competitive task accuracy. Additional ablation studies investigate the contributions of instance-level rules, domain-level guidance, and iterative execution feedback.
Critical strengths
S1. Practical Training-Free Framework.
TraCE offers a practical approach to improving reasoning efficiency without requiring additional model training or access to internal model states. This makes the framework potentially applicable to large-scale and black-box reasoning models.
S2. Promising Empirical Results Across Multiple Tasks.
The experiments cover mathematical reasoning, scientific reasoning, and agent planning benchmarks on two large reasoning models. TraCE achieves substantial reductions in output tokens across multiple settings while generally maintaining competitive accuracy. The ablation studies provide preliminary evidence supporting the contributions of its individual components.
**S3. Execution Feedback-Guided Rule Refinement.
TraCE incorporates execution feedback to iteratively refine efficiency rules based on both answer correctness and reasoning length, rather than relying solely on rules extracted from existing trajectories. This provides a useful mechanism for improving the effectiveness of reasoning strategies and reducing unnecessary computation.
Critical weaknesses
W1. Unclear Methodological Novelty Beyond Existing Experience and Skill Learning Frameworks.
TraCE follows a pipeline similar to existing experience-based learning and skill optimization frameworks, such as ExpeL, by extracting reusable knowledge from successful trajectories, refining it through feedback, and retrieving relevant guidance for new tasks. While TraCE focuses on reasoning efficiency, it remains unclear what fundamentally distinguishes its methodology from adapting existing experience-learning approaches to this objective. Moreover, components such as domain-level rule consolidation and MMR-based retrieval largely rely on established techniques. The paper would benefit from a clearer explanation of its unique methodological contributions and direct comparisons with existing frameworks under the same reasoning-efficiency objective.
W2. Insufficient Validation of Rule Effectiveness, Applicability, and Generalization.
TraCE primarily validates rule modifications based on correctness and reduced output length on the same source instance, which does not necessarily guarantee their effectiveness or transferability to unseen problems. Furthermore, embedding-based domain routing and MMR-based rule selection rely on semantic similarity, which may not accurately capture the underlying reasoning structure or determine whether a rule's applicability conditions are satisfied. Although the framework instructs the model to ignore inapplicable rules, it lacks explicit applicability verification. Domain-level consolidation may also introduce overly broad rules or lose important applicability conditions. While the paper demonstrates downstream improvements on held-out benchmarks, it lacks systematic evaluations of individual rule validity, retrieval reliability, and robustness to rule–instance mismatches.
W3. Insufficient Statistical Reliability and Accuracy–Efficiency Trade-off Analysis.
The paper reports substantial reductions in output tokens, but the statistical reliability of the results remains insufficiently examined. Several accuracy differences are relatively small, particularly on benchmarks with limited sample sizes, and the paper primarily reports point estimates without confidence intervals or repeated-run variance. Moreover, certain settings exhibit noticeable accuracy degradation, such as GLM-5.3-Flash on AIME25, where accuracy decreases from 100% to 90%. Without repeated evaluations and statistical uncertainty analysis, it is difficult to determine whether the reported accuracy differences are robust. A more systematic analysis of accuracy–efficiency trade-offs would further strengthen the empirical conclusions.
Questions for authors
Q1. Rule Effectiveness and Generalization.
How do the authors verify that extracted rules are genuinely effective and transferable beyond the source instances used during refinement? Could the authors evaluate individual rules on held-out problems and report their validity, applicability, and effectiveness, including whether domain-level consolidation preserves these properties?
Q2. Retrieval Reliability and Robustness.
How reliably does embedding-based similarity identify rules that are actually applicable to a target problem? Could the authors evaluate the impact of retrieving irrelevant or mismatched rules, and compare the current retrieval mechanism with applicability-aware alternatives?
Q3. Statistical Significance and Evaluation Variance.
Are the reported accuracy and token-efficiency improvements statistically significant, particularly when performance differences are small? Could the authors provide confidence intervals, paired statistical comparisons, and robustness analyses across different random seeds or rule repositories?
Q4. Novelty and Comparison with Existing Frameworks.
What are the key methodological differences between TraCE and existing experience-based learning frameworks, such as ExpeL? Could the authors provide more direct comparisons under comparable computational budgets and reasoning-efficiency objectives to better establish the contribution of the proposed framework?