- Rating
- 3
- Confidence
- 2
- Informativeness
- 2
- Presentation
- 3
- Soundness
- 2
Read review
Paper summary
In this paper, the authors propose ReTreVal, a training-free inference-time framework that allows an LLM to accumulate typed failure experience across a problem sequence without updating model weights. It couples complexity-adaptive tree exploration, per-node tool-augmented refinement, dual-critique scoring with historical approach-skepticism penalties, typed-failure backtracking to unexplored root-level siblings, and a persistent self-rewriting memory. Across MATH-500 (using Qwen 2.5 32B Instruct) and 1,000 MMLU-Pro problems (using Gemini 2.5 Flash), ReTreVal reaches 85.8% (ground-truth-assisted) and 84.2% (label-free) accuracy on MATH-500, and 54.4% (label-free) on MMLU-Pro, outperforming single-path, tree-search, and self-consistency baselines.
Critical strengths
First, when validation rejects an initial candidate solution, ReTreVal introduces a typed-failure backtracking strategy to avoid repeating the same flawed path with generic feedback, and instead switches to the highest-scoring unexplored root-level sibling node. Before refining that sibling, the proposed method diagnoses the failed attempt along two explicit dimensions: root cause and approach type. It further injects this structured failure record directly into the sibling's critique history so the same error class cannot recur.
Furthermore, a primary barrier to deploying test-time cross-problem memory is unbounded context growth, noise accumulation, and sensitivity to problem ordering. The paper provides concrete empirical evidence that its consolidation rules solve the context-bloat problem while still delivering substantial accuracy gains: across the full 500-problem label-free MATH-500 run, the memory store stabilized at just four entries and injected at most three entries, averaging 66 tokens per prompt, ensuring per-prompt memory overhead never scales with the number of problems.
Critical weaknesses
Table 2 gives the label-free comparison on MATH-500 with memory-equipped baselines. ReTreVal scores 84.2% and Self-Refine with a persistent memory buffer scores 83.2%. This gap is marginal, if we consider the run-to-run performance variation shown in the order study of Section 6.6. This is an important test to support the main claim of the paper, 'LLMs can improve on their own in the inference time'. However, the marginally small gain of ReTreVal seems undermining the claim. In contrast, the gap between ReTreVal is larger on MMLU data, as reported in Table 4. Is there any interpretation to the varying magnitude of the performance gap across two different datasets ?
Questions for authors
1/ What are the exact average input/output token counts, total LLM calls per problem, and wall-clock latency for ReTreVal compared to each baseline in Tables 1–4?
2/ Please elaborate on why, on MATH-500, the performance gap between ReTreVal and Self-Refine is marginal, and yet on MMLU, this gap becomes significantly larger. Does the gap variation originate from any difference between the characteristics of the two datasets?