- Rating
- 2
- Confidence
- 2
- Informativeness
- 2
- Presentation
- 2
- Soundness
- 2
Read review
Paper summary
The paper studies whether test-time learning can improve a model without repeatedly reinforcing the same solution strategies. Solver.EXE combines exploitation and exploration through four agents: a Reasoner uses the evolving memory, an Explorer searches for alternative strategies, an Extractor distills insights and warnings, and a Curator maintains the memory. Experiments on mathematics, STEM, and coding tasks show improvements over the backbone and exploitation-based baselines.
Critical strengths
- The paper presents a clear test-time learning pipeline, separating memory-guided solving, strategy exploration, memory extraction, and memory curation.
- The paper reports results across several backbones and task types, and provides prompts, pseudocode, and local runtime measurements that help readers understand the setup.
Critical weaknesses
- The main math benchmarks are small: Apex has 12 problems and ArxivMath has 32. Apex is also selected from problems that several frontier models fail. The four-run standard deviations only measure sampling variation. Please report per-problem uncertainty and narrow the frontier-model claims if needed.
- The benefit of structured exploration is not fully clear. In Table 9, Solver.EXE improves over Simple Explore by only 1.4 points after two rounds, without error bars. Are these differences significant, and which problems account for the gain?
- The ablations do not isolate all components. The paper removes the Explorer and Curator, but not the Extractor, the two memory types, or the DEVIATE/DRAFT modes. The “w/o Explorer” setting also reallocates its rollouts to the Reasoner, which mixes exploration effects with budget allocation.
- The exploration–exploitation framing is somewhat loose. RL is not inherently exploratory, and prior methods also sample multiple stochastic trajectories. Moreover, if the frozen backbone cannot find a useful new strategy, the Explorer may produce paraphrases or invalid plans; Figure 5 suggests that exploration can hurt easy problems. Please quantify useful exploratory rollouts and consider an adaptive fallback.
- The cost comparison is difficult to interpret because Solver.EXE uses extra calls for exploration, extraction, and curation, while reference models use different sampling budgets and API defaults. Please report total tokens, calls, and latency.
Questions for authors
See weaknesses section.
4 comments
Rebuttal-Part2
W4: Prior methods also sample multiple stochastic trajectories. Please quantify useful exploratory rollouts and consider an adaptive fallback.
A: Thank you for your comment.
Benefit of structured exploration. The reason we borrow the concept of the exploration–exploitation trade-off from RL [1] is that we observed an important limitation in prior test-time learning (TTL) methods:
- In RL [2], external rewards guide subsequent updates to the model's output distribution.
- In prior TTL methods without external rewards, stochastic sampling alone offers limited guidance on how to avoid repeating earlier strategies or mistakes.
We therefore developed our structured exploration method to guide the model to discover alternative solution strategies both effectively and efficiently. To show that, we compare our structured exploration method with prior stochastic methods with Qwen3.5-35B-A3B as follows:
ArxivMath
| Method | Pass@16 | Pass@32 | Pass@64 | Pass@128 |
|---|---|---|---|---|
| Stochastic trajectories | 45.3 | 47.4 | 48.4 | 50.0 |
| Explorer | 48.1 | 56.2 | — | — |
| Reasoner | 50.3 | 56.2 | — | — |
Apex
| Method | Pass@16 | Pass@32 | Pass@64 | Pass@128 |
|---|---|---|---|---|
| Stochastic trajectories | 31.2 | 38.4 | 41.4 | 41.7 |
| Explorer | 37.3 | 41.7 | — | — |
| Reasoner | 41.5 | 50.0 | — | — |
On ArxivMath, our structured exploration at 32 rollouts outperforms stochastic sampling at 128 rollouts (56.2 vs. 50.0). On Apex, the Explorer matches the backbone's Pass@128 with 32 rollouts and solves two problems the backbone misses. These results demonstrate that our structured exploration has the potential to discover useful information for solving these problems.
Useful exploration quantification. To quantify this, we use GPT-6.1 Sol (medium) to annotate 704 Explorer rollouts across four iterations of Solver.EXE. We label a rollout as useful if it introduces a solution approach distinct from previously attempted strategies and either outlines an actionable plan or makes nontrivial progress toward a solution. The table reports the percentage of rollouts labeled useful at each iteration.
| Iteration | ArxivMath | Apex |
|---|---|---|
| Iter 1 | 85.2% | 77.1% |
| Iter 2 | 84.4% | 83.3% |
| Iter 3 | 91.4% | 79.2% |
| Iter 4 | 82.8% | 85.4% |
On both benchmarks, more than 3/4 of the Explorer rollouts are labeled useful at every iteration. For reference, the corresponding rate for Simple Explore is 33.8%. This suggests that the Explorer continues to find new ways to make progress even in later rounds.
Adaptive fallback. To implement an adaptive fallback mechanism, at the exploration step, we first sample 4 Explorer rollouts and ask the model to judge whether each produces a plan that is both useful and valid. If fewer than 2/4 rollouts meet this criterion, we skip further Explorer sampling. The table below reports the results.
ArxivMath
| Iteration | Adaptive Pass@1 (%) | Solver.EXE Pass@1 (%) | Token change per problem |
|---|---|---|---|
| Iter 1 | 31.3 | 30.8 | −70.0k |
| Iter 2 | 33.7 | 34.2 | −86.9k |
| Iter 3 | 36.9 | 36.5 | −88.3k |
| Iter 4 | 37.1 | 37.2 | −57.2k |
Apex
| Iteration | Adaptive Pass@1 (%) | Solver.EXE Pass@1 (%) | Token change per problem |
|---|---|---|---|
| Iter 1 | 26.0 | 26.3 | −86.7k |
| Iter 2 | 31.5 | 31.3 | −65.5k |
| Iter 3 | 32.3 | 33.2 | −97.2k |
| Iter 4 | 34.9 | 36.5 | −79.5k |
The adaptive fallback reduces token usage per problem while keeping Pass@1 close to that of the full Solver.EXE pipeline. We hope these results will motivate further work on adaptive exploration at test time.
[1] Sutton and Barto (2018). Reinforcement Learning: An Introduction, 2nd ed.
[2] DeepSeek-AI (2025). DeepSeek-R1: Incentivizing Reasoning Capability in LLMs via Reinforcement Learning
W5: Please report total tokens, calls, and latency.
A: We report API costs in the main text as a common basis for comparison because GPU-hours are unavailable for the proprietary models. We report the actual cost comparison below. For the test-time learning methods, latency was measured on 8 H200 GPUs under each method's default four-round setting.
| Model or method | Output tokens per problem | Mean model calls per problem | Mean latency per problem | API cost per problem |
|---|---|---|---|---|
| GPT-5.2 | 38.3k | 1 | 10.4 min | $0.5 |
| Opus-4.6 | 68.4k | 1 | 18.5 min | $1.71 |
| Solver.EXE | 3.75M | 148.1 | 26.4 min | $1.65 |
| RSE | 4.09M | 133 | 30.6 min | $2.20 |
| RSA | 3.66M | 256 | 25.2 min | $1.81 |
- Among the TTL methods, Solver.EXE keeps inference costs within a reasonable range while achieving substantial performance gains.
- Compared with proprietary models, Solver.EXE enables near-frontier performance on self-hosted hardware without incurring proprietary API fees.
A kind reminder as the deadline is approaching
Dear Reviewer PqBx,
We hope this message finds you well. As the discussion period is entering its final day, we wanted to check in to ensure that we have adequately addressed your concerns. Your constructive feedback and suggestions are very important to us, and we would greatly appreciate any further thoughts you might have.
Thank you again for your time and effort in reviewing our paper and rebuttal.
Best,
The authors
Rebuttal Summary and Key Revisions
Thank you for taking the time to review our paper and share your feedback. As the rebuttal period draws to a close, we have summarized our main responses and revisions below for easy reference.
- W1. We report per-problem uncertainty in Table 9. Solver.EXE's gains over Opus-4.6 are statistically significant on multiple problems in both benchmarks. We also report frontier-comparable performance on HLE-Math and HLE-STEM (200 problems) and GPQA-Diamond (198 problems). Additionally, we used more conservative expressions in lines 26–27, 39–41, 72–75, 272–273, and 320–323.
- W2. We extended the comparison between Simple Explore and Solver.EXE to four rounds with error bars in Table 10. Structured exploration yields substantial gains, primarily on medium-difficulty problems (base model Pass@1 in %%) and harder problems (base model Pass@1 %).
- W3. We added ablation results for all components. The results show that each component contributes to Solver.EXE's performance.
- W4.
- To assess the advantage of our structured exploration mechanism over prior stochastic methods, we compared Explorer trajectories with up to 128 stochastic rollouts from the base model. The Explorer found additional correct solutions to problems that the stochastic method could not solve even at a large search budget.
- We used GPT-6.1 Sol to assess rollout usefulness. It labeled more than 75% of structured exploration trajectories as useful, compared with just 33.8% for vanilla exploration.
- We introduced an adaptive mechanism that lets the model assess the usefulness of its initial rollouts and decide whether to continue exploring. This reduces token use while keeping Pass@1 close to that of the full Solver.EXE pipeline, suggesting an interesting direction for future work on exploration mechanism design.
- W5. We report output tokens, calls, and latency for test-time learning methods and proprietary APIs. Solver.EXE achieves substantial performance gains over test-time learning baselines at comparable inference costs. It also achieves near-frontier performance on self-hosted hardware without incurring proprietary API fees.
We hope this summary is helpful and hope that you will take these new results and revisions into account in your final assessment. Thank you again for your time and thoughtful review.
Rebuttal-Part1
We sincerely appreciate your recognition of the clarity of our method and the strong empirical gains it achieves. We hope our responses below address your concerns.
A: Thank you for your suggestion.
Per-problem uncertainty. We report per-problem comparisons in Table 9 with 95% Newcombe–Wilson confidence intervals. After Holm correction, the directional Fisher exact tests favor Solver.EXE on 3 Apex and 5 ArxivMath problems, and favor Opus-4.6 on 2 and 5 problems, respectively. Notably, with test-time orchestration, Solver.EXE produced correct answers on 3 Apex problems and 2 ArxivMath problems for which Opus-4.6 produced none.
Scale of benchmark. Our paper has also included evaluations on HLE-Math and HLE-STEM with 200 problems, and GPQA-Diamond with 198 problems. Across these benchmarks, Solver.EXE improves over both the backbone and test-time learning baseline, while achieving performance comparable to the referenced model results.
Scope of claim. We have tightened the wording of the relevant claims, clarifying that open-source backbones equipped with Solver.EXE can achieve frontier-comparable performance on certain benchmarks. Our core claim remains that Solver.EXE consistently improves over both the raw backbone and prior test-time learning baselines.
A: Yes, and the benefit of structured exploration is substantial. In Table 10, we have extended the comparison between Simple Explore and Solver.EXE from two to four evolution rounds. The results are shown as follows:
ArxivMath
Apex
After four rounds, Solver.EXE leads Simple Explore by +3.6 percentage points on ArxivMath and +4.3 points on Apex. The gains come primarily from 4 medium-difficulty problems (baseline Pass@1∈(20%, 80%]) and 7 challenging problems (base Pass@1 ≤20%). This suggests that our structured exploration could effectively guide the model to discover useful strategies on problems it struggles with and thus raise the performance ceiling.
A: We have added the ablation results using Qwen3.5-35B-A3B and show them below.
ArxivMath
Apex
Across both benchmarks, the ablations suggest that the Extractor, both memory types, and both exploration modes contribute to Solver.EXE's performance.