- Rating
- 3
- Confidence
- 3
- Informativeness
- 2
- Presentation
- 3
- Soundness
- 2
Read review
Paper summary
The paper proposes WHALE, which alternates between fine-tuning an agent's model weights under a fixed harness and searching for a better harness under the fixed model. The harness is the code controlling prompts, tool I/O, context and termination. Weight updates use online rejection-sampling fine-tuning; harness search uses Meta-Harness. An adaptive version switches phases when the training signal stops improving.
With Qwen3.5-2B/4B on search QA, AIME math and chess puzzles, WHALE beats weight-only, harness-only and a prompt-only variant of Fast–Slow Training by 4–24 points. The paper also reports that the limiting component depends on the domain, and that short alternating phases beat running each component once to completion.
Critical strengths
The paper addresses a real gap: agents depend on both their weights and their harness, yet these are usually optimized separately. Its behavior analysis gives a plausible picture of how the two interact in each domain, and the schedule study is a useful design for future work. That is the paper's main value.
Critical weaknesses
The evidence doesn't yet show that the gains come from joint optimization. Checkpoints are selected by test accuracy, each configuration is run once, and the math test set is small, so many reported differences are uncertain. Beyond that, the gains may come largely from loosening the starting harness's limits, from a weaker weight-update baseline, or from splitting training into short phases. None of these is ruled out. Validation-based selection, multiple seeds, an RL baseline, and controls that isolate the harness's contribution would make me reconsider my score.
Questions for authors
- All results are the best test accuracy reached during training. What happens if you choose checkpoints on a separate validation set instead? How much do the numbers vary across random seeds?
- Does the harness actually change from cycle to cycle, especially in math? If you kept the same training schedule but skipped harness search, how close would that get to WHALE?
- If weight-only used the same turn limits and retrieval settings as WHALE's final harness, how much of the gap would remain?
- Why use rejection-sampling fine-tuning rather than RL like Search-R1 and ReTool? Did you try WHALE with GRPO?
- It would be good to see results on a larger model, along with the proposer's compute cost.
8 comments
Harness Evolution Across Cycles in Math
2-1. Does the harness actually change from cycle to cycle, especially in math?
Yes. In Math, WHALE with changes the selected harness throughout training. The changes affect code execution, error recovery, interaction budgets, and termination. Representative examples are:
| Cycle | Selected harness | Changes introduced by this cycle |
|---|---|---|
| 1 | MH1/h10 | Increases the assistant-turn limit from 2 to 5 and adds a final-turn instruction to commit a boxed answer. |
| 2 | MH2/h12 | Adds AST-aware code normalization and replays previously successful code before executing new code, preserving earlier variable definitions across sandbox calls. |
| 5 | MH5/h15 | Adds error-specific feedback, filters unsupported tool arguments, and blocks further tool execution after three consecutive errors while requesting a final answer. |
| 6 | MH6/h5 | Increases the turn limit from 5 to 7 and adjusts the timing of intermediate and final-answer reminders. |
| 7 | MH7/h9 | Checks new code for syntax errors before sandbox execution and returns focused diagnostics without adding invalid code to the replay history. |
| 9 | MH9/h3 | Changes the stopping rule: a response containing neither a tool call nor a boxed answer receives a recovery instruction and can continue within the remaining budget. |
For example, cycles 7 and 9 modify execution and termination behavior while keeping the seven-turn limit unchanged. Harness updates therefore continue beyond the early increases in interaction budget. We also allow the incoming harness to remain selected: cycle 3 retained the harness selected in cycle 2.
2-2. If you kept the same training schedule but skipped harness search, how close would that get to WHALE?
If we keep WHALE’s training schedule and omit harness search, we obtain the Weight-only experiment shown in Figure 2. Have I understood your question correctly? If I misunderstood, please feel free to ask a follow-up question. Thank you.
Assessing Initial Harness Restrictions with Harness-First Stagewise Optimization
3. If weight-only used the same turn limits and retrieval settings as WHALE's final harness, how much of the gap would remain?
Thank you for this suggestion. We did not run the exact experiment you proposed: weight-only training from the initial weights with WHALE’s final harness held fixed. Instead, we ran Stagewise (MH → RSFT) to examine whether relaxing the initial harness’s turn and tool-use limits before weight training could explain WHALE’s gains. We first ran MH for the full harness-search budget with the initial weights fixed, then froze the selected harness and trained the model from its initial weights using RSFT for the full weight-update budget. Please let us know if this control does not sufficiently address your concern or if you would like us to run an additional experiment.
Test results. Tables 7 and 8 in Appendix F.2 compare Stagewise (MH → RSFT) with the fixed WHALE schedules, Adaptive WHALE, and the baselines. Bold marks the highest average, and italics mark the second highest.
Table 7. SearchQA test mean@8 accuracy (%)
| Run | 2Wiki | Bamboogle | MuSiQue | PopQA | TriviaQA | Avg. |
|---|---|---|---|---|---|---|
| Weight-only | 24.72 | 32.41 | 14.69 | 40.41 | 67.31 | 35.91 |
| Harness-only | 26.41 | 38.03 | 14.56 | 38.31 | 65.09 | 36.48 |
| WHALE (0.2, 2) | 36.75 | 47.38 | 20.88 | 46.63 | 73.00 | 44.93 |
| WHALE (0.2, 6) | 38.00 | 55.25 | 24.00 | 51.50 | 74.13 | 48.58 |
| WHALE (0.6, 2) | 36.38 | 52.13 | 25.38 | 45.50 | 73.75 | 46.63 |
| WHALE (0.6, 6) | 32.41 | 47.41 | 20.13 | 46.88 | 71.72 | 43.71 |
| Stagewise (RSFT → MH) | 28.13 | 41.50 | 16.50 | 46.25 | 73.13 | 41.10 |
| Stagewise (MH → RSFT) | 36.88 | 52.13 | 23.38 | 44.38 | 71.25 | 45.60 |
| Adaptive WHALE | 41.50 | 55.50 | 29.75 | 49.50 | 75.25 | 50.30 |
Table 8. Math test mean@8 accuracy (%)
| Run | AIME24 | AIME25 | AIME26 | AMC23 | AMC24 | Minerva | Avg. |
|---|---|---|---|---|---|---|---|
| Weight-only | 11.56 | 15.42 | 14.27 | 49.45 | 37.08 | 23.59 | 25.23 |
| Harness-only | 0.63 | 1.25 | 0.21 | 10.55 | 5.07 | 7.69 | 4.23 |
| WHALE (0.2, 2) | 14.17 | 17.08 | 8.75 | 42.19 | 37.50 | 20.17 | 23.31 |
| WHALE (0.2, 6) | 25.03 | 23.36 | 18.78 | 50.39 | 43.61 | 24.23 | 30.90 |
| WHALE (0.6, 2) | 18.75 | 15.00 | 12.92 | 40.94 | 39.44 | 17.65 | 24.12 |
| WHALE (0.6, 6) | 24.48 | 23.23 | 18.23 | 53.44 | 40.00 | 25.71 | 30.85 |
| Stagewise (RSFT → MH) | 13.75 | 17.50 | 15.83 | 47.19 | 41.39 | 23.58 | 26.54 |
| Stagewise (MH → RSFT) | 2.08 | 7.92 | 4.17 | 29.69 | 23.89 | 15.26 | 13.83 |
| Adaptive WHALE | 25.83 | 25.83 | 27.50 | 58.44 | 48.61 | 25.87 | 35.35 |
In SearchQA, Stagewise (MH → RSFT) reached 45.60%, below WHALE with (E, I) = (0.2, 6), (0.6, 2), and adaptive WHALE. In Math, Stagewise (MH → RSFT) reached 13.83%, below every WHALE schedule.
Harness comparison. Table 9 compares the initial harness (h_init), the harness from the Adaptive WHALE pair with the highest validation accuracy (h_WHALE), and the harness used in Stagewise (MH → RSFT) (h_MH→RSFT).
Table 9. Harness configurations in SearchQA and Math
| Domain | Configuration | h_init | h_WHALE | h_MH→RSFT |
|---|---|---|---|---|
| SearchQA | Max. assistant turns | 2 | 4 | 4 |
| SearchQA | Documents per request | 1 | 3 | 8 |
| SearchQA | Tokens per document | 200 | 200 | 200 |
| SearchQA | Queries per tool call | 1 | 1–3 | 1–3 |
| Math | Max. assistant turns | 2 | 7 | 5 |
In SearchQA, h_MH→RSFT matches or exceeds the h_WHALE on all four configurations. Adaptive WHALE achieves higher test accuracy. This result suggests that alternating weight and harness updates contributes to the gains beyond the effect of relaxing these limits alone.
In Math, h_MH→RSFT allows more assistant turns than h_init. However, the Weight-only baseline achieves higher test accuracy with the h_init. This result suggests that increasing the turn limit alone is insufficient. It also suggests that prolonged harness search may have over-optimized the harness for the initial weights. Alternating updates at moderate intervals lets the weights and harness adapt to each other throughout training.
Stagewise (MH → RSFT) outperformed some WHALE schedules in SearchQA, but none in Math. This pattern is consistent with the dominance regimes discussed in Section 6.1. In weight-dominant Math, searching harness with the initial weights fixed may provide a noisy signal about which harness will perform well after weight updates. In harness-dominant SearchQA, an extensive harness search before weight updates can still perform well. However, when we encounter a new domain, we cannot tell in advance whether it is harness-dominant or weight-dominant. Thus, alternating weight and harness updates at moderate intervals makes WHALE a reasonable choice in this setting.
Please see Appendix F in revised paper for details.
WHALE Retains Its Gains Across Four Runs with Validation-Based Selection
We thank reviewer AW9J for the insightful and constructive feedback. Your comments helped us strengthen our evaluation and clarify what the results support. Please see Appendix E, F, H, and I of the revised paper for further details. Please let us know if you have any remaining questions or concerns.
1. What happens if checkpoints are selected on a separate validation set? How much do the results vary across random seeds?
As you pointed out, the previous version selected weight–harness pairs based on test-set performance. This procedure may have led to overfitting to the test sets. In the revised paper, we use validation sets that are disjoint from their corresponding test sets. For each method, we select the weight–harness pair with the highest validation mean@8 accuracy and then evaluate it on the test set. We also ran each method four times, using a different random seed for each run.
Appendix E.1 describes the training, validation, and test splits. As you suggested, we also expanded the Math test set. Tables 4–6 in Appendix E.2 report the mean ± standard deviation across the four seeds. The Δ rows report gains in percentage points, with one-sided paired t-test p-values in parentheses (H₁: WHALE > baseline).
Table 4. SearchQA test mean@8 accuracy (%)
Table 5. Mathematical Reasoning test mean@8 accuracy (%)
Table 6. Chess Puzzles test mean@8 accuracy (%)
WHALE retains a statistically significant advantage over every baseline on overall test accuracy when pair selection relies exclusively on validation accuracy. The gains range from 7.23 to 11.04 percentage points in SearchQA, from 4.62 to 26.62 in Mathematical Reasoning, and from 6.96 to 11.50 in Chess Puzzles.