acceptodds
Under review as a conference paper at ICLR 2027

WHALE: A SIMPLE RECIPE FOR JOINT HARNESS-WEIGHT OPTIMIZATION

Abstract

Agent performance depends jointly on the model parameters and the executable harness code that manages context and control flow. Optimizing either component in isolation can leave the system bottlenecked by its frozen counterpart: weight updates can change which harness is effective, while harness updates can change which model capabilities are exposed. Existing joint-adaptation methods optimize weights and textual prompts but leave the broader harness fixed. We propose **W**eight-**H**arness **A**lternating **LE**arning (**WHALE**), a simple recipe that alternates two phases: updating the model under the current harness, then searching for a better harness under the updated model. We instantiate these two phases with online rejection-sampling fine-tuning and Meta-Harness, respectively. When to switch is a key design choice: to separate real improvements from noise without over-optimizing against a changing counterpart, WHALE uses either fixed phase durations or an adaptive patience rule over training signals. Using Qwen3.5-2B/4B agents across three domains (search question answering, mathematical reasoning, and chess puzzles), WHALE outperforms weight-only, harness-only, and Fast-Slow Training by 4.15-24.38 percentage points in best mean@8 accuracy. Either component can be the bottleneck: harness search matches peak weight-only accuracy with far fewer rollouts in SearchQA, but improves math accuracy only after a weight update. Small interleaved updates also outperform stagewise weight-then-harness optimization in accuracy and rollout cost.

Primary area:
other topics in machine learning (i.e., none of the above)
Keywords:
Large Language Model, LLM Agents, Agent Harness

Reviews

Among:
Shape:
1234this paper, Top 49%
Figure 1. Mean review rating of the reviewed papers at ICLR 2027. This paper averages 2.33 over 3 reviews, above 51% of the others.
Reviewer_AW9JOpenReview ↗
Rating
3
Confidence
3
Informativeness
2
Presentation
3
Soundness
2
Read review
Paper summary

The paper proposes WHALE, which alternates between fine-tuning an agent's model weights under a fixed harness and searching for a better harness under the fixed model. The harness is the code controlling prompts, tool I/O, context and termination. Weight updates use online rejection-sampling fine-tuning; harness search uses Meta-Harness. An adaptive version switches phases when the training signal stops improving.

With Qwen3.5-2B/4B on search QA, AIME math and chess puzzles, WHALE beats weight-only, harness-only and a prompt-only variant of Fast–Slow Training by 4–24 points. The paper also reports that the limiting component depends on the domain, and that short alternating phases beat running each component once to completion.

Critical strengths

The paper addresses a real gap: agents depend on both their weights and their harness, yet these are usually optimized separately. Its behavior analysis gives a plausible picture of how the two interact in each domain, and the schedule study is a useful design for future work. That is the paper's main value.

Critical weaknesses

The evidence doesn't yet show that the gains come from joint optimization. Checkpoints are selected by test accuracy, each configuration is run once, and the math test set is small, so many reported differences are uncertain. Beyond that, the gains may come largely from loosening the starting harness's limits, from a weaker weight-update baseline, or from splitting training into short phases. None of these is ruled out. Validation-based selection, multiple seeds, an RL baseline, and controls that isolate the harness's contribution would make me reconsider my score.

Questions for authors
  1. All results are the best test accuracy reached during training. What happens if you choose checkpoints on a separate validation set instead? How much do the numbers vary across random seeds?
  2. Does the harness actually change from cycle to cycle, especially in math? If you kept the same training schedule but skipped harness search, how close would that get to WHALE?
  3. If weight-only used the same turn limits and retrieval settings as WHALE's final harness, how much of the gap would remain?
  4. Why use rejection-sampling fine-tuning rather than RL like Search-R1 and ReTool? Did you try WHALE with GRPO?
  5. It would be good to see results on a larger model, along with the proposer's compute cost.
8 comments
AuthorsOpenReview ↗

WHALE Retains Its Gains Across Four Runs with Validation-Based Selection

We thank reviewer AW9J for the insightful and constructive feedback. Your comments helped us strengthen our evaluation and clarify what the results support. Please see Appendix E, F, H, and I of the revised paper for further details. Please let us know if you have any remaining questions or concerns.

1. What happens if checkpoints are selected on a separate validation set? How much do the results vary across random seeds?

As you pointed out, the previous version selected weight–harness pairs based on test-set performance. This procedure may have led to overfitting to the test sets. In the revised paper, we use validation sets that are disjoint from their corresponding test sets. For each method, we select the weight–harness pair with the highest validation mean@8 accuracy and then evaluate it on the test set. We also ran each method four times, using a different random seed for each run.

Appendix E.1 describes the training, validation, and test splits. As you suggested, we also expanded the Math test set. Tables 4–6 in Appendix E.2 report the mean ± standard deviation across the four seeds. The Δ rows report gains in percentage points, with one-sided paired t-test p-values in parentheses (H₁: WHALE > baseline).

Table 4. SearchQA test mean@8 accuracy (%)

Method2WikiBamb.MuSiQuePopQATriviaAvg.
Harness-only26.41 ± 1.1838.03 ± 1.4614.56 ± 1.2138.31 ± 1.4365.09 ± 1.3036.48 ± 0.18
Weight-only24.72 ± 1.8032.41 ± 1.2214.69 ± 0.8640.41 ± 1.0367.31 ± 0.7535.91 ± 0.48
FST20.28 ± 1.0025.69 ± 0.7113.59 ± 0.1640.34 ± 1.2063.45 ± 1.8132.67 ± 0.33
WHALE32.41 ± 1.2247.41 ± 0.8220.13 ± 1.0846.88 ± 0.2071.72 ± 0.6243.71 ± 0.42
Δ vs. Harness-only+6.00 (0.0054)+9.38 (0.0001)+5.56 (0.0038)+8.56 (0.0005)+6.63 (0.0006)+7.23 (0.0000)
Δ vs. Weight-only+7.69 (0.0015)+15.00 (0.0003)+5.44 (0.0006)+6.47 (0.0008)+4.41 (0.0007)+7.80 (0.0000)
Δ vs. FST+12.13 (0.0000)+21.72 (0.0000)+6.53 (0.0005)+6.53 (0.0011)+8.27 (0.0019)+11.04 (0.0000)

Table 5. Mathematical Reasoning test mean@8 accuracy (%)

MethodAIME24AIME25AIME26AMC23AMC24MinervaAvg.
Harness-only0.63 ± 0.541.25 ± 0.900.21 ± 0.2410.55 ± 1.795.07 ± 0.667.69 ± 0.334.23 ± 0.38
Weight-only11.56 ± 1.6815.42 ± 0.9014.27 ± 1.8849.45 ± 1.5637.08 ± 0.8323.59 ± 0.2825.23 ± 0.16
FST13.85 ± 1.9418.23 ± 1.2414.17 ± 1.2349.45 ± 0.9742.50 ± 1.4219.19 ± 0.2526.23 ± 0.48
WHALE24.48 ± 1.0423.23 ± 0.4018.23 ± 0.9253.44 ± 0.6840.00 ± 1.3225.71 ± 0.3430.85 ± 0.16
Δ vs. Harness-only+23.85 (0.0000)+21.98 (0.0000)+18.02 (0.0000)+42.89 (0.0000)+34.93 (0.0000)+18.03 (0.0000)+26.62 (0.0000)
Δ vs. Weight-only+12.92 (0.0001)+7.81 (0.0001)+3.96 (0.0053)+3.98 (0.0132)+2.92 (0.0345)+2.13 (0.0011)+5.62 (0.0000)
Δ vs. FST+10.63 (0.0007)+5.00 (0.0033)+4.06 (0.0084)+3.98 (0.0049)−2.50 (0.9273)+6.53 (0.0000)+4.62 (0.0001)

Table 6. Chess Puzzles test mean@8 accuracy (%)

MethodLichessAvg.
Harness-only18.76 ± 0.1818.76 ± 0.18
Weight-only20.00 ± 0.3020.00 ± 0.30
FST23.30 ± 0.4123.30 ± 0.41
WHALE30.26 ± 0.4030.26 ± 0.40
Δ vs. Harness-only+11.50 (0.0000)+11.50 (0.0000)
Δ vs. Weight-only+10.27 (0.0000)+10.27 (0.0000)
Δ vs. FST+6.96 (0.0002)+6.96 (0.0002)

WHALE retains a statistically significant advantage over every baseline on overall test accuracy when pair selection relies exclusively on validation accuracy. The gains range from 7.23 to 11.04 percentage points in SearchQA, from 4.62 to 26.62 in Mathematical Reasoning, and from 6.96 to 11.50 in Chess Puzzles.

AuthorsOpenReview ↗

Harness Evolution Across Cycles in Math

2-1. Does the harness actually change from cycle to cycle, especially in math?

Yes. In Math, WHALE with (E,I)=(0.6,6)(E,I)=(0.6,6) changes the selected harness throughout training. The changes affect code execution, error recovery, interaction budgets, and termination. Representative examples are:

CycleSelected harnessChanges introduced by this cycle
1MH1/h10Increases the assistant-turn limit from 2 to 5 and adds a final-turn instruction to commit a boxed answer.
2MH2/h12Adds AST-aware code normalization and replays previously successful code before executing new code, preserving earlier variable definitions across sandbox calls.
5MH5/h15Adds error-specific feedback, filters unsupported tool arguments, and blocks further tool execution after three consecutive errors while requesting a final answer.
6MH6/h5Increases the turn limit from 5 to 7 and adjusts the timing of intermediate and final-answer reminders.
7MH7/h9Checks new code for syntax errors before sandbox execution and returns focused diagnostics without adding invalid code to the replay history.
9MH9/h3Changes the stopping rule: a response containing neither a tool call nor a boxed answer receives a recovery instruction and can continue within the remaining budget.

For example, cycles 7 and 9 modify execution and termination behavior while keeping the seven-turn limit unchanged. Harness updates therefore continue beyond the early increases in interaction budget. We also allow the incoming harness to remain selected: cycle 3 retained the harness selected in cycle 2.

2-2. If you kept the same training schedule but skipped harness search, how close would that get to WHALE?

If we keep WHALE’s training schedule and omit harness search, we obtain the Weight-only experiment shown in Figure 2. Have I understood your question correctly? If I misunderstood, please feel free to ask a follow-up question. Thank you.

AuthorsOpenReview ↗

Assessing Initial Harness Restrictions with Harness-First Stagewise Optimization

3. If weight-only used the same turn limits and retrieval settings as WHALE's final harness, how much of the gap would remain?

Thank you for this suggestion. We did not run the exact experiment you proposed: weight-only training from the initial weights with WHALE’s final harness held fixed. Instead, we ran Stagewise (MH → RSFT) to examine whether relaxing the initial harness’s turn and tool-use limits before weight training could explain WHALE’s gains. We first ran MH for the full harness-search budget with the initial weights fixed, then froze the selected harness and trained the model from its initial weights using RSFT for the full weight-update budget. Please let us know if this control does not sufficiently address your concern or if you would like us to run an additional experiment.

Test results. Tables 7 and 8 in Appendix F.2 compare Stagewise (MH → RSFT) with the fixed WHALE schedules, Adaptive WHALE, and the baselines. Bold marks the highest average, and italics mark the second highest.

Table 7. SearchQA test mean@8 accuracy (%)

Run2WikiBamboogleMuSiQuePopQATriviaQAAvg.
Weight-only24.7232.4114.6940.4167.3135.91
Harness-only26.4138.0314.5638.3165.0936.48
WHALE (0.2, 2)36.7547.3820.8846.6373.0044.93
WHALE (0.2, 6)38.0055.2524.0051.5074.1348.58
WHALE (0.6, 2)36.3852.1325.3845.5073.7546.63
WHALE (0.6, 6)32.4147.4120.1346.8871.7243.71
Stagewise (RSFT → MH)28.1341.5016.5046.2573.1341.10
Stagewise (MH → RSFT)36.8852.1323.3844.3871.2545.60
Adaptive WHALE41.5055.5029.7549.5075.2550.30

Table 8. Math test mean@8 accuracy (%)

RunAIME24AIME25AIME26AMC23AMC24MinervaAvg.
Weight-only11.5615.4214.2749.4537.0823.5925.23
Harness-only0.631.250.2110.555.077.694.23
WHALE (0.2, 2)14.1717.088.7542.1937.5020.1723.31
WHALE (0.2, 6)25.0323.3618.7850.3943.6124.2330.90
WHALE (0.6, 2)18.7515.0012.9240.9439.4417.6524.12
WHALE (0.6, 6)24.4823.2318.2353.4440.0025.7130.85
Stagewise (RSFT → MH)13.7517.5015.8347.1941.3923.5826.54
Stagewise (MH → RSFT)2.087.924.1729.6923.8915.2613.83
Adaptive WHALE25.8325.8327.5058.4448.6125.8735.35

In SearchQA, Stagewise (MH → RSFT) reached 45.60%, below WHALE with (E, I) = (0.2, 6), (0.6, 2), and adaptive WHALE. In Math, Stagewise (MH → RSFT) reached 13.83%, below every WHALE schedule.

Harness comparison. Table 9 compares the initial harness (h_init), the harness from the Adaptive WHALE pair with the highest validation accuracy (h_WHALE), and the harness used in Stagewise (MH → RSFT) (h_MH→RSFT).

Table 9. Harness configurations in SearchQA and Math

DomainConfigurationh_inith_WHALEh_MH→RSFT
SearchQAMax. assistant turns244
SearchQADocuments per request138
SearchQATokens per document200200200
SearchQAQueries per tool call11–31–3
MathMax. assistant turns275

In SearchQA, h_MH→RSFT matches or exceeds the h_WHALE on all four configurations. Adaptive WHALE achieves higher test accuracy. This result suggests that alternating weight and harness updates contributes to the gains beyond the effect of relaxing these limits alone.

In Math, h_MH→RSFT allows more assistant turns than h_init. However, the Weight-only baseline achieves higher test accuracy with the h_init. This result suggests that increasing the turn limit alone is insufficient. It also suggests that prolonged harness search may have over-optimized the harness for the initial weights. Alternating updates at moderate intervals lets the weights and harness adapt to each other throughout training.

Stagewise (MH → RSFT) outperformed some WHALE schedules in SearchQA, but none in Math. This pattern is consistent with the dominance regimes discussed in Section 6.1. In weight-dominant Math, searching harness with the initial weights fixed may provide a noisy signal about which harness will perform well after weight updates. In harness-dominant SearchQA, an extensive harness search before weight updates can still perform well. However, when we encounter a new domain, we cannot tell in advance whether it is harness-dominant or weight-dominant. Thus, alternating weight and harness updates at moderate intervals makes WHALE a reasonable choice in this setting.

Please see Appendix F in revised paper for details.

Reviewer_75GTOpenReview ↗
Rating
2
Confidence
3
Informativeness
2
Presentation
2
Soundness
1
Read review
Paper summary

The paper proposes WHALE to jointly optimize model weights and executable agent harnesses. Each cycle fine-tunes the model on its own verified successful trajectories under the current harness, then uses Meta-Harness to search for a better harness while holding the updated model fixed. The aim is to let both components adapt as the other changes. Phase lengths are either fixed or determined by a patience rule on training scores. Experiments on search question answering, mathematics, and chess compare this approach with weight-only, harness-only, prompt-only, and stagewise optimization, and examine the effect of phase length.

Critical strengths
  1. SIA [1], HASE [2], and Co-Harness [3] already optimize model weights and executable harnesses together. Co-Harness is particularly close: it alternates harness and model updates and fine-tunes on high-quality trajectories generated under the improved harness. The paper does not discuss these works or compare against them. The authors need to explain what the RSFT and Meta-Harness combination, or the scheduling experiments, adds beyond these methods.

  2. Algorithms 1 and 2 return the model–harness pair with the highest test accuracy, and the experiments mainly report the best test score observed during training. Selecting on the test set can bias the reported results upward and complicate comparisons between methods. Although phase switching uses training scores, the final pair is still selected by test accuracy. The adaptive rules specify when to switch phases, but a plateau in one phase does not establish convergence of the alternating process. The experiments use finite budgets, while the pseudocode has no outer stopping condition. The authors should specify how the overall run ends and use a separate validation set to select its output.

  3. Five fixed schedules are tested, but the adaptive window, minimum phase length, patience, candidate count, and scoring settings are not varied. Each candidate is evaluated once per problem on the same 256 problems, and any improvement resets patience. Small score differences may therefore reflect sampling noise and affect both harness selection and phase switching. Increasing the number of search rounds does not make each candidate's score more precise. Sensitivity tests and results across several training seeds are needed. On Math, the adaptive version scores 26.46%, below the best fixed schedule at 28.33%, so its ability to replace schedule tuning is not yet clear.

  4. The initial SearchQA and Math harnesses allow two assistant turns, while the search space permits up to sixteen. Some of the improvement may come from allowing more interaction. The full-harness versus prompt-only comparison does not isolate this effect or the contributions of retrieval, parsing, and feedback changes. Comparisons with matched inference budgets and ablations of these components would help identify where the gains come from. Rollout counts also do not capture differences in trajectory length, tool calls, or proposer computation, so they are not enough to establish the computational advantage.

References

[1] Prannay Hebbar et al. SIA: Self Improving AI with Harness & Weight Updates. arXiv:2605.27276, 2026. https://arxiv.org/abs/2605.27276

[2] Haochen Luo et al. Harness-Aware Self-Evolving: Co-Evolving Model Weights, Harness, and Task Solutions. arXiv:2607.03935, 2026. https://arxiv.org/abs/2607.03935

[3] Zhengyu Chen et al. Co-Harness: Co-Evolving Harnesses and Model Weights for LLM Agents. arXiv:2607.22688, 2026. https://arxiv.org/abs/2607.22688

Critical weaknesses
  1. All five GSM8K models improve over standard ES, but the paper does not report variation across training seeds or confidence intervals. The average GSM8K gains over GRPO and ESSAM are about 0.27 and 0.25 percentage points, and EMA adds only 0.23–0.31 percentage points in the paired ablations. Results from several training seeds are needed to judge whether these small differences are reproducible.

  2. The ablations cover only GSM8K with a 0.5B model. Without random replay under the same total budget, it is unclear how much of the gain comes from the targeted rule and how much comes from repeating problems. Simple pass-rate filtering and alternative replay priorities would also be useful comparisons. The paper does not test sensitivity to EMA decay, replay ratio, or mastery threshold. Checking the main ablations on a larger model or harder task would help show whether the findings extend beyond this setting.

  3. Each epoch continues until all fresh problems have been visited, and replay takes up part of each batch. Matching epoch counts can therefore give DART more updates and generations. The paper says the ablations use the same training budget, but does not report the actual update counts, problem visits, and generated tokens for each setting. Training steps are explicitly matched for instruction tuning and code generation; the same information is needed for GSM8K, MATH, and the 14B experiments. The MATH baselines are taken from prior work, so it is also unclear whether their training and evaluation settings are comparable to DART's.

  4. The runtime and memory comparison includes GRPO and GRPO+LoRA but omits standard ES. It therefore does not show the overhead of adding weighting and replay to ES. Per-step timings are also difficult to compare when the amount of generation per step and the total number of steps differ. The authors should report runtime and memory against standard ES, along with performance at a fixed total compute budget or the total time and generation needed to reach the same score.

Questions for authors
  1. SIA [1], HASE [2], and Co-Harness [3] already optimize model weights and executable harnesses together. Co-Harness is particularly close: it alternates harness and model updates and fine-tunes on high-quality trajectories generated under the improved harness. The paper does not discuss these works or compare against them. The authors need to explain what the RSFT and Meta-Harness combination, or the scheduling experiments, adds beyond these methods.

  2. Algorithms 1 and 2 return the model–harness pair with the highest test accuracy, and the experiments mainly report the best test score observed during training. Selecting on the test set can bias the reported results upward and complicate comparisons between methods. Although phase switching uses training scores, the final pair is still selected by test accuracy. The adaptive rules specify when to switch phases, but a plateau in one phase does not establish convergence of the alternating process. The experiments use finite budgets, while the pseudocode has no outer stopping condition. The authors should specify how the overall run ends and use a separate validation set to select its output.

  3. Five fixed schedules are tested, but the adaptive window, minimum phase length, patience, candidate count, and scoring settings are not varied. Each candidate is evaluated once per problem on the same 256 problems, and any improvement resets patience. Small score differences may therefore reflect sampling noise and affect both harness selection and phase switching. Increasing the number of search rounds does not make each candidate's score more precise. Sensitivity tests and results across several training seeds are needed. On Math, the adaptive version scores 26.46%, below the best fixed schedule at 28.33%, so its ability to replace schedule tuning is not yet clear.

  4. The initial SearchQA and Math harnesses allow two assistant turns, while the search space permits up to sixteen. Some of the improvement may come from allowing more interaction. The full-harness versus prompt-only comparison does not isolate this effect or the contributions of retrieval, parsing, and feedback changes. Comparisons with matched inference budgets and ablations of these components would help identify where the gains come from. Rollout counts also do not capture differences in trajectory length, tool calls, or proposer computation, so they are not enough to establish the computational advantage.

References

[1] Prannay Hebbar et al. SIA: Self Improving AI with Harness & Weight Updates. arXiv:2605.27276, 2026. https://arxiv.org/abs/2605.27276

[2] Haochen Luo et al. Harness-Aware Self-Evolving: Co-Evolving Model Weights, Harness, and Task Solutions. arXiv:2607.03935, 2026. https://arxiv.org/abs/2607.03935

[3] Zhengyu Chen et al. Co-Harness: Co-Evolving Harnesses and Model Weights for LLM Agents. arXiv:2607.22688, 2026. https://arxiv.org/abs/2607.22688

6 comments
AuthorsOpenReview ↗

Relationship to SIA, HASE, and Co-Harness

We thank reviewer 75GT for the careful reading and constructive feedback. Your comments helped us clarify WHALE’s contributions in relation to closely related work and strengthen the experimental evaluation in the revised paper. We address your questions below. Please see Appendix E, F and J of the revised paper for further details. Please let us know if you have any remaining questions or concerns.

1. Relationship to SIA, HASE, and Co-Harness

Thank you for pointing us to these relevant papers. We summarize their differences from WHALE below.

  • SIA lets a Feedback-Agent choose harness or weight updates and the training algorithm. WHALE studies explicit fixed schedules and training-signal-based stopping rules through controlled comparisons.
  • HASE trains a single policy to solve tasks and edit harnesses. WHALE separates weight training from external proposer-based harness search and keeps the task verifier fixed.
  • Co-Harness also alternates harness repair and model fine-tuning. WHALE emphasizes controlled schedule comparisons and analysis of harness-dominant and weight-dominant regimes across domains.

These works share our goal of joint weight-harness improvement. We will cite all three in the revised manuscript and clarify these relationships.

AuthorsOpenReview ↗

Validation-Based Selection and Overall Stopping

2. Validation-Based Selection and Overall Stopping

Thank you for highlighting these issues. We agree that selecting the best test score can bias the reported results, and that a plateau within one phase does not establish convergence of the alternating process.

Overall termination and algorithm output. We will make the following conditions explicit in Algorithms 1 and 2:

  • Termination: The overall run ends when both the pre-specified total weight-update budget and total harness-search budget have been exhausted. The adaptive patience rules determine when to switch phases within these budgets; they are not a convergence criterion for the overall process.
  • Output: Both algorithms return the evaluated weight-harness pair with the highest validation mean@8 accuracy. Test accuracy does not enter the output-selection rule. Phase switching continues to use training scores.

Validation-based test evaluation. In the revised paper, we use validation sets that are disjoint from their corresponding test sets. We select each method's best weight-harness pair using validation mean@8 accuracy, fix that pair, and evaluate it on the test set. We also ran each method four times, using a different random seed for each run.

Appendix E.1 describes the data splits. Tables 4, 5, and 6 in Appendix E.2 report the mean ± standard deviation over the four runs. Δ rows show gains in percentage points, with p-values from one-sided paired t-tests in parentheses (H1: WHALE > baseline).

Table 4. SearchQA test mean@8 accuracy (%)

Method2WikiBamb.MuSiQuePopQATriviaAvg.
Harness-only26.41 ± 1.1838.03 ± 1.4614.56 ± 1.2138.31 ± 1.4365.09 ± 1.3036.48 ± 0.18
Weight-only24.72 ± 1.8032.41 ± 1.2214.69 ± 0.8640.41 ± 1.0367.31 ± 0.7535.91 ± 0.48
FST20.28 ± 1.0025.69 ± 0.7113.59 ± 0.1640.34 ± 1.2063.45 ± 1.8132.67 ± 0.33
WHALE32.41 ± 1.2247.41 ± 0.8220.13 ± 1.0846.88 ± 0.2071.72 ± 0.6243.71 ± 0.42
Δ vs. Harness-only+6.00 (0.0054)+9.38 (0.0001)+5.56 (0.0038)+8.56 (0.0005)+6.63 (0.0006)+7.23 (0.0000)
Δ vs. Weight-only+7.69 (0.0015)+15.00 (0.0003)+5.44 (0.0006)+6.47 (0.0008)+4.41 (0.0007)+7.80 (0.0000)
Δ vs. FST+12.13 (0.0000)+21.72 (0.0000)+6.53 (0.0005)+6.53 (0.0011)+8.27 (0.0019)+11.04 (0.0000)

Table 5. Mathematical Reasoning test mean@8 accuracy (%)

MethodAIME24AIME25AIME26AMC23AMC24MinervaAvg.
Harness-only0.63 ± 0.541.25 ± 0.900.21 ± 0.2410.55 ± 1.795.07 ± 0.667.69 ± 0.334.23 ± 0.38
Weight-only11.56 ± 1.6815.42 ± 0.9014.27 ± 1.8849.45 ± 1.5637.08 ± 0.8323.59 ± 0.2825.23 ± 0.16
FST13.85 ± 1.9418.23 ± 1.2414.17 ± 1.2349.45 ± 0.9742.50 ± 1.4219.19 ± 0.2526.23 ± 0.48
WHALE24.48 ± 1.0423.23 ± 0.4018.23 ± 0.9253.44 ± 0.6840.00 ± 1.3225.71 ± 0.3430.85 ± 0.16
Δ vs. Harness-only+23.85 (0.0000)+21.98 (0.0000)+18.02 (0.0000)+42.89 (0.0000)+34.93 (0.0000)+18.03 (0.0000)+26.62 (0.0000)
Δ vs. Weight-only+12.92 (0.0001)+7.81 (0.0001)+3.96 (0.0053)+3.98 (0.0132)+2.92 (0.0345)+2.13 (0.0011)+5.62 (0.0000)
Δ vs. FST+10.63 (0.0007)+5.00 (0.0033)+4.06 (0.0084)+3.98 (0.0049)−2.50 (0.9273)+6.53 (0.0000)+4.62 (0.0001)

Table 6. Chess Puzzles test mean@8 accuracy (%)

MethodLichessAvg.
Harness-only18.76 ± 0.1818.76 ± 0.18
Weight-only20.00 ± 0.3020.00 ± 0.30
FST23.30 ± 0.4123.30 ± 0.41
WHALE30.26 ± 0.4030.26 ± 0.40
Δ vs. Harness-only+11.50 (0.0000)+11.50 (0.0000)
Δ vs. Weight-only+10.27 (0.0000)+10.27 (0.0000)
Δ vs. FST+6.96 (0.0002)+6.96 (0.0002)

On the overall test set for each domain, WHALE outperforms every baseline by a statistically significant margin. The gains range from 7.23 to 11.04 percentage points in SearchQA, from 4.62 to 26.62 in Mathematical Reasoning, and from 6.96 to 11.50 in Chess Puzzles. These results show that WHALE retains its advantage when pair selection relies exclusively on validation accuracy.

Please see Appendix E of the revised paper for the data splits and test results.

AuthorsOpenReview ↗

Sensitivity of Adaptive WHALE

3. Sensitivity of Adaptive WHALE

Thank you for this feedback. We added a targeted sensitivity analysis of Adaptive WHALE in Appendix J. We jointly reduced the weight-phase averaging window, minimum phase length, and patience from 0.2 to 0.1 epoch, keeping the harness-search settings fixed.

The table below summarizes test mean@8 accuracy (%) from Tables 13 and 14. Bold and underlining mark the highest and second-highest score, respectively.

RunSearchQAMath
Weight-only35.9125.23
Harness-only36.484.23
WHALE (0.2, 2)44.9323.31
WHALE (0.2, 6)48.5830.90
WHALE (0.6, 2)46.6324.12
WHALE (0.6, 6)43.7130.85
WHALE (1.0, 10)44.0829.66
Stagewise (RSFT → MH)41.1026.54
Stagewise (MH → RSFT)45.6013.83
Adaptive WHALE (0.2)50.3035.35
Adaptive WHALE (0.1)50.05‾\underline{50.05}30.93‾\underline{30.93}

Halving the three stopping parameters lowered accuracy by 0.25 percentage points in SearchQA and 4.42 percentage points in Math. Both adaptive WHALE settings still achieved higher average scores than the fixed schedules and stagewise controls in these comparisons. These results show sensitivity to the stopping parameters, particularly in Math, while supporting adaptive phase switching as a practical alternative to selecting fixed per-cycle budgets.

The original Math comparison of 26.46% versus 28.33% used AIME 2024/2025. Under the revised validation-based selection and test protocol in Appendix E, the default Adaptive WHALE scores 35.35%, compared with 30.85% for the strongest fixed schedule. These are results under a different evaluation protocol, rather than a correction to the original AIME-only scores.

Sampling noise. We agree that candidate scores are stochastic estimates, so small score differences can affect harness selection and phase switching. Our patience rule is a heuristic for phase transitions; it does not guarantee that individual candidate improvements are statistically significant. Nevertheless, Adaptive WHALE achieves higher average test accuracy when pairs are selected using a separate validation set and evaluated on the test set.

We also report test results over four sampling seeds for the main-method comparisons in our earlier response, "2. Validation-Based Selection and Overall Stopping." These are repeated test evaluations of fixed validation-selected weight-harness pairs.

Please see Appendix J of the revised paper for details.

Reviewer_1eBjOpenReview ↗
Rating
2
Confidence
3
Informativeness
2
Presentation
3
Soundness
2
Read review
Paper summary

This work proposes a framework for alternatively optimizing two key components of AI agents: model weights and the harness. Experiments on three datasets demonstrate that the proposed approach consistently outperforms weight-only, harness-only, and Fast–Slow Training. Furthermore, extensive experiments investigate the adaptability of this alternating optimization framework across different tasks and analyze how different alternating schedules affect performance.

Critical strengths
  1. Relevant and well-motivated problem. Agent performance depends on both model capabilities and the surrounding execution logic. Extending joint adaptation beyond textual prompts to tool interfaces, context management, parsing, and termination policies is practically meaningful.
  2. Clear and modular approach. The alternating procedure is straightforward and accommodates different weight-update and harness-search algorithms. The paper clearly describes its concrete implementation and adaptive switching rule.
  3. Encouraging empirical coverage. The evaluation spans three distinct domains, and Table 2 reports improvements across every evaluated benchmark subset. The single-component, prompt-restricted, and stagewise controls address relevant aspects of the contribution.
  4. Useful behavioral analysis. Figure 3 and Appendix D connect performance changes to retrieval, answer formatting, truncation, and tool use. These analyses provide more insight than aggregate accuracy alone.
Critical weaknesses
  1. The paper lacks evaluation on out-of-distribution (OOD) test sets. Alternatively optimizing both the model weights and the harness on a fixed dataset may lead to over-optimizing against that specific dataset, raising concerns about the generalization ability of the proposed framework.
  2. Restrictive initial harnesses make the source of the gains difficult to isolate. Appendix B.2 specifies that the initial SearchQA and Math harnesses allow only two assistant turns, with SearchQA returning a single passage truncated to 200 tokens. The examples in Appendix D illustrate failures directly associated with these restrictions, while the optimized harnesses increase the interaction budget and modify termination behavior. Thus, part of the improvement may arise from correcting limitations of the initial setup, leaving the additional benefit of repeated weight–harness co-adaptation unclear. The harness-only and weight-then-harness baselines partially address this concern, but neither establishes whether a reasonably configured fixed harness followed by weight training—or harness search followed by training with the selected harness frozen—would recover comparable gains. Such a comparison under matched inference budgets would help isolate the contribution of repeated alternation. It would also clarify whether the reported “harness-dominant” and “model-dominant” regimes reflect broader task characteristics or primarily the particular initialization and resource constraints used in these experiments.
  3. The paper lacks a thorough evaluation of the generalization capabilities of the optimized model weights and harness independently. Alternating optimization of these two components may lead to strong coupling between them, potentially limiting their transferability. It remains unclear whether pairing the optimized model with a different harness, or applying the optimized harness to a different model, would result in a significant performance degradation. Additional cross-combination experiments would help assess the extent of this coupling and the independent generalizability of each component.
Questions for authors

Please see weaknesses.

8 comments
AuthorsOpenReview ↗

Validation-Based Pair Selection and Held-Out Test Evaluation

We thank reviewer 1eBj for the insightful and constructive feedback. Your comments helped us gain valuable insights and draw helpful conclusions. We address your comments below. Please see Appendix E, F and G of the revised paper for further details. Please let us know if you have any remaining questions or concerns.

1. The paper lacks evaluation on out-of-distribution (OOD) test sets.

As you pointed out, we selected weight–harness pairs based on test-set performance in the previous version. This selection procedure may have led to overfitting to the test sets. In the revised paper, we constructed validation sets that are disjoint from their corresponding test sets. We selected the best weight–harness pair based on validation mean@8 accuracy and then evaluated it on the test sets.

Appendix E.1 describes the training, validation, and test splits. Tables 4, 5, and 6 in Appendix E.2 report the results for SearchQA, Mathematical Reasoning, and Chess Puzzles, respectively. We reproduce all three tables below. Each table reports the mean ± standard deviation over four runs. Δ rows show gains in percentage points, with p-values in parentheses (H1: WHALE > baseline).

Table 4. SearchQA test mean@8 accuracy (%)

Method2WikiBamb.MuSiQuePopQATriviaAvg.
Harness-only26.41 ± 1.1838.03 ± 1.4614.56 ± 1.2138.31 ± 1.4365.09 ± 1.3036.48 ± 0.18
Weight-only24.72 ± 1.8032.41 ± 1.2214.69 ± 0.8640.41 ± 1.0367.31 ± 0.7535.91 ± 0.48
FST20.28 ± 1.0025.69 ± 0.7113.59 ± 0.1640.34 ± 1.2063.45 ± 1.8132.67 ± 0.33
WHALE32.41 ± 1.2247.41 ± 0.8220.13 ± 1.0846.88 ± 0.2071.72 ± 0.6243.71 ± 0.42
Δ vs. Harness-only+6.00 (0.0054)+9.38 (0.0001)+5.56 (0.0038)+8.56 (0.0005)+6.63 (0.0006)+7.23 (0.0000)
Δ vs. Weight-only+7.69 (0.0015)+15.00 (0.0003)+5.44 (0.0006)+6.47 (0.0008)+4.41 (0.0007)+7.80 (0.0000)
Δ vs. FST+12.13 (0.0000)+21.72 (0.0000)+6.53 (0.0005)+6.53 (0.0011)+8.27 (0.0019)+11.04 (0.0000)

Table 5. Mathematical Reasoning test mean@8 accuracy (%)

MethodAIME24AIME25AIME26AMC23AMC24MinervaAvg.
Harness-only0.63 ± 0.541.25 ± 0.900.21 ± 0.2410.55 ± 1.795.07 ± 0.667.69 ± 0.334.23 ± 0.38
Weight-only11.56 ± 1.6815.42 ± 0.9014.27 ± 1.8849.45 ± 1.5637.08 ± 0.8323.59 ± 0.2825.23 ± 0.16
FST13.85 ± 1.9418.23 ± 1.2414.17 ± 1.2349.45 ± 0.9742.50 ± 1.4219.19 ± 0.2526.23 ± 0.48
WHALE24.48 ± 1.0423.23 ± 0.4018.23 ± 0.9253.44 ± 0.6840.00 ± 1.3225.71 ± 0.3430.85 ± 0.16
Δ vs. Harness-only+23.85 (0.0000)+21.98 (0.0000)+18.02 (0.0000)+42.89 (0.0000)+34.93 (0.0000)+18.03 (0.0000)+26.62 (0.0000)
Δ vs. Weight-only+12.92 (0.0001)+7.81 (0.0001)+3.96 (0.0053)+3.98 (0.0132)+2.92 (0.0345)+2.13 (0.0011)+5.62 (0.0000)
Δ vs. FST+10.63 (0.0007)+5.00 (0.0033)+4.06 (0.0084)+3.98 (0.0049)−2.50 (0.9273)+6.53 (0.0000)+4.62 (0.0001)

Table 6. Chess Puzzles test mean@8 accuracy (%)

MethodLichessAvg.
Harness-only18.76 ± 0.1818.76 ± 0.18
Weight-only20.00 ± 0.3020.00 ± 0.30
FST23.30 ± 0.4123.30 ± 0.41
WHALE30.26 ± 0.4030.26 ± 0.40
Δ vs. Harness-only+11.50 (0.0000)+11.50 (0.0000)
Δ vs. Weight-only+10.27 (0.0000)+10.27 (0.0000)
Δ vs. FST+6.96 (0.0002)+6.96 (0.0002)

On the overall test set for each domain, WHALE outperforms every baseline by a statistically significant margin. The gains range from 7.23 to 11.04 percentage points in SearchQA, from 4.62 to 26.62 in Mathematical Reasoning, and from 6.96 to 11.50 in Chess Puzzles. These results show that WHALE retains its advantage when pair selection relies exclusively on validation accuracy.

AuthorsOpenReview ↗

HARNESS SEARCH BEFORE WEIGHT UPDATES

2. Restrictive initial harnesses make the source of the gains difficult to isolate.

We tested whether relaxing limits in the initial harness, such as the maximum number of assistant turns or tool calls, could explain WHALE's gains. We ran Meta-Harness (MH) for the full harness-search budget with the initial weights fixed. We then fixed the accepted harness and trained the model from its initial weights using RSFT for the full weight-update budget. We call this schedule Stagewise (MH → RSFT).

Test results. Tables 7 and 8 in Appendix F.2 compare Stagewise (MH → RSFT) with the fixed WHALE schedules, Adaptive WHALE, and the baselines. Bold marks the highest average, and italics mark the second highest.

Table 7. SearchQA test mean@8 accuracy (%)

Run2WikiBamboogleMuSiQuePopQATriviaQAAvg.
Weight-only24.7232.4114.6940.4167.3135.91
Harness-only26.4138.0314.5638.3165.0936.48
WHALE (0.2, 2)36.7547.3820.8846.6373.0044.93
WHALE (0.2, 6)38.0055.2524.0051.5074.1348.58
WHALE (0.6, 2)36.3852.1325.3845.5073.7546.63
WHALE (0.6, 6)32.4147.4120.1346.8871.7243.71
WHALE (1.0, 10)28.2548.6323.7547.3872.3844.08
Stagewise (RSFT → MH)28.1341.5016.5046.2573.1341.10
Stagewise (MH → RSFT)36.8852.1323.3844.3871.2545.60
Adaptive WHALE41.5055.5029.7549.5075.2550.30

Table 8. Math test mean@8 accuracy (%)

RunAIME24AIME25AIME26AMC23AMC24MinervaAvg.
Weight-only11.5615.4214.2749.4537.0823.5925.23
Harness-only0.631.250.2110.555.077.694.23
WHALE (0.2, 2)14.1717.088.7542.1937.5020.1723.31
WHALE (0.2, 6)25.0323.3618.7850.3943.6124.2330.90
WHALE (0.6, 2)18.7515.0012.9240.9439.4417.6524.12
WHALE (0.6, 6)24.4823.2318.2353.4440.0025.7130.85
WHALE (1.0, 10)23.7520.4219.1751.2539.7223.6729.66
Stagewise (RSFT → MH)13.7517.5015.8347.1941.3923.5826.54
Stagewise (MH → RSFT)2.087.924.1729.6923.8915.2613.83
Adaptive WHALE25.8325.8327.5058.4448.6125.8735.35

In SearchQA, Stagewise (MH → RSFT) reached 45.60%, below WHALE with (E, I) = (0.2, 6), (0.6, 2), and adaptive WHALE. In Math, Stagewise (MH → RSFT) reached 13.83%, below every WHALE schedule.

Harness comparison. Table 9 compares the initial harness (h_init), the harness from the Adaptive WHALE pair with the highest validation accuracy (h_WHALE), and the harness used in Stagewise (MH → RSFT) (h_MH→RSFT).

Table 9. Harness configurations in SearchQA and Math

DomainConfigurationh_inith_WHALEh_MH→RSFT
SearchQAMax. assistant turns244
SearchQADocuments per request138
SearchQATokens per document200200200
SearchQAQueries per tool call11–31–3
MathMax. assistant turns275

In SearchQA, h_MH→RSFT matches or exceeds the h_WHALE on all four configurations. Adaptive WHALE achieves higher test accuracy. This result suggests that alternating weight and harness updates contributes to the gains beyond the effect of relaxing these limits alone.

In Math, h_MH→RSFT allows more assistant turns than h_init. However, the Weight-only baseline achieves higher test accuracy with the h_init. This result suggests that increasing the turn limit alone is insufficient. It also suggests that prolonged harness search may have over-optimized the harness for the initial weights. Alternating updates at moderate intervals lets the weights and harness adapt to each other throughout training.

Stagewise (MH → RSFT) outperformed some WHALE schedules in SearchQA, but none in Math. This pattern is consistent with the dominance regimes discussed in Section 6.1. In weight-dominant Math, searching harness with the initial weights fixed may provide a noisy signal about which harness will perform well after weight updates. In harness-dominant SearchQA, an extensive harness search before weight updates can still perform well. However, when we encounter a new domain, we cannot tell in advance whether it is harness-dominant or weight-dominant. Thus, alternating weight and harness updates at moderate intervals makes WHALE a reasonable choice in this setting.

Please see Appendix F in revised paper for details.

AuthorsOpenReview ↗

WHALE’s Optimized Weights and Harnesses Generalize Across Cycles

3. The paper lacks a thorough evaluation of the generalization capabilities of the optimized model weights and harness independently.

To assess each component’s ability to generalize, we paired the selected weights with other harnesses and the selected harness with other weight checkpoints.

The tables below reproduce Figure 8 in appendix G. It covers cycles 1–6 in SearchQA and cycles 1, 3, 5, 7, and 9 in Math from the WHALE training runs under (E, I)=(0.6, 6). For each included cycle, we selected the best weights and harness by validation mean@8 accuracy. We then measured test mean@8 accuracy for the resulting combinations using the test sets. Bold marks the selected pair based on the validation score. Underlining marks the other pairings that use either the selected weights or the selected harness.

SearchQA

Weight cycleHarness 1Harness 2Harness 3Harness 4Harness 5Harness 6Mean
133.5735.1234.75‾\underline{34.75}18.6826.6529.6229.73
243.9243.8544.40‾\underline{44.40}30.3038.4234.5239.24
343.7544.1240.90‾\underline{40.90}34.0039.8837.7540.07
445.32‾\underline{45.32}45.27‾\underline{45.27}45.8838.83‾\underline{38.83}42.52‾\underline{42.52}40.83‾\underline{40.83}43.11
542.0842.5545.12‾\underline{45.12}39.8541.7539.4541.80
642.0042.4243.77‾\underline{43.77}35.8538.6238.2040.15
Mean41.7742.2342.4732.9237.9836.73—

Mathematical Reasoning

Weight cycleHarness 1Harness 3Harness 5Harness 7Harness 9Mean
17.718.226.818.137.24‾\underline{7.24}7.62
318.2019.5219.0419.4119.22‾\underline{19.22}19.08
526.2927.5427.7327.9827.80‾\underline{27.80}27.47
727.1729.5329.6028.6530.23‾\underline{30.23}29.04
929.27‾\underline{29.27}29.18‾\underline{29.18}29.10‾\underline{29.10}30.13‾\underline{30.13}30.8729.71
Mean21.7322.8022.4622.8623.07—

In both domains, the weights from the selected pair achieve the highest mean accuracy across the evaluated harnesses. The harness from that pair also achieves the highest mean accuracy across the evaluated weight checkpoints. These results suggest that WHALE jointly optimizes the weights and harness while preserving each component’s ability to generalize across pairings.

Please see Appendix G of the revised paper for details.

From OpenReview, CC BY 4.0. Authors as signed there; shown unedited.

Related papers

Loading the map…

Discussion (0)

Sign in to comment.