acceptodds
Under review as a conference paper at ICLR 2027

Harness Training: A Post-Training View of Agent Harness Evolution

Abstract

Large language model agents depend on an agent harness that comprises persistent prompts, skills, tools, middleware, and workflows governing their behavior. Harness evolution updates this state after deployment without changing model weights. Existing work is commonly categorized by the component being edited or the search algorithm, rather than by whether an update is retained to reproduce target behavior or improve execution outcomes. We frame harness evolution as Harness Training, an analogue of parameter post-training at the level of learning objectives. This view distinguishes Harness-SFT, which learns from target trajectories, from Harness-RL, which learns from reward feedback. Experiments show that Harness-SFT makes agent executions more closely follow target trajectories, while Harness-RL changes retained behavior in the direction predicted by the reward objective. When combined, Harness-SFT increases the availability of task-solving edits, and Harness-RL uses reward feedback to identify and retain them, improving reward search. Based on this finding, we develop Signal-Targeted Harness Training (STHT), which applies Harness-SFT to tasks failed by the initial harness before initializing Harness-RL from the supervised state. Across diverse experimental settings, STHT reaches validation thresholds with fewer rollouts and outperforms competing methods on unseen tasks. These gains transfer across harnesses, reward optimizers, and teachers. Code is available anonymously at https://anonymous.4open.science/r/harness-train-45C6/.

Primary area:
foundation or frontier models, including LLMs
Keywords:
large language model, agent harness, agent self-evolution

Reviews

Among:
Shape:
1234this paper, Bottom 27%
Figure 1. Mean review rating of the reviewed papers at ICLR 2027. This paper averages 2.00 over 3 reviews, above 27% of the others.
Reviewer_Ro4AOpenReview ↗
Rating
2
Confidence
3
Informativeness
2
Presentation
2
Soundness
2
Read review
Paper summary

The paper studies how to improve an agent’s prompts, skills, tools and workflow while keeping its model fixed. Harness-SFT chooses edits that make the agent follow teacher demonstrations more closely, while Harness-RL chooses edits using task rewards. The proposed method, STHT, first learns from demonstrations for tasks the initial agent failed, then continues with reward-based search. Experiments cover SpreadsheetBench, a DocVQA pilot, ALFWorld, and a coding subset of EvoAgentBench. STHT has the highest reported mean of the methods in Table 1 in all 8 settings, averaging 80.37% success across the 5 primary settings versus 77.28% with randomly selected demonstrations.

Critical strengths
  1. The distinction between what an edit changes and how it's selected is useful. It's a clear way to organize methods that modify different parts of an agent. The authors are also clear that the comparison with model training is just an analogy. They don’t claim that editing a harness works like gradient-based training.

  2. The experiments include a few different useful controls. The comparison with random task selection uses the same number of demonstrations, and the gains hold across different harnesses, reward optimizers, and teachers. On ALFWorld, targeting failures also beats selecting successful tasks or tasks whose executions differ most from the teacher’s. I appreciated the comparison between starting harnesses with almost identical performance. The supervised starting point still supports better subsequent search, which suggests that its benefit goes beyond just the higher initial score.

Critical weaknesses
  1. The variation across runs is not clear. The main gains over random selection range from 1.78 to 4.64 percentage points, but the paper doesn't provide standard deviations, intervals, or statistical tests. The results are described as three-run averages, but Section 6.1 presents the ALFWorld result as 239/274 solved tasks, and Appendix E.3 refers to “14 of 32 and 21 of 32 groups in the reported run”. These counts could be consistent with averages, but the aggregation needs explaining.

  2. I’m not sure the comparisons are fair on cost or on baseline strength. STHT gets to the validation threshold in fewer rollouts, but it costs 8.7% more overall than random selection, and a lot more than RL alone. It's not clear whether that cost includes generating and verifying teacher demonstrations. There's also no RL-only comparison given STHT’s total budget. The default optimizer, Parent Acceptance, is the weakest method in Table 12. Bandit Allocation reaches 91.61% on ALFWorld there, compared with STHT’s 87.23% in Table 1. The two use different training setups, so this isn’t a direct comparison. But the question of whether the budget would be better spent on a stronger search is still open.

  3. Parts of the protocol need clarifying. Section 4 says training tasks choose candidate edits, but the SpreadsheetBench code sets the selection split to validation. If so, validation AUC and time to threshold are measured on selection data, but the test results are still held out. The same ALFWorld result also shows up as Table 11’s Success-only row, Table 12’s Parent Acceptance row, and Table 1’s RL-only entry, each described differently. Finally, it’s unclear what the modifier was told in each Table 11 condition, which matters for reading it as RL-like behavior.

Questions for authors
  1. Could you report per-run results and uncertainty estimates for Tables 1-4? How do the counts in Section 6.1 and Appendix E.3 relate to the three-run averages, and does Appendix H use the same configuration as Table 1?

  2. When deciding whether to keep a candidate edit, which tasks do you evaluate it on? Training tasks, the validation split named in the SpreadsheetBench code, or a separate selection fold like in Appendix H? How many tasks and rollouts go into each decision? Are those rollouts, and the initial failure-screening runs, counted in the 256-rollout budget and in time to threshold? How does validation AUC select a single checkpoint?

  3. Does total cost include teacher generation and verification? Could you compare STHT with reward-only search at the same total cost?

  4. Which editing scopes produced Table 11’s Success-only and Table 12’s Parent Acceptance rows? What reward instructions did the modifier get in each Table 11 condition, and what did the shared-candidate comparison described in Appendix D.2 show?

2 comments

We thank the reviewer for the careful reading of both the manuscript and the code.

Q1. Aggregation, uncertainty, and Appendix H

  • Counts vs. three-run averages (Appendix B.8, Eq. 33). If the three seeds solve a,b,ca,b,c of NN test tasks, we report p=100N round(a+b+c3)p=\frac{100}{N}\,\mathrm{round}(\frac{a+b+c}{3}), so table percentages correspond to integer counts in the prose (239/274 = 87.23% on ALFWorld; 14/32 and 21/32 groups in E.3). The ± value is the sample SD of 100a/N100a/N, 100b/N100b/N, 100c/N100c/N.
  • Appendix H is not the Table 1 configuration. It is a single-seed diagnostic (10 Skill-SFT + 10 full-harness RL rounds, 8-task in-training selection fold) reaching 206/280 (73.57%). Table 1 uses 32 rounds (256 rollouts) and three seeds (78.21±0.74%). This is now stated in Appendix H (Table 18).
  • Mean ± SD is now reported in Tables 1–4. Selected rows from Table 1 (test %):
SettingUniform SFT→RLSFT→RLSTHT
Spreadsheet/Pi76.43±0.9077.14±1.2978.21±0.74
DocVQA/Pi83.69±0.5684.76±0.6786.63±0.53
EvoAgentBench/Pi61.63±2.4262.79±4.0365.12±2.42
ALFWorld/Pi84.67±1.4785.40±1.5987.23±1.93
Spreadsheet/Codex80.00±0.5581.79±0.6284.64±0.21
5-setting macro (35B)77.28±0.5578.38±0.8480.37±0.68
3-setting macro (9B)72.04±0.5273.61±0.5175.93±0.17

Table 2 (STHT vs. Uniform): successful-round rate 0.711±0.036 vs. 0.618±0.037; validation AUC 0.804±0.007 vs. 0.776±0.007. Table 3 (failure-targeted vs. Uniform): 0.716±0.118 vs. 0.625±0.121 rounds with success; AUC 0.817±0.015 vs. 0.790±0.015.

Q2. Retention split, rollout accounting, checkpoint selection

  • Candidate retention uses only training tasks, never validation or test. In the main protocol, each of the 32 rounds samples one task xt∈D2x_t\in\mathcal{D}_2 and evaluates m=8m=8 candidates on it (8 rollouts per decision, 256 total). Appendix H instead uses an 8-task fold carved from the training split.
  • SELECTION_SPLIT_NAME. In modifier-system-v1 this variable is only a label in the modifier's boundary warning, and its value "SpreadsheetBench validation" was misleading. We verified that the runner's retention step evaluates candidates only on training tasks, and we will correct the label in the repository. The modifier never sees validation tasks or traces; validation is used only for AUC, TqT_q, and selecting H⋆H^\star.
  • Accounting. B=256B=256 and TqT_q count only Stage-2 Harness-RL candidate rollouts. All Stage-1 costs (failure screening on D1\mathcal{D}_1, teacher generation, SFT rollouts and edits), modifier calls, and validation checkpointing are included in end-to-end token cost, including Cost at TqT_q (Tables 2, 12, 13).
  • Checkpoint selection. AUC summarizes the curve; the test checkpoint is H⋆=arg⁡max⁡HℓRˉval(Hℓ)H^\star=\arg\max_{H_\ell}\bar R_{\text{val}}(H_\ell), with ties broken by the latest checkpoint (Eq. 12).

Q3. Teacher cost, equal-cost reward-only search, optimizer strength

  • Teacher generation and verification are included. Macro STHT cost is 50.6M tokens (35.5M task model + 6.3M teacher + 8.8M modifier; Table 13).
  • STHT is not the cheapest way to reach a fixed threshold. RL only reaches it at lower token cost (0.464 vs. 0.560 normalized). STHT's advantage is accuracy: higher success at the same total cost and higher final success (80.37% vs. 76.08% macro). In Table 5, with STHT's teacher and SFT budget reallocated to reward search, reward-only reaches 82.62% (DocVQA/Pi) and 77.14% (Spreadsheet/Codex), and the strongest external system (AutoSaddler) reaches 83.42% and 81.43%, vs. 86.63% and 84.64% for STHT.
  • Bandit Allocation (Table 15). Standalone Bandit (91.61%) runs multi-lineage search over all 60 ALFWorld training tasks, whereas Table 1 splits them into D1\mathcal{D}_1/D2\mathcal{D}_2 and uses Parent Acceptance for all methods. Failure targeting is orthogonal to the search rule and improves stronger optimizers (Group-Relative, AutoSaddler; Table 4).

Q4. Scopes and modifier instructions for Tables 14–15

  • Both tables start from Pi with full-harness scope {P,S,T,W,L}\{P,S,T,W,L\} (typo fixed in Appendix D.1), so Table 14 Success-only, Table 15 Parent Acceptance, and Table 1 ALFWorld/Pi RL only are the same run (86.13%).
  • Round-1 shared candidates (Appendix D.2). The modifier receives one reward-neutral diagnosis prompt, and the same 8 cached executions are scored under each reward. Selection alone shifts tool calls (14.8→11.2), steps (13.4→9.6), and verification coverage (12.5%→48.2%) relative to RsuccessR_{\text{success}}.
  • Multi-round search (Table 14). The modifier additionally receives the active REWARD_CONTRACT (Appendix I), and the shifts compound to 9.80 tool calls, 8.20 steps, and 64.96% verification coverage on held-out tasks.
AuthorsOpenReview ↗

A brief note on the response (Submission 54869)

Dear Reviewer Ro4A,

Thank you again for your detailed review of both the paper and the code. With the discussion period ending soon, we wanted to check whether our response has resolved your questions on the statistics, protocol, and cost comparisons:

  1. Uncertainty and aggregation (Tables 1–4, Appendix B.8 & H): added mean ± SD across the three seeds, explained the integer-count rounding rule (Eq. 33), and clarified the single-seed diagnostic setup in Appendix H (Table 18).
  2. Selection split and rollout accounting (Appendix B.4, Tables 11–13): confirmed that per-round retention uses only training tasks (not the validation split), clarified how B=256B=256, TqT_q, and Stage-1 costs are accounted, and stated the arg⁡max⁡Rˉval\arg\max \bar R_{\text{val}} checkpoint rule.
  3. Equal-cost reward-only search and teacher cost (Tables 5, 13) & Table 14–15 scopes/prompts (Appendix D.1–D.2).

Please let us know if anything remains unclear; we are happy to provide any further details.

Best regards, The Authors

Reviewer_CdsgOpenReview ↗
Rating
2
Confidence
3
Informativeness
2
Presentation
2
Soundness
2
Read review
Paper summary

The paper proposes to view "harness evolution" through the lens of parameter post-training. It separates Harness-SFT (retain harness edits that make the frozen agent's tool-call trajectories closer to teacher trajectories, measured by normalized edit distance) from Harness-RL (retain edits by task reward). The paper argues that Harness-SFT behaves like SFT (responds to task–trajectory pairing, data scale and teacher quality), Harness-RL behaves like RL (responds to reward design and update rule), and proposes STHT: execute the initial harness on half of the training tasks, obtain teacher trajectories only for the failed tasks, apply sequential skill-scoped Harness-SFT, then run Harness-RL from the resulting harness on the other half.

Critical strengths
  1. The composition study is carefully controlled.

  2. STHT evaluation covers a breadth of dataset/harness/actor settings.

Critical weaknesses
  1. The algorithmic contribution is thin and the margins are small. The conceptual contribution is largely a re-labelling of existing methods. It's more explanation-based and not problem-based. And the empirical margins supporting the specific "failure targeting" claim are small: STHT vs all-task SFT→RL is +1.1 to +2.9 points.

  2. SFT→RL starts from Teacher SFT trained on the full 60-task split with demonstrations for both D1 and D2, yet loses to STHT, which uses demonstrations for at most ~30 failed D1 tasks. Success-Targeted supervision is likewise worse than Uniform (Table 3). So more correct teacher demonstrations hurt, and the paper's explanation does not say why. Plausible causes would make this an artifact of the update procedure rather than a general principle. The RQ1–RQ3 mechanism experiments are also each run on one benchmark with the Pi harness, and Teacher SFT alone gains only ~1–3 points (Table 1), so the "SFT-like" regime is weak in absolute terms.

Questions for authors
  1. Why does all-task teacher supervision (Teacher SFT → SFT→RL) underperform failure-only supervision?

  2. How much of STHT's final gain comes from the skill text vs the full-harness RL edits to P/T/W/L? A STHT variant with Skill-only RL, and a Full-harness-RL-only run from H0 with the same budget, would separate these.

2 comments

We thank the reviewer for the constructive review and for highlighting our controlled composition study and broad evaluation. The corresponding results are in Sections 6.2 and 6.4 (Table 5) and Appendix B.7 (Tables 12–13).

Q1 / W2. Why all-task and success-targeted supervision underperform failure targeting

Supervising tasks that H0H_0 already solves (rx(0)=1r^{(0)}_x=1) hurts for two reasons (Section 6.2):

  1. Stylistic over-constraining. When H0H_0 already solves a task, the residual NED to the teacher reflects ordering or style differences between two valid solutions, not a missing capability. Minimizing it with sequential Skill-SFT (Eq. 10) writes non-essential constraints into SKILL.md. Table 3 supports this: High-NED (largest distance, regardless of success) trails failure targeting (85.04±1.84% vs. 87.23±1.93%), and Success-Targeted lowers both edit availability (0.497 vs. 0.625) and test success (82.12±1.72% vs. 84.67±1.47%) relative to Uniform.
  2. Recency and dilution in sequential discrete updates. SKILL.md is a compact in-context artifact that is sensitive to update order (Appendix F, Table 16). Since H0H_0 already solves 50–78% of training tasks, over 70% of All-Task SFT→RL's Stage-1 updates target solved tasks, which dilutes or overwrites remedies needed for failures and lowers downstream edit availability from 0.711 (STHT) to 0.651 (Table 2).

Q2. Skill text (SS) vs. full-harness RL edits (P,T,W,LP,T,W,L), and equal-budget RL

Following your suggestion, Section 6.4 / Table 5 compares, from the same H0H_0 under a matched total-cost cap, STHT Stage 1 only, STHT with Skill-only RL, reward-only full-harness RL (STHT's teacher and SFT budget reallocated to reward search), external Harness-RL systems, and Full STHT:

Method (matched total cost)Scope (SFT→RL)DocVQA/PiSpreadsheet/Codex
H0H_0—→—77.8176.43
Reward-only full-harness RL—→{P..L}82.6277.14
SkillOpt / GEPA—→{S} / {P,S}82.89 / 82.0980.36 / 80.00
Meta-Harness / AutoSaddler—→{P..L}83.16 / 83.4281.07 / 81.43
STHT Stage 1 only{S}→—81.82 (+4.01)77.86 (+1.43)
STHT, Skill-only RL{S}→{S}84.49 (+6.68)80.36 (+3.93)
Full STHT{S}→{P..L}86.63 (+8.82)84.64 (+8.21)
  • Equal-budget reward-only RL trails Full STHT by 4.01 and 7.50 pp, so the gain comes from spending part of the budget on failure-targeted supervision, not from budget size.
  • The split is task-dependent. On DocVQA, Skill-only STHT realizes 6.68 of the 8.82 pp gain (76%); opening P,T,W,LP,T,W,L adds 2.14 pp. On SpreadsheetBench/Codex, Skill-only realizes 3.93 of 8.21 pp (48%), and P,T,W,LP,T,W,L contribute the larger share (4.28 pp, 52%), addressing non-procedural bottlenecks such as layout salience in PP (Appendix H.2).

W1 / W2. Contribution, margins, and standalone Teacher SFT

  • Problem-driven design. We use the standard propose–execute–evaluate–retain loop as a controlled testbed to verify post-training regularities in discrete harness evolution (RQ1–RQ3), and use them to address sparse-reward stalling: 58.8% of RL-only rounds contain no verifier-passing candidate (Table 2).
  • Margins. Against Uniform SFT→RL (matched supervision budget), STHT gains +1.78 to +4.64 pp in all 8 settings (+3.09 pp macro on 35B, +3.89 pp on 9B). Against All-Task SFT→RL, it gains +1.99 pp (35B) and +2.32 pp (9B) macro, with 50–78% fewer teacher demonstrations and 19% fewer tokens at TqT_q (28.3M vs. 35.1M, Table 13).
  • Standalone Teacher SFT is restricted to ΩSFT={S}\Omega_{\text{SFT}}=\{S\} and optimizes imitation (NED), not reward. Its main role is catalytic: it reshapes the local edit space so that subsequent RL finds verifier-passing edits more often (0.711 vs. 0.412 for RL only across the five primary settings, Table 2; Figures 7–8).
AuthorsOpenReview ↗

A brief note on the response (Submission 54869)

Dear Reviewer Cdsg,

Thank you again for your constructive feedback. As the discussion deadline approaches, we wanted to check whether our response and new experiments have addressed your two main questions:

  1. Separating Skill text (SS) vs. P/T/W/LP/T/W/L edits and equal-budget RL (Table 5): we added Stage-1-only, Skill-only RL, and budget-matched full-harness reward-only search from H0H_0, showing that equal-budget reward-only trails Full STHT by 4.01 and 7.50 pp and quantifying the task-dependent share of Skill vs. full-harness edits.
  2. Why failure-only supervision beats all-task and success-targeted supervision (Section 6.2, Table 3): supervising already-solved tasks introduces stylistic over-constraining into SKILL.md and dilutes failure remedies under sequential updates, lowering downstream edit availability (0.651 vs. 0.711).

We would be happy to discuss any remaining points before the window closes.

Best regards, The Authors

Reviewer_fsPkOpenReview ↗
Rating
2
Confidence
2
Informativeness
2
Presentation
2
Soundness
2
Read review
Paper summary

This paper presents Harness Training, an objective-level framework for optimizing LLM agents while keeping the task-model parameters fixed. It distinguishes the component being edited, e.g., prompts, skills, tools, middleware, or workflows, from the learning signal. Harness-SFT retains harness edits that make agent behavior closer to target trajectories, while Harness-RL selects edits using task rewards, preferences, or verifier outcomes. Building on this distinction, the paper proposes Signal-Targeted Harness Training (STHT), which identifies tasks failed by the initial harness, applies teacher-trajectory supervision specifically to these tasks, and then performs full-harness reward search on a separate data split. Experiments on SpreadsheetBench, an instrumented DocVQA pilot, ALFWorld, and the Code Implementation subset of EvoAgentBench show that STHT consistently outperforms the paper’s internal training variants. Across five primary settings, STHT achieves a macro test success of 80.37%, compared with 78.38% for the strongest non-STHT variant and 77.28% for Uniform SFT→RL. Additional experiments study trajectory quality, reward interventions, reward-matched initialization, edit availability, task selection, and transfer across actors, teachers, harnesses, and optimizers.

Critical strengths
  1. The paper provides a useful framework for understanding agent harness optimization by distinguishing trajectory imitation from reward-based search, independently of which component is modified.

  2. The experiments investigate how supervised initialization benefits subsequent reward search. After matching initial performance, supervised initialization still improves validation AUC from 0.713 to 0.753, suggesting that its benefits extend beyond a stronger starting point.

Critical weaknesses
  1. Limited algorithmic novelty. The method is essentially LLM-guided discrete search with a propose–execute–evaluate–retain loop. STHT mainly adds failure-targeted supervision and better search initialization.

  2. Insufficient external baselines. The paper compares internal variants but not complete existing systems such as GEPA, Meta-Harness, or SkillOpt under matched budgets.

  3. Efficiency claims are incomplete. STHT reduces downstream rollouts but costs about 8.7% more end-to-end than Uniform SFT→RL; absolute compute and API costs are unreported.

Questions for authors

Please see the Critical Weaknesses.

2 comments

We thank the reviewer for recognizing our objective-level framework and the reward-matched initialization study (Figure 7). All new experiments and cost breakdowns below are included in the revised manuscript.

W1. Contribution vs. the standard self-evolving loop

We agree that the propose–execute–evaluate–retain loop is the standard substrate of self-evolving agents (Voyager, OPRO, AFlow, GEPA, Meta-Harness, AutoSaddler), and we do not claim novelty for this loop. Our contribution is twofold:

  1. Post-training regularities in discrete harness space (RQ1–RQ3). SFT-like: trajectory-supervised updates respond to task–trajectory pairing (Correct vs. Shuffled vs. No-Trace), demonstration scale (N∈{8,16,32,48,All}N\in\{8,16,32,48,\text{All}\}), and teacher quality (Figures 3–4). RL-like: outcome-driven updates shift tool calls, trajectory length, and verification coverage in the direction of the reward, even on an identical shared candidate pool (Appendix D.2), and show exploration–exploitation trade-offs across update rules (Tables 14–15). SFT–RL interaction: against a reward-only checkpoint with nearly identical starting reward (Δ=0.002\Delta=0.002) and per-task profile (Δ=0.021\Delta=0.021), Teacher SFT raises the fraction of RL rounds containing a verifier-passing edit from 43.8% to 65.6% (m=8m=8) and cuts rollouts to the first retained update by 33.3% (Figures 7–8).
  2. STHT follows from these findings. RQ3 shows that SFT helps RL mainly by making task-solving edits available where reward search stalls, and Table 3 shows that supervising already-solved tasks over-constrains the Skill. STHT therefore targets supervision at initial failures Dfail(H0)\mathcal{D}_{\text{fail}}(H_0). Because it builds on the standard loop, it transfers across harnesses (Pi, Codex), actors (35B, 9B), teachers (GPT-5.5, Qwen3.5), and optimizers (Parent Acceptance, Group-Relative, AutoSaddler; Table 4).

W2. Matched-budget comparison with complete external systems

We added Section 6.4 / Table 5: SkillOpt, GEPA, Meta-Harness, and AutoSaddler run as complete pipelines from the same H0H_0 (Qwen3.6-35B-A3B actor, GPT-5.5 optimizer) under a common total-cost cap matched to STHT, plus budget-matched reward-only search and STHT ablations.

MethodScope (SFT→RL)DocVQA/PiSpreadsheet/Codex
H0H_0—→—77.8176.43
SkillOpt—→{S}82.8980.36
GEPA—→{P,S}82.0980.00
Meta-Harness—→{P..L}83.1681.07
AutoSaddler—→{P..L}83.4281.43
Reward-only (ours)—→{P..L}82.6277.14
STHT Stage 1 only{S}→—81.8277.86
STHT, Skill-only RL{S}→{S}84.4980.36
Full STHT{S}→{P..L}86.6384.64

All four external systems are pure Harness-RL in our taxonomy (Table 6). At the same total cost, Full STHT exceeds the strongest one, AutoSaddler, by +3.21 pp on both settings, and equal-cost reward-only search by +4.01 / +7.50 pp. STHT can also serve as an initializer for external optimizers such as AutoSaddler (Table 4).

W3. Cost accounting

Table 2 and Appendix B.7 (Tables 12–13) now report absolute tokens (prompt + completion) for all three roles: task model θ0\theta_0, teacher TT (including verified trajectory generation), and modifier GG. Macro average over the five primary settings:

MethodTask modelTeacherModifierFull cost (norm.)Median TqT_qCost at TqT_q (norm.)
RL only25.2M—6.2M31.4M (0.620)19223.5M (0.464)
Teacher SFT8.6M9.3M5.4M23.3M (0.460)n/a—
Self SFT→RL35.6M—8.4M44.0M (0.870)14430.8M (0.608)
Uniform SFT→RL32.1M5.9M8.6M46.6M (0.920)11229.2M (0.577)
SFT→RL34.0M9.3M11.6M54.9M (1.085)9635.1M (0.694)
STHT35.5M6.3M8.8M50.6M (1.000)8028.3M (0.560)
  • TqT_q counts only Stage-2 Harness-RL rollouts. All Stage-1 costs (failure screening on D1\mathcal{D}_1, teacher generation, SFT rollouts and edits) are included in the token costs, including Cost at TqT_q.
  • Over the full schedule, STHT costs 8.7% more than Uniform SFT→RL (failure screening and longer failure traces) and 7.8% less than SFT→RL. At TqT_q, it is cheaper than Uniform (0.577), Self SFT→RL (0.608), and SFT→RL (0.694).
  • STHT is not the cheapest way to reach a fixed threshold: RL only reaches it at lower token cost (0.464 vs. 0.560). STHT's advantage is accuracy: higher success at the same total cost (Table 5) and higher final success (80.37% vs. 76.08% macro).
AuthorsOpenReview ↗

A brief note on the response (Submission 54869)

Dear Reviewer fsPk,

Thank you again for your time and thoughtful review. As the discussion period is drawing to a close, we wanted to check whether our response and revised manuscript have addressed your concerns—specifically:

  1. External baselines at matched cost (Table 5): direct comparisons with SkillOpt, GEPA, Meta-Harness, and AutoSaddler from the same H0H_0 under a common total-cost cap, where STHT reaches 86.63% / 84.64% vs. 83.42% / 81.43% for the strongest external baseline.
  2. Complete cost accounting (Tables 2, 12–13): absolute token counts across the task model, teacher, and modifier, plus both full-schedule cost and cost at threshold TqT_q.
  3. Clarified contribution scope: positioning the post-training regularities (RQ1–RQ3) and the edit-availability mechanism relative to the standard self-evolving loop.

Please let us know if any question remains; we would be glad to clarify further.

Best regards, The Authors

Comments on the paper

1 comment
Area_Chair_ctNdOpenReview ↗

AC Comment

Thank you to the authors for the detailed rebuttal and additional experiments. I noticed that none of the three reviewers has provided follow-up feedback so far. As the discussion period is approaching its end, I would appreciate it if fsPk, Cdsg, and Ro4A could take a moment to review the authors' responses and revised manuscript. I would appreciate it if you could share whether these results address your original concerns and update your ratings if warranted.

From OpenReview, CC BY 4.0. Authors as signed there; shown unedited.

Related papers

Loading the map…

Discussion (0)

Sign in to comment.