acceptodds
Under review as a conference paper at ICLR 2027

AutoSFT: An Agentic System for Automated Data Research in Supervised Fine-Tuning

46 views

Abstract

Language model agents increasingly automate post-training, but when given the full training stack they converge on supervised fine-tuning and spend most effort revising training data. Existing systems give the agent one kind of control, selecting rows from a fixed pool or calling a generator, whereas a human data engineer adjusts proportions, prunes, repairs, replaces, synthesizes, and orders the data. We design a data action space for these changes, used only to label what a run did, and ask whether an automated system can make the full range rather than one of them, and what structure that requires. We first run this loop with a single agent, and it reaches only one action, adjusting category proportions, while its context fills with execution output and is discarded mid-search. We therefore introduce AutoSFT, a multi-agent system where a meta agent separates execution from analysis, gathers proposals from independent angles, and calls a synthesis subagent for capabilities the pool cannot cover, with the model, the trainer, and the budget fixed within each comparison. Adding each component in turn helps: the executor halves context loss, the analysis subagents lift the best IFEval by , and synthesis adds IFEval and IFBench. The same loop transfers to code, where analysis adds MBPP+ over a single agent that does not improve. Every result comes from runs that retrain from the base model each round, so each component's effect is individually attributable. Our findings suggest that automating SFT data research depends more on covering diverse data actions than on a stronger single action.

Primary area:
foundation or frontier models, including LLMs
Keywords:
automated AI research, LLM agents, supervised fine-tuning, data-centric AI, multi-agent systems

Reviews

Among:
Shape:
1234this paper, Top 30%
Figure 1. Mean review rating of the reviewed papers at ICLR 2027. This paper averages 2.67 over 3 reviews, above 70% of the others.
Reviewer_qyxwOpenReview ↗
Rating
3(+1)
Confidence
2
Informativeness
2
Presentation
3
Soundness
2
Read review
Paper summary

This paper introduces AutoSFT, a multi-agent system for improving training data for supervised fine-tuning a large language model. The system contains a meta-agent, an executor, analysis agents, and a synthesis agent. The authors run experiments using Qwen3-1.7B-Base on IFEval and IFBench (instruction following) and MBPP+ (code generation). The results show that analysis agents improve the best IFEval score by 6.5 points within the first 30 rounds; in a separate comparison, adding a synthesis agent improves the best IFEval and IFBench scores by 3.43 and 2.93 points, respectively; on MBPP+, analysis improves the best score by 3.7 points; the executor reduces the main agent’s context compactions from 10 to 5 during the process. The paper mainly argues that automated SFT data research benefits from a structure that supports a broader range of data engineering actions.

Critical strengths

The authors carry out a substantial amount of experimental work, including many rounds of model training and evaluation across different configurations. The component comparisons and analysis of the agents’ data modifications provide useful insights into how the system behaves in practice.

Critical weaknesses
  1. The experiments do not clearly separate the contribution of the proposed role structure from those of additional analysis instructions and inference computation. It remains unclear whether a general agent prompted to delegate subtasks and perform the same analyses could achieve similar results.

  2. As acknowledged in the limitations, each configuration has only one complete search run, and the experiments use only one base model across a limited range of tasks. The stability and generalization of the reported gains are uncertain.

Questions for authors

Could the authors provide more detail on whether the single-agent baseline and AutoSFT have comparable overall computational or monetary costs, including subagent analysis and data generation?

4 comments
AuthorsOpenReview ↗

Response to Reviewer qyxw

W1: "The experiments do not clearly separate the contribution of the proposed role structure from those of additional analysis instructions and inference computation. It remains unclear whether a general agent prompted to delegate subtasks and perform the same analyses could achieve similar results."

Thank you for this comment. [Global Response 1] reports the comparison. One agent receives the four analysis angles in a single prompt, with the same tools, execution isolation, base checkpoint, pool and per-round token cap, and it is free to delegate subtasks of its own. Over the 18 rounds AutoSFT ran it reaches 63.57 and 62.17 against 69.52, and given 25 rounds to hold total time comparable its best is 63.78.

W2: "As acknowledged in the limitations, each configuration has only one complete search run, and the experiments use only one base model across a limited range of tasks. The stability and generalization of the reported gains are uncertain."

Thank you for raising both points.

Stability. [Global Response 3] separates evaluation, training and search noise and reports each one. There are now two complete independent searches per configuration, and the two observed ranges are disjoint.

Generalisation. Also in [Global Response 3], on a second backbone Qwen3-4B-Base over 20 rounds, the analysis configuration reaches a best IFEval of 70.06 against 63.71. The effect is larger at this scale than at 1.7B. Beyond the code generation already in the submission, SciCode and AIME26 are the domains we plan to add next, since a single complete search consumes on the order of 320 to 680M agent tokens and 25 to 39 node-hours, where a node-hour is one 8×H800 node running for one hour.

Q1: "Could the authors provide more detail on whether the single-agent baseline and AutoSFT have comparable overall computational or monetary costs, including subagent analysis and data generation?"

Thank you for this question. [Global Response 4] reports the accounting, and it includes subagent analysis and data generation. The headline is cost to reach a score rather than cost per round. On MBPP+, exceeding the best score the single agent reaches anywhere in 25 rounds takes 37% of its GPU node-hours, 16% of its agent tokens and 22% of its cost, and the single agent spent six times the agent tokens of the configuration that beat it.

Rebuttal Summary to qyxw

Thank you for your reviews.

  • Separating role structure from analysis instructions and inference compute W1W1. We ran the condition that holds the analyses, tools, isolation and budget fixed, leaving the agent free to delegate subtasks of its own. It reaches 63.57 and 62.17 over two independent runs against 69.52, and 63.78 when given 25 rounds against 18 to match total time.

  • Stability and generalisation W2W2. We treat these as two claims. For stability, two complete independent searches per configuration with disjoint ranges, plus newly measured evaluation and training noise. For generalisation, a second backbone, Qwen3-4B-Base, where the analysis configuration reaches 70.06 against 63.71. SciCode and AIME26 are the domains we plan to add next.

  • Cost including subagents and generation Q1Q1. Reported per configuration and, more usefully, as cost to reach a score: 37% of the node-hours, 16% of the agent tokens and 22% of the cost to exceed what the single agent reaches in 25 rounds.

We would be glad to know whether this addresses what you had in mind.

We just wanted to kindly check in as the discussion period is approaching its end. If you have any remaining concerns or questions regarding our response, we would be happy to address them promptly. Thank you again for your time and consideration.

Thank you for the detailed response. The rebuttal largely addresses my concerns, and I will raise my overall score from 2 to 3

Thank you for your positive feedback and for raising your score. We are glad that our response addressed your concerns, and we sincerely appreciate your time and careful consideration.

Reviewer_9hc3OpenReview ↗
Rating
3
Confidence
3
Informativeness
2
Presentation
3
Soundness
3(+1)
Read review
Paper summary

AutoSFT combines a meta agent, an executor, analysis agents, and a synthesis agent to automate SFT data optimization. Its contributions are a six-category data-action taxonomy, an analysis of single-agent limitations, and a modular research workflow. Experiments on Qwen3-1.7B-Base use fixed training settings and retraining from the base checkpoint. Analysis improves IFEval by 4.75 points under equal wall-clock time and MBPP+ by 3.70 points; a separate synthesis ablation adds 3.43 IFEval and 2.93 IFBench points.

Critical strengths
  1. Detailed agent traces connect context pressure and repetitive data reweighting to concrete architectural interventions, providing useful insights for building automated research systems.

  2. Improvements across instruction following and coding, supported by controlled training settings and concrete intervention examples, demonstrate the workflow’s practical potential.

Critical weaknesses
  1. Benchmark scores and per-example failures guide search and synthesis, but an untouched final evaluation is not clearly reported. N-gram filtering does not rule out adaptive benchmark overfitting. The held-out results mentioned in the Ethics Statement need clarification.

  2. Each configuration has only one independent search; repeated checkpoint evaluations do not establish search robustness. Without a single-agent baseline receiving equivalent analytical guidance and a matched inference budget, the specific benefit of multi-agent organization remains uncertain.

Questions for authors
  1. Which evaluation data were completely withheld during search, and where are the held-out results mentioned in the Ethics Statement?
3 comments
AuthorsOpenReview ↗

Response to Reviewer 9hc3

W1: "Benchmark scores and per-example failures guide search and synthesis, but an untouched final evaluation is not clearly reported. N-gram filtering does not rule out adaptive benchmark overfitting. The held-out results mentioned in the Ethics Statement need clarification."

Q1: "Which evaluation data were completely withheld during search, and where are the held-out results mentioned in the Ethics Statement?"

Thank you for pressing on this. We answer W1 and Q1 together. The Ethics Statement describes a protocol the submitted paper does not report, and we have now run that evaluation. [Global Response 2] gives the full per-experiment accounting. What was completely withheld during search is Multi-IF, which enters no channel in any experiment and which we evaluate on every intermediate checkpoint of both configurations. It requires constraints to hold across three conversational turns in eight languages. The analysis configuration leads by 3.97 points on it. We agree that n-gram filtering does not rule out adaptive benchmark overfitting.

W2: "Each configuration has only one independent search; repeated checkpoint evaluations do not establish search robustness. Without a single-agent baseline receiving equivalent analytical guidance and a matched inference budget, the specific benefit of multi-agent organization remains uncertain."

Thank you for these two requirements, which we address separately.

Search robustness. There are now two complete independent searches per configuration in the §5.2 setting. The two observed ranges are disjoint, 62.35 and 64.71 for the baseline against 67.10 and 69.52 for the analysis configuration. [Global Response 3] reports all three sources of randomness separately.

A single-agent baseline with equivalent analytical guidance and a matched inference budget. We have added it. The four analysis angles are written into one agent prompt, with the same tools and execution isolation, and the agent is free to delegate subtasks of its own. Over the 18 rounds AutoSFT ran it reaches 63.57 and 62.17 against 69.52, and given 25 rounds to hold total time comparable its best is 63.78. [Global Response 1] reports both ladders and the budget accounting.

Rebuttal Summary to 9hc3

Thank you for a review whose two concerns were precisely the ones that needed further experiments.

On the untouched final evaluation: the Ethics Statement promised results the paper did not report, which was our omission, and we have now run that evaluation. We report Multi-IF on every intermediate checkpoint of both configurations. Multi-IF requires constraints to hold across three conversational turns in eight languages.

On search robustness and the matched single-agent baseline: there are now two complete independent searches per configuration, whose observed ranges are disjoint, and the inlined-prompt baseline, which does not reach AutoSFT either at matched rounds or at 25 rounds against 18.

If anything remains unresolved, or if our reading of either comment is off, we would be glad to hear it while the discussion is open.

Thank you for your detailed reply. I have no further questions at this point, I have raised my soundness to 3 and my overall assessment remains unchanged.

Thank you for your careful review and for raising the soundness score. We appreciate your time and consideration.

Reviewer_ChZ3OpenReview ↗
Rating
2
Confidence
2
Informativeness
2
Presentation
2
Soundness
2
Read review
Paper summary

This paper studies how LLM agents can automate SFT data optimization beyond a set of data-selection strategies. It proposes AutoSFT, a multi-agent system that separates experiment execution from analysis, combines complementary analysis perspectives, and synthesizes data for underrepresented capabilities. Experiments on instruction following and code generation evaluate candidate data recipes by retraining from the same base model. The results show that execution isolation reduces context compaction, analysis agents broaden the data operations explored and improve performance, and targeted synthesis provides further gains.

Critical strengths
  • Clear motivation supported by concrete observations. The pilot study documents repetitive data-reweighting decisions and frequent context compaction. It provides a concrete basis for the proposed system design.
  • Useful empirical analyses across two domains. Component-level comparisons and capability-level case studies provide insight into the system’s behavior, including unsuccessful interventions. The coding experiments also provide initial evidence that the workflow extends beyond instruction following
Critical weaknesses
  • The paper lacks a single agent baseline performs the same four analyses with equivalent prompts, tools, execution isolation, and inference budgets. The gains may therefore reflect structured guidance or additional computation. The broader coverage of data actions is also observed alongside better performance, but this does not establish whether action diversity is necessary or whether a few effective interventions account for most of the gains.
  • Benchmark scores and failure cases repeatedly guide data updates and synthesis. So it is difficult to distinguish generalizable gains from benchmark specific adaptation. The held-out evaluation mentioned in the Ethics Statement is not clearly documented, and n-gram filtering does not address adaptation to evaluation feedback.
  • Comparisons focus on internal variants. It lacks representative data-optimization methods or a strong one-shot baseline. The synthesis gains may come from better verified supervision rather than repeated feedback-driven revision. The performance & cost advantage of this iterative search over simpler alternatives is also unclear.
Questions for authors
  • Which evaluation sets were excluded from feedback, synthesis-target selection, and checkpoint selection? Please identify the held-out results mentioned in the Ethics Statement.
  • Have the authors checked the statistical significance of the reported improvements? Given the randomness in agent search, training, and evaluation, it is unclear whether the gains are robust across independent runs.
3 comments
AuthorsOpenReview ↗

Response to Reviewer ChZ3 [Part 1]

W1: "The paper lacks a single agent baseline performs the same four analyses with equivalent prompts, tools, execution isolation, and inference budgets. The gains may therefore reflect structured guidance or additional computation. The broader coverage of data actions is also observed alongside better performance, but this does not establish whether action diversity is necessary or whether a few effective interventions account for most of the gains."

Thank you for this comment, which names the one condition our ablation table was missing. The single agent baseline described here is reported in [Global Response 1]. We answer the second half of the comment, on whether action diversity is necessary.

Table 4 in Appendix C.1 reports, for each data action, the fraction of the rounds using it that set a score above every earlier round in the same run. Two things in that table bear on the question. Adjusting constraint-type proportions appears in all 30 rounds of the analysis run, so its 33.3% is also that run's overall rate of setting a new best. And the base run's actions all fall inside the mixture family, so the pruning and repair rows have no base counterpart at all.

Data actionWith analysisBase
Mixture: constraint type33.3% 10/3010/3017.2% 5/295/29
Mixture: number of constraints34.8% 8/238/2314.3% 1/71/7
Mixture: rare constraint types16.7% 1/61/633.3% 2/62/6
Mixture: language25.0% 4/164/160.0% 0/20/2
Mixture: pool-wide selection50.0% 2/42/49.1% 1/111/11
Mixture: restore earlier data22.2% 2/92/925.0% 2/82/8
Mixture: full dataset reselection50.0% 1/21/220.0% 2/102/10
Pruning: validation results41.7% 10/2410/24never used
Pruning: official checks35.0% 7/207/20never used
Pruning: text length42.9% 6/146/14never used
Repair: editing instructions and answers42.9% 6/146/14never used
Resupervision: replacing answers only0.0% 0/30/3never used

The action outside the mixture family is the action the base run never takes. The three pruning actions and the repair action set a new best in 35.0% to 42.9% of the rounds that use them, against the 33.3% overall rate of the analysis run, while the single resupervision action set none in its three uses. On the seven mixture actions the two runs share, the analysis run has the higher rate on five.

We have a mixture-only search in the paper. By post-hoc labelling it functions as an action-space ablation: the single agent of Section 3.1 spends all 33 rounds in the mixture family, and the §5.2 baseline likewise never leaves it. Those searches plateau at 62.35 over 49 rounds, while the runs reaching four families reach 67.10 in 30 rounds, as shown in Figure 8.

W2: "Benchmark scores and failure cases repeatedly guide data updates and synthesis. So it is difficult to distinguish generalizable gains from benchmark specific adaptation. The held-out evaluation mentioned in the Ethics Statement is not clearly documented, and n-gram filtering does not address adaptation to evaluation feedback."

Q1: "Which evaluation sets were excluded from feedback, synthesis-target selection, and checkpoint selection? Please identify the held-out results mentioned in the Ethics Statement."

Thank you for pressing on this. We answer W2 and Q1 together, since they concern the same gap. The Ethics Statement describes a protocol the submitted paper does not report, and we have now run that evaluation. [Global Response 2] gives the per-experiment table of which benchmark entered which of the three channels. In short, Multi-IF enters none of them in any experiment. On Multi-IF the analysis configuration leads the baseline by 3.97 points on the checkpoint selected by IFEval. We agree that n-gram filtering does not rule out adaptive benchmark overfitting.

W3: "Comparisons focus on internal variants. It lacks representative data-optimization methods or a strong one-shot baseline. The synthesis gains may come from better verified supervision rather than repeated feedback-driven revision. The performance & cost advantage of this iterative search over simpler alternatives is also unclear."

AuthorsOpenReview ↗

Response to Reviewer ChZ3 [Part 2]

Thank you for the comment, which we answer these three in turn.

aa Representative data-optimization methods. Five external methods are now reported in [Global Response 4], all under the same base checkpoint, pool and token budget: IFD, Superfiltering and RDS+ as one-pass selection, AutoSelection and DataMaster as closed-loop search. The strongest of them reaches 64.35 IFEval, against 69.52 for the analysis configuration.

bb A strong one-shot baseline. A strong agent codex+GPT−55˙mediumcodex + GPT-5\.5 medium receives the pool statistics and the benchmark description and writes a recipe in a single call, trained once. It reaches 60.11 IFEval, above all three one-pass selection methods and above a uniform mixture. We read one-shot as a single informed non-iterative attempt, and would be glad to run a different condition if that reading is wrong.

cc Whether the synthesis gains come from verified supervision rather than feedback-driven revision. This is a fair reading, and our evidence on it is indirect. Figure 15 in Appendix C shows two capabilities under the same verifier. Repeat prompt sits near 40% for eight rounds and rises to 75.6% in the round a targeted batch arrives, while odd-even syllables receives verified data in three separate rounds and scores zero each time. Since verification is identical in both cases, verified supply alone does not appear to account for the difference. Separately, in [Global Response 1], giving the meta agent the synthesis prompt directly holds the verifiers and the admission gate fixed and changes only the organisation, and AutoSFT still leads by 3.36 IFEval and 2.49 IFBench at matched rounds.

On the cost advantage over simpler alternatives. [Global Response 4] reports cost per configuration including subagent analysis and data generation. On MBPP+, exceeding the best score the base single agent reaches in 25 rounds takes 37% of its GPU node-hours, 16% of its agent tokens and 22% of its cost.

Q2: "Have the authors checked the statistical significance of the reported improvements? Given the randomness in agent search, training, and evaluation, it is unclear whether the gains are robust across independent runs."

Thank you for raising this. [Global Response 3] reports this, organised around the three sources of randomness named in the comment. Evaluation noise puts a single checkpoint within ±0.7 IFEval points. Training noise moves a fixed recipe by 1.3 to 2.0 points. There are now two complete independent searches per configuration, and the two observed ranges do not overlap. What bounds the number of repeats is cost, since a single complete search consumes on the order of 320 to 680M agent tokens and 25 to 39 node-hours, where a node-hour is one 8×H800 node running for one hour.

Rebuttal Summary to ChZ3

Thank you for a review that identified the two things our submission most needed. We summarise what has changed.

  • The missing baseline W1W1. We ran the experiment that one agent given the same four analyses, the same tools, the same execution isolation and a matched budget, and free to delegate subtasks, twice independently. At matched rounds it reaches 63.57 and 62.17 against 69.52. A round of that condition takes less time, so we also gave it 25 rounds to hold total time comparable, and its best is 63.78, still 5.74 points below. The same contrast on the synthesis component repeats the pattern.

  • Action diversity (W1, second half). You were right that we had shown co-occurrence, not necessity. We now report the per-action new-best rates from Table 4 in full, and note that the pruning and repair actions behind them are ones the base single agent never takes in any round.

  • Held-out evaluation W2,Q1W2, Q1. The Ethics Statement promised results the paper did not contain. That was our omission and we have run the evaluation. We now report Multi-IF on every checkpoint of both configurations, together with a per-experiment table of which benchmark entered which of the three channels.

  • External baselines and cost W3W3. Five external methods added, three one-pass selection methods and two closed-loop search methods, alongside no-search and one-shot references. On cost, exceeding the base single agent's best score takes 37% of its GPU node-hours, 16% of its agent tokens and 22% of its cost.

  • Significance Q2Q2. We decompose the randomness as you framed it and report what each layer licenses.

If any part of our reading of your comments is wrong, the one-shot condition in particular, we would be glad to be corrected and to run the right condition.

We just wanted to kindly check in as the discussion period is approaching its end. If you have any remaining concerns or questions regarding our response, we would be happy to address them promptly. Thank you again for your time and consideration.

Comments on the paper

7 comments
AuthorsOpenReview ↗

[Global Response 1] Matched-Budget, Prompt-Equivalent Single-Agent Baseline

We thank all three reviewers for converging on this point. It names the one cell our ablation table was missing, and we have now run it on two separate components.

  • Reviewer ChZ3 (W1): "The paper lacks a single agent baseline performs the same four analyses with equivalent prompts, tools, execution isolation, and inference budgets. The gains may therefore reflect structured guidance or additional computation."
  • Reviewer 9hc3 (W2): "Without a single-agent baseline receiving equivalent analytical guidance and a matched inference budget, the specific benefit of multi-agent organization remains uncertain."
  • Reviewer qyxw (W1): "The experiments do not clearly separate the contribution of the proposed role structure from those of additional analysis instructions and inference computation. It remains unclear whether a general agent prompted to delegate subtasks and perform the same analyses could achieve similar results."

Setup. One agent receives the four analysis angles written into a single prompt, with the same tools, execution isolation, base checkpoint, pool and per-round token cap as AutoSFT. The agent is also free to delegate subtasks of its own. What it lacks is only our role decomposition, in which the four analyses run in four separate contexts and return four independent proposals that the meta agent weighs against each other. We ran this condition twice. We ran the analogous condition for synthesis, where the generation prompt goes to the meta agent directly, with the same verifiers and the same admission gate.

Analysis component (IFEval, 3M-token budget, avg@10). A round of the inlined condition takes less wall-clock time than a round of AutoSFT, so we gave it 25 rounds against AutoSFT's 18 in order to hold the total time budget roughly equal. Per-round scores for all four runs follow the summary.

ConditionRoundsBest over the full run
Base4964.71 R46R46
Base + inlined analysis prompt, run 12563.78 R19R19
Base + inlined analysis prompt, run 22562.17 R13R13
Base + Analysis (AutoSFT)1869.52 R16R16

Under a matched time budget, two runs of the inlined prompt reach 63.57 and 62.17 against 69.52 over the 18 rounds, peak at 63.78 and 62.17, against 69.52 for AutoSFT and 64.71 for the baseline over the full run, leaving gaps of 5.74 and 7.35 points to AutoSFT. AutoSFT scores higher than both inlined runs in 17 of the 18 rounds in which AutoSFT and the two inlined runs all have checkpoints.

Per-round IFEval, all four runs. Values are mean ± SD over 10 repeated evaluations. A dash marks a round the run did not reach.

RoundBaseInlined run 1Inlined run 2Base + Analysis
R153.45 ± 1.4253.53 ± 1.0245.20 ± 1.4861.21 ± 0.48
R257.33 ± 1.1759.30 ± 0.5360.48 ± 1.2261.49 ± 0.30
R357.33 ± 1.5855.74 ± 0.5556.77 ± 1.6260.73 ± 1.25
R459.76 ± 1.2659.15 ± 0.6152.93 ± 0.4155.48 ± 1.42
R561.67 ± 0.9753.36 ± 1.2345.04 ± 1.0859.60 ± 2.28
R660.36 ± 0.8757.58 ± 0.5447.21 ± 0.4164.32 ± 0.62
R763.47 ± 0.5358.27 ± 1.1058.83 ± 0.9567.01 ± 0.88
R860.44 ± 0.2752.99 ± 0.6849.60 ± 1.9467.40 ± 1.40
R963.01 ± 0.1158.50 ± 1.0558.81 ± 0.3568.79 ± 1.06
R1063.23 ± 1.4158.74 ± 1.5658.46 ± 0.4768.72 ± 1.12
R1159.42 ± 0.9145.62 ± 0.4854.96 ± 0.7666.89 ± 0.49
R1264.00 ± 0.4757.04 ± 0.4661.66 ± 0.2869.44 ± 0.93
R1363.66 ± 0.7258.48 ± 1.0362.17 ± 0.9968.74 ± 1.45
R1462.49 ± 0.6058.26 ± 1.0360.82 ± 0.5365.91 ± 0.43
R1563.81 ± 1.0559.54 ± 1.2957.80 ± 1.2268.38 ± 1.61
R1663.31 ± 0.7858.60 ± 0.9358.41 ± 1.0069.52 ± 1.87
R1761.04 ± 0.8562.51 ± 1.0859.08 ± 0.7467.66 ± 1.08
R1863.90 ± 0.3163.57 ± 0.2658.22 ± 0.7467.93 ± 0.74
R1962.76 ± 0.4263.78 ± 1.0659.02 ± 1.72—
R2062.90 ± 0.2061.31 ± 0.4561.20 ± 1.12—
R2163.50 ± 0.4062.75 ± 0.9560.49 ± 0.79—
R2263.25 ± 1.4560.03 ± 0.4957.14 ± 0.71—
R2363.25 ± 0.6263.74 ± 0.7959.70 ± 1.06—
R2462.60 ± 0.7463.12 ± 0.5261.12 ± 0.20—
R2562.94 ± 1.0963.39 ± 1.1660.90 ± 1.33—
R2662.39 ± 2.59———
R2763.82 ± 0.63———
R2862.80 ± 1.52———
R2963.35 ± 0.97———
R3064.54 ± 0.86———
R3163.56 ± 0.67———
R3262.36 ± 0.62———
R3363.52 ± 1.21———
R3462.95 ± 0.87———
R3561.76 ± 0.76———
R3662.32 ± 1.43———
R3762.44 ± 1.11———
R3850.37 ± 1.32———
R3964.58 ± 2.30———
R4064.11 ± 0.45———
R4164.57 ± 0.65———
R4263.46 ± 1.16———
R4362.85 ± 0.48———
R4463.87 ± 0.25———
R4562.06 ± 1.68———
R4664.71 ± 0.96———
R4762.57 ± 1.50———
R4863.31 ± 1.00———
R4962.84 ± 1.12———
AuthorsOpenReview ↗

Follow-up to Global Response 1

The inlined runs are unstable rather than uniformly lower. They fall below 60 in 16 and 17 of their 25 rounds, against 5 of 25 for the base baseline and 2 of 18 for AutoSFT, and their worst rounds reach 45.62 and 45.04, against a worst of 53.45 for the baseline and 55.48 for AutoSFT.

Synthesis component IFEval/IFBench,100M−tokenbudgetIFEval / IFBench, 100M-token budget. The same pattern appears on a different component and a different budget. Per-round scores follow the summary.

ConditionRoundsBest IFEval, R≤18Best IFBench, R≤18Best over the full run
Base, selection only3174.2646.5978.15 R22R22 / 49.02 R26R26
Base + inlined synthesis prompt4578.2249.4678.93 R43R43 / 50.67 R26R26
Base + Synthesis (AutoSFT)1881.5851.9581.58 R16R16 / 51.95 R18R18

Inlining the synthesis prompt leaves the run 3.36 IFEval and 2.49 IFBench below AutoSFT at matched rounds, and given 2.5 times the rounds, 45 against 18, its best scores of 78.93 and 50.67 are still below what the subagent version reaches by round 16 to 18.

Per-round IFEval and IFBench, all three runs. Values are mean ± SD over 10 repeated evaluations.

RoundBase IFEvalInlined IFEvalAutoSFT IFEvalBase IFBenchInlined IFBenchAutoSFT IFBench
R173.34 ± 0.9173.82 ± 0.9874.28 ± 1.0846.59 ± 1.6947.34 ± 1.9047.50 ± 1.81
R271.75 ± 0.7775.01 ± 1.7474.83 ± 0.9445.25 ± 0.8247.67 ± 2.3249.53 ± 1.96
R372.92 ± 1.2877.24 ± 1.0975.46 ± 0.9943.92 ± 2.0347.81 ± 1.4649.81 ± 1.60
R469.43 ± 1.0675.83 ± 0.4976.92 ± 1.0545.14 ± 1.0346.47 ± 0.9149.58 ± 1.17
R570.41 ± 1.4273.91 ± 0.7376.90 ± 0.6245.45 ± 1.5347.60 ± 1.4349.13 ± 1.38
R672.16 ± 1.2276.09 ± 0.3877.12 ± 0.6346.08 ± 1.1848.98 ± 2.2848.14 ± 1.42
R771.03 ± 0.5875.11 ± 0.2876.96 ± 1.1444.19 ± 1.4448.11 ± 1.9849.05 ± 1.31
R870.68 ± 1.1176.82 ± 0.9676.84 ± 0.6544.41 ± 1.4847.41 ± 0.9749.93 ± 1.52
R972.27 ± 1.1376.25 ± 0.8379.59 ± 1.1344.96 ± 1.2747.54 ± 1.4850.19 ± 1.45
R1071.04 ± 1.0875.62 ± 1.3379.94 ± 0.8845.70 ± 1.7646.90 ± 0.5349.51 ± 1.28
R1173.92 ± 0.5373.07 ± 0.7779.46 ± 0.8646.24 ± 1.2946.21 ± 1.3350.68 ± 1.51
R1274.26 ± 1.3875.80 ± 0.9781.14 ± 0.9545.43 ± 1.3848.31 ± 1.0250.12 ± 1.69
R1373.20 ± 0.7376.18 ± 0.5879.36 ± 0.9344.69 ± 1.6649.46 ± 1.4049.23 ± 1.43
R1473.21 ± 0.9275.72 ± 0.5080.23 ± 0.8444.40 ± 1.0146.79 ± 0.4949.92 ± 1.14
R1573.33 ± 0.9477.25 ± 0.8081.46 ± 0.6445.24 ± 1.5647.31 ± 0.8851.87 ± 2.22
R1671.69 ± 1.2574.51 ± 0.8681.58 ± 0.7144.06 ± 1.9746.62 ± 1.2250.68 ± 1.61
R1771.13 ± 0.8677.60 ± 1.7279.78 ± 1.2944.07 ± 1.4049.22 ± 1.0750.74 ± 0.82
R1873.80 ± 0.8678.22 ± 0.8081.22 ± 0.7144.30 ± 1.5248.69 ± 1.1851.95 ± 1.32
R1973.84 ± 1.1178.25 ± 0.82—43.20 ± 1.6848.51 ± 1.39—
R2074.10 ± 1.2178.37 ± 0.74—43.72 ± 1.2848.30 ± 1.77—
R2171.77 ± 0.9177.72 ± 0.50—44.36 ± 2.0048.14 ± 0.32—
R2278.15 ± 1.0377.33 ± 1.32—48.00 ± 1.5547.79 ± 1.05—
R2377.78 ± 0.6277.57 ± 0.39—48.56 ± 1.3848.47 ± 0.93—
R2475.41 ± 1.2976.55 ± 0.37—46.83 ± 1.3048.53 ± 1.88—
R2575.35 ± 1.1076.66 ± 0.42—45.21 ± 1.4546.38 ± 1.28—
R2677.83 ± 1.2576.68 ± 0.39—49.02 ± 1.2050.67 ± 1.95—
R2777.74 ± 0.5577.59 ± 0.30—47.41 ± 1.4149.35 ± 1.64—
R2877.94 ± 0.8277.34 ± 1.02—48.51 ± 2.0348.26 ± 1.53—
R2976.21 ± 0.8177.38 ± 0.60—45.64 ± 0.8348.25 ± 1.01—
R3077.79 ± 1.1176.88 ± 0.85—48.32 ± 1.5649.69 ± 2.03—
R3176.91 ± 0.5676.26 ± 0.37—47.83 ± 1.5347.71 ± 0.42—
R32—77.01 ± 0.87——49.36 ± 1.57—
R33—78.76 ± 1.33——48.50 ± 1.88—
R34—77.71 ± 0.59——48.51 ± 1.02—
R35—76.89 ± 1.30——47.64 ± 0.89—
R36—76.30 ± 0.67——48.05 ± 0.53—
R37—76.70 ± 1.26——49.63 ± 1.76—
R38—77.79 ± 0.90——50.15 ± 2.37—
R39—76.98 ± 1.12——48.73 ± 0.58—
R40—78.01 ± 0.47——49.37 ± 0.78—
R41—75.92 ± 0.25——50.11 ± 2.07—
R42—77.02 ± 1.12——45.78 ± 1.59—
R43—78.93 ± 0.96——48.84 ± 0.84—
R44—77.42 ± 1.07——50.08 ± 1.02—
R45—76.60 ± 0.44——49.24 ± 2.94—
AuthorsOpenReview ↗

[Global Response 2] Held-Out Evaluation

We thank the reviewers for pressing on this, which identified a gap in what the submission reports.

  • Reviewer ChZ3 W2,Q1W2, Q1: "Benchmark scores and failure cases repeatedly guide data updates and synthesis. So it is difficult to distinguish generalizable gains from benchmark specific adaptation. The held-out evaluation mentioned in the Ethics Statement is not clearly documented, and n-gram filtering does not address adaptation to evaluation feedback." / "Which evaluation sets were excluded from feedback, synthesis-target selection, and checkpoint selection? Please identify the held-out results mentioned in the Ethics Statement."
  • Reviewer 9hc3 W1,Q1W1, Q1: "Benchmark scores and per-example failures guide search and synthesis, but an untouched final evaluation is not clearly reported. N-gram filtering does not rule out adaptive benchmark overfitting. The held-out results mentioned in the Ethics Statement need clarification." / "Which evaluation data were completely withheld during search, and where are the held-out results mentioned in the Ethics Statement?"

The sentence "we report off-target and held-out benchmarks" describes the protocol we intended, and the submitted paper does not report those results. We have now run that evaluation and report it below. We also agree that n-gram filtering addresses contamination of generated text and does not address adaptation to evaluation feedback.

A benchmark can influence a run in three ways: its scores can be returned to the agent as feedback, it can determine which capabilities the synthesis agent writes data for, and it can be used to choose which checkpoint we report. Yes means the benchmark was used that way during the run and No means it was not.

ExperimentBenchmarkFeedbackSynth. targetCkpt selectionRole
§5.2 AnalysisIFEvalYesn/aYesTarget
§5.2 AnalysisMulti-IFNon/aNoUntouched newnew
§5.3 SynthesisIFEvalYesYesYesTarget
§5.3 SynthesisIFBenchYesYesYesCo-target
§5.3 SynthesisMulti-IFNoNoNoUntouched newnew
§5.4 CodeMBPP+YesYesYesTarget

IFBench is a co-target in the synthesis comparison, where it appears in the stopping objective and is evaluated every round. Multi-IF enters none of the three channels in any experiment, and it is the set we report below. Multi-IF requires a model to keep satisfying constraints across three conversational turns. By design, we did not introduce multi-turn structure or any non-English language into the candidate pool, the feedback read by the agent, the capabilities targeted by the synthesis agent, or any checkpoint decision. We report turn-3 accuracy averaged over the eight languages.

Multi-IFBaseBase + AnalysisMargin
On the checkpoint selected by IFEval32.1136.08+3.97
Best over all rounds35.0838.42+3.34

Per-round IFEval and Multi-IF, both runs. A dash marks a round the run did not reach.

RoundBase IFEval+Analysis IFEvalBase Multi-IF+Analysis Multi-IF
R153.4561.2127.7334.50
R257.3361.4929.6532.93
R357.3360.7328.8432.28
R459.7655.4834.2230.28
R561.6759.6031.2032.06
R660.3664.3231.0036.50
R763.4767.0133.0635.22
R860.4467.4026.2436.79
R963.0168.7931.5436.27
R1063.2368.7235.0337.53
R1159.4266.8926.8936.14
R1264.0069.4432.9836.06
R1363.6668.7432.5738.42
R1462.4965.9135.0833.42
R1563.8168.3832.6837.61
R1663.3169.5233.8036.08
R1761.0467.6633.8036.92
R1863.9067.9334.5236.93
R1962.76—34.98—
R2062.90—30.63—
R2163.50—33.41—
R2263.25—34.16—
R2363.25—33.28—
R2462.60—33.16—
R2562.94—34.53—
R2662.39—32.86—
R2763.82—35.00—
R2862.80—34.59—
R2963.35—34.23—
R3064.54—34.05—
R3163.56—33.23—
R3262.36—34.99—
R3363.52—34.24—
R3462.95—35.05—
R3561.76—32.61—
R3662.32—32.69—
R3762.44—32.00—
R3850.37—27.34—
R3964.58—33.96—
R4064.11—34.62—
R4164.57—33.51—
R4263.46—34.62—
R4362.85—34.02—
R4463.87—34.11—
R4562.06—31.81—
R4664.71—32.11—
R4762.57—32.56—
R4863.31—33.67—
R4962.84—32.18—

On IFEval, the metric the search optimises, the analysis configuration leads by 4.81 points, or 7.4% of the baseline score. On Multi-IF, it leads by 3.97 points on the IFEval-selected checkpoint and by 3.34 points at each configuration's best round, or 12.4% and 9.5% in relative terms. This is the pattern expected if the loop improves instruction following in general, and not the pattern expected if it were adapting to the benchmark it optimises.

From OpenReview, CC BY 4.0. Authors as signed there; shown unedited.

Related papers

Loading the map…

Discussion (0)

Sign in to comment.