acceptodds
Under review as a conference paper at ICLR 2027

Beyond Outcome Filtering: Diagnosing and Correcting Behavioral Bias in Agentic SFT

Abstract

Supervised fine-tuning (SFT) for agentic language models is often built from demonstrations selected by task outcome. This creates a mismatch: curation evaluates whether a trajectory succeeds, while SFT learns from the entire trajectory. Successful demonstrations can therefore retain undesirable behaviors such as redundant retrieval, repetition, excessive verbosity, and failures to use available evidence. We introduce DiaCurate (Diagnose-and-Curate), a diagnostic-driven framework for agentic SFT data curation that identifies behavioral failures, applies targeted edits to demonstrations, and re-diagnoses the policy after retraining to detect collateral regressions. On a deep-research agent evaluated with HLE and BrowseComp, SFT on an outcome-filtered reference corpus improves task accuracy while increasing search inefficiency by 25.9 and 17.2 points and verbosity by 19.2 and 26.1 points on corresponding development sets. Directly shortening trajectories reduces unnecessary search but increases premature commitment and degrades accuracy. Re-diagnosis motivates an evidence-use refinement that reduces search inefficiency while preserving task performance, reaching 35.2 and 34.1 avg@3 versus 33.5 and 33.3 for the reference. These results show that successful trajectories are not necessarily good demonstrations: agentic SFT curation should account for both task outcomes and the behaviors learned from training trajectories.

Primary area:
foundation or frontier models, including LLMs
Keywords:
LLM, SFT, Bias, Training data, Agentic

Reviews

Among:
Shape:
1234this paper, Top 30%
Figure 1. Mean review rating of the reviewed papers at ICLR 2027. This paper averages 2.67 over 3 reviews, above 70% of the others.
Reviewer_QACkOpenReview ↗
Rating
2
Confidence
3
Informativeness
2
Presentation
2
Soundness
2
Read review
Paper summary

This paper identifies a behavioral blind spot in outcome-based filtering for agentic supervised fine-tuning (SFT): such filtering constrains only whether the final answer is correct, while SFT learns from the entire trajectory, so successful demonstrations can still teach undesirable behaviors such as redundant search and excessive verbosity. The authors propose DiaCurate, a diagnose-and-curate framework that profiles the trained policy with a nine-axis behavioral diagnostic, applies targeted edits to demonstrations, and retrains and re-diagnoses the resulting policy to detect collateral regressions. The framework is instantiated on a deep-research agent trained from a 5,298-example corpus and evaluated on HLE and BrowseComp. The key findings are that outcome-filtered SFT improves accuracy while substantially worsening search inefficiency and verbosity; that shortening trajectories to reduce over-search increases premature commitment and degrades accuracy; and that re-diagnosis motivates an evidence-use refinement (TE3) that reduces search inefficiency while preserving accuracy.

Critical strengths
  • Interesting, clearly quantified findings about outcome-filtered SFT.

The paper concretely demonstrates that outcome-filtered SFT improves task accuracy while substantially increasing search inefficiency and verbosity, making the behavioral blind spot tangible rather than speculative. This is a valuable and timely empirical observation.

Critical weaknesses
  • Limited technical novelty.

The underlying components, including behavioral diagnosis and feedback-driven filtering and editing, are established. The paper therefore contributes a framework and an empirical demonstration rather than a new method, which limits its contribution for a top-tier venue.

Questions for authors
  1. Generality across models. The study uses a single model (Nemotron 3.5 Lightning). Do the behavioral blind spot of outcome filtering and the collateral-regression pattern generalize to other model families and scales, and can you provide evidence across additional models?

  2. Generality across domains. The study focuses on a single agent domain, deep web research. Do the findings extend to other agent domains, such as coding or general tool-use agents, where the notions of "sufficient evidence" and "over-search" may differ considerably?

  3. Reliability of the truncation point. The termination edits rely on an LLM-judge-based sufficiency judgment to decide where each trajectory is truncated, and the paper itself notes that this is not a proof of sufficiency. How accurate is this truncation point, and can you validate it with human evaluation of the sufficiency judgments?

5 comments

Thank you for the thoughtful review.

Re: "Limited technical novelty. The underlying components, including behavioral diagnosis and feedback-driven filtering and editing, are established. The paper therefore contributes a framework and an empirical demonstration rather than a new method." (weakness 1)

We acknowledge that components such as LLM-based error-mode diagnosis and teacher-trace editing have been studied in prior work, including AgentDebug [1] and CurateEvo [2]. DiaCurate builds on these components with two contributions.

First, we introduce an iterative framework that connects behavioral diagnosis, targeted editing, and policy evaluation to improve the behavior learned from outcome-correct demonstrations. Diagnosis identifies which behaviors to change; targeted edits modify their supervision; retraining and re-diagnosis reveal both the intended improvements and collateral failures. A predefined gate requires improvement in the target behavior while keeping the increase in checkable error within tolerance, and rejected edits are revised using the new diagnosis. The TE1-TE3 progression shows why this integration matters: TE1/TE2 reduce search inefficiency but induce premature commitment and fail the gate. Re-diagnosis motivates TE3, which combines localized search edits with explicit evidence-use guidance. TE3 passes the gate on both development sets and improves held-out HLE/BrowseComp avg@3 from 33.5/33.3 to 35.2/34.1. This connects the framework to a concrete accuracy improvement through diagnosis-guided intervention redesign.

Second, we establish a scientific finding: outcome-correct agentic traces retain behavioral inefficiencies that provide headroom for improving downstream task performance. SFT on these successful traces improves accuracy while substantially worsening search inefficiency and verbosity. Our interventions show that this remaining behavioral deficiency is actionable: changing the supervision in already successful demonstrations can improve the resulting policy beyond the tested selection-only baselines. The gate also separates successful from harmful edits in our experiments: all accepted variants improve pass@3 on both benchmarks, while the rejected termination variants reduce avg@3 on both (Table 3). These findings link behavioral deficiencies within successful training traces, their targeted correction, and measurable improvements in the trained agent.

Re: "Generality across models. The study uses a single model (Nemotron 3.5 Lightning). Do the behavioral blind spot of outcome filtering and the collateral-regression pattern generalize to other model families and scales, and can you provide evidence across additional models?" (question 1)

To test whether DiaCurate generalizes across model families, we repeated the DR study with Qwen3-4B-Thinking-2507, following the same procedure as in the paper.

Outcome-filtered SFT again improves accuracy while worsening behavioral quality, and further filtering does not resolve this gap. The policy trained with our answer-compaction edit (AC1), accepted based on development diagnostics, improves both avg@3 and pass@3 on BrowseComp-400 over the outcome-filtered SFT baseline. These results support the transfer of diagnosis-guided curation to a second model family, with an accepted behavioral edit translating into better held-out benchmark performance.

MethodVersionWeb-Discovery-Dev F ↓Web-Discovery-Dev ΔTWeb-Discovery-Dev ΔEDecisionBrowseComp-400 avg@3 (%) ↑BrowseComp-400 pass@3 (%) ↑
Base model–26.0–––4.88.2
Outcome-filtered SFT(reference)30.6ref.ref.–29.342.8
Error-mode-filtered SFT–30.6−2.1+3.3–29.143.2
Answer-compaction SFTAC1 (ours)28.8−12.7+0.8accept31.246.5

Re: "Generality across domains. The study focuses on a single agent domain, deep web research. Do the findings extend to other agent domains, such as coding or general tool-use agents, where the notions of 'sufficient evidence' and 'over-search' may differ considerably?" (question 2)

We have extended the study to WebShop [3], a widely used language-agent benchmark evaluated in ReAct [4] and Language Agent Tree Search [5]. It changes both the domain and the execution requirements: the agent navigates product pages, selects options, and purchases an item satisfying the user’s constraints. Actions change the interface state and available choices, and success is evaluated on the purchased item. This tests interactive shopping beyond evidence gathering and answer generation. We apply the same curation loop with the same teacher and student, adapting the diagnostics and edits to shopping.

We first diagnose fresh policy rollouts on the development set using six axes: action violation (actions refused by the current page), redundant searches (repeated queries), product revisits (reopening an already visited product), dead searches (leaving search results without opening a product), action-budget exhaustion (reaching the action limit without purchasing), and premature purchase (buying with fewer options selected than the instruction requests). We report the mean across these axes as F. Successful demonstrations retain redundant and dead searches, and these behaviors persist in the trained policy.

We target the search-related failure modes redundant searches and dead searches, defining T as their mean. As in the paper, acceptance requires T to decrease by at least 3 points and E to increase by no more than 3 points relative to the outcome-filtered SFT reference. Here E is the percentage of rollouts in which no opened product page displays all requested options within the price limit, guarding against evidence-acquisition failures that cutting search behavior could inadvertently induce.

We apply search editing (SE1) to successful teacher and policy demonstrations. For repeated queries, we remove the intervening detour that returns to the same search state; for dead searches, we remove the search and its unproductive continuation before the next search. We retain the remaining exploration and purchase steps, keeping the edited trajectory outcome-correct and internally consistent.

After retraining, we re-diagnose fresh rollouts on WebShop-Dev and apply this acceptance rule. The accepted policy achieves higher performance on the held-out WebShop-Test benchmark than the outcome-filtered reference and selection-only baseline.

These results support the framework’s applicability beyond deep research using task-specific diagnostic axes. We will add the WebShop study as an additional main section of the paper.

MethodVersionWebShop-Dev F ↓WebShop-Dev ΔTWebShop-Dev ΔEDecisionWebShop-Test avg@3 (%) ↑WebShop-Test pass@3 (%) ↑
Base model–30.1–––31.944.2
Outcome-filtered SFT(reference)17.7ref.ref.–41.952.2
Error-mode-filtered SFT–13.3−6.1+0.5–42.253.4
Search editingSE1 (ours)9.4−11.1−1.5accept44.154.2

Re: "Reliability of the truncation point. The termination edits rely on an LLM-judge-based sufficiency judgment ... How accurate is this truncation point, and can you validate it with human evaluation of the sufficiency judgments?" (question 3)

Thank you for raising this point. To ensure reliable truncation decisions, we have already applied a conservative grounding guardrail: for each trajectory judged sufficient and proposed for truncation, we verify by exact matching that the judge’s quoted evidence appears in the retained prefix. If this check fails, we reject the edit and keep the original trajectory. We will clarify this guardrail in the paper appendix.

We additionally assess truncation-point reliability through repeated LLM judge runs on 250 randomly sampled edited TE1/TE2 trajectories. The original cut was within three tool-call turns of the median across repeated judgments in 97% of trajectories, indicating high consistency of the cut location. We also perform human audits of 50 TE1/TE2 cut points. The annotator judges the retained evidence is sufficient in 49/50 cases (98%). In the remaining case, a candidate is named but one question constraint is not fully verified. This supports the sufficiency check’s reliability in the audited sample while revealing a residual failure mode: traces cut before all constraints are verified could teach the model to commit prematurely. This possible failure mode is consistent with the premature commitment and performance degradation observed for TE1/TE2. These observed failures motivated TE3, which combines targeted removal of unproductive search turns with an added sentence teaching how to use existing evidence to commit to an answer, while retaining the remaining research and verification.

We will add these reliability and human-audit results to the paper appendix.

Reference for the whole response:

[1] Where LLM Agents Fail and How They Can Learn From Failures.

[2] CurateEvo: Data-Curation Evolving for Agentic Post-Training.

[3] WebShop: Towards Scalable Real-World Web Interaction with Grounded Language Agents.

[4] ReAct: Synergizing Reasoning and Acting in Language Models.

[5] Language Agent Tree Search Unifies Reasoning, Acting, and Planning in Language Models.

Reviewer_yDQTOpenReview ↗
Rating
3(+1)
Confidence
3
Informativeness
2
Presentation
2
Soundness
2
Read review
Paper summary

The paper identifies a mismatch in agentic SFT: demonstrations are selected by task outcome, but the policy imitates the entire trajectory, so successful demonstrations can still teach undesirable behaviors such as redundant search and verbose answers. It proposes DiaCurate, a curation loop that diagnoses the trained policy with nine rule-based behavioral metrics, edits the training data to target the diagnosed behavior, retrains, and re-diagnoses to catch collateral regressions, accepting an edit only if the target improves without raising error beyond a fixed tolerance. On a deep-research agent, editing over-searching out of the demonstrations backfires, while the accepted evidence-anchoring edit reduces over-searching and slightly improves both reporting benchmarks over the outcome-filtered reference.

Critical strengths

The paper addresses a problem of practical relevance to agentic SFT — demonstrations selected by task outcome can still teach undesirable behaviors — and reports its failed interventions in full: the truncation variants that induce premature commitment are rejected by the pre-registered acceptance rule. The nine behavioral diagnostics are deterministic code rather than LLM judgment, with equations, detectors, thresholds, and eligible-population denominators fully specified, so the measurements are reproducible.

Critical weaknesses
  1. The gating logic is not carried through. Acceptance gates only the target axis (≥3 improvement) and checkable error (≤3 rise); the other seven axes merely "guide revision". AC1 is accepted despite evidence-use failure rising 36.2→41.8 and protocol violations 14.9→18.0, TE3 despite premature commitment rising 7.4→10.5 (development sets). The whack-a-mole the paper criticizes passes its own gate.

  2. Insufficient experiments, with unattributed gains. All results come from one model, task domain, and corpus, in single runs without seed variance. The corpus regenerates all reasoning with Qwen3.5-27B, yet no ablation contrasts original with regenerated reasoning, so the 11-point SFT gain is never decomposed into protocol compliance, distillation, or capability.

  3. Contamination and training settings are unclear. No contamination check between training corpus and reporting benchmarks is reported, though most supervised tokens are teacher-authored and the benchmarks predate the teachers' cutoffs; the training configuration is absent from the main text and ambiguous for the selection baselines.

Questions for authors
  1. Appendix B.2 fixes all conditions at 6 epochs / 132 updates, while B.4 says the selection baselines use a "shared compute budget" — for the smaller corpora, which was held fixed, updates or epochs? Please add a compute-matched control.

  2. What contamination controls were applied between the training questions and the teacher-generated text versus HLE-500/BrowseComp-400/GAIA? Were n-gram or embedding-based checks run, given that the benchmarks predate the teachers' training cutoffs?

7 comments

Re: "The gating logic is not carried through. Acceptance gates only the target axis (≥3 improvement) and checkable error (≤3 rise); the other seven axes merely 'guide revision'. AC1 is accepted despite evidence-use failure rising 36.2→41.8 and protocol violations 14.9→18.0, TE3 despite premature commitment rising 7.4→10.5. The whack-a-mole the paper criticizes passes its own gate." (weakness 1)

The gating logic is carried through exactly as specified in the paper, and every decision in Table 3 follows it. Section 4.4 explicitly requires ΔT ≤ −3 and ΔE ≤ 3 on each development set and defines E = f_commit + f_cover + p_seen f_extract, where p_seen is the fraction of checkable rollouts with a detected reference-answer match. The claim that “the other seven axes merely guide revision” therefore overlooks that premature commitment, insufficient coverage, and evidence-use failure already enter the acceptance gate jointly through E. Alongside the target, these account for four of the nine diagnostic axes; the five remaining process axes guide revision, as stated in the text.

TE3 is correctly accepted: search inefficiency decreases by 4.5 and 4.2 points, while E rises by only 0.3 and 0.4 points on Research-Reasoning-Dev and Web-Discovery-Dev, respectively. AC1 likewise meets both criteria: verbosity decreases by 9.4 and 16.0 points, while E rises by 2.4 and 0.7 points, in the same set order. The cited component increases do not violate the aggregate criterion; protocol violations have no separate acceptance threshold. TE1 and TE2 are correctly rejected because their 4.1–14.6-point increases in E exceed the 3-point tolerance, despite improving the target.

We gate E jointly because deep research depends on the information retrieved into context for subsequent reasoning. E combines failures in evidence coverage and use of retrieved information. An edit can shift errors between these aspects while keeping their combined error within tolerance, so we assess overall reliability rather than any single component. We will explain this rationale more directly in the text.

Re: "All results come from one model" (weakness 2)

To test whether DiaCurate generalizes across model families, we repeated the DR study with Qwen3-4B-Thinking-2507, following the same procedure as in the paper.

Outcome-filtered SFT again improves accuracy while worsening behavioral quality, and further filtering does not resolve this gap. The policy trained with our answer-compaction edit (AC1), accepted based on development diagnostics, improves both avg@3 and pass@3 on BrowseComp-400 over the outcome-filtered SFT baseline. These results support the transfer of diagnosis-guided curation to a second model family, with an accepted behavioral edit translating into better held-out benchmark performance.

MethodVersionWeb-Discovery-Dev F ↓Web-Discovery-Dev ΔTWeb-Discovery-Dev ΔEDecisionBrowseComp-400 avg@3 (%) ↑BrowseComp-400 pass@3 (%) ↑
Base model–26.0–––4.88.2
Outcome-filtered SFT(reference)30.6ref.ref.–29.342.8
Error-mode-filtered SFT–30.6−2.1+3.3–29.143.2
Answer-compaction SFTAC1 (ours)28.8−12.7+0.8accept31.246.5

Re: "one ... task domain, and corpus" (weakness 2)

We have extended the study to WebShop [1], a widely used language-agent benchmark evaluated in ReAct [2] and Language Agent Tree Search [3]. It changes both the domain and the execution requirements: the agent navigates product pages, selects options, and purchases an item satisfying the user’s constraints. Actions change the interface state and available choices, and success is evaluated on the purchased item. This tests interactive shopping beyond evidence gathering and answer generation. We apply the same curation loop with the same teacher and student, adapting the diagnostics and edits to shopping.

We first diagnose fresh policy rollouts on the development set using six axes: action violation (actions refused by the current page), redundant searches (repeated queries), product revisits (reopening an already visited product), dead searches (leaving search results without opening a product), action-budget exhaustion (reaching the action limit without purchasing), and premature purchase (buying with fewer options selected than the instruction requests). We report the mean across these axes as F. Successful demonstrations retain redundant and dead searches, and these behaviors persist in the trained policy.

We target the search-related failure modes redundant searches and dead searches, defining T as their mean. As in the paper, acceptance requires T to decrease by at least 3 points and E to increase by no more than 3 points relative to the outcome-filtered SFT reference. Here E is the percentage of rollouts in which no opened product page displays all requested options within the price limit, guarding against evidence-acquisition failures that cutting search behavior could inadvertently induce.

We apply search editing (SE1) to successful teacher and policy demonstrations. For repeated queries, we remove the intervening detour that returns to the same search state; for dead searches, we remove the search and its unproductive continuation before the next search. We retain the remaining exploration and purchase steps, keeping the edited trajectory outcome-correct and internally consistent.

After retraining, we re-diagnose fresh rollouts on WebShop-Dev and apply this acceptance rule. The accepted policy achieves higher performance on the held-out WebShop-Test benchmark than the outcome-filtered reference and selection-only baseline.

These results support the framework’s applicability beyond deep research using task-specific diagnostic axes. We will add the WebShop study as an additional main section of the paper.

MethodVersionWebShop-Dev F ↓WebShop-Dev ΔTWebShop-Dev ΔEDecisionWebShop-Test avg@3 (%) ↑WebShop-Test pass@3 (%) ↑
Base model–30.1–––31.944.2
Outcome-filtered SFT(reference)17.7ref.ref.–41.952.2
Error-mode-filtered SFT–13.3−6.1+0.5–42.253.4
Search editingSE1 (ours)9.4−11.1−1.5accept44.154.2

Re: "single runs without seed variance" (weakness 2)

We additionally perform two independent training runs for both the reference and TE3 with seeds 2026 and 2027, supplementing the original runs with seed 42 under the same settings. The table reports avg@3 as mean ± SD across these three training seeds. TE3 consistently outperforms the reference on HLE-500 and BrowseComp-400, accounting for variation across training seeds.

ConditionHLE-500 avg@3 (%)BrowseComp-400 avg@3 (%)
Reference33.4 ± 0.232.5 ± 1.1
TE334.8 ± 0.436.5 ± 2.5

Re: "The corpus regenerates all reasoning with Qwen3.5-27B, yet no ablation contrasts original with regenerated reasoning, so the 11-point SFT gain is never decomposed into protocol compliance, distillation, or capability." (weakness 2)

We would like to clarify the role of the reference run. Our paper studies which undesirable behaviors remain after SFT on demonstrations that pass outcome-correctness checks, and whether targeted curation can correct them. Our wording “outcome-filtered” might lead you to think that we emphasize the filtering operation itself, but we simply mean that this reference run is trained entirely on outcome-correct demonstrations. We will revise the description of the run in the paper to make this clear. The approximately 11-point accuracy gain establishes that training on the reference corpus improves task performance. Decomposing that gain into protocol compliance, distillation, or capability acquisition is outside the scope of this study; we make no claim that outcome filtering or reasoning regeneration alone produces it.

The original RedSearcher corpus was released in early 2026. Given the substantial progress in frontier models’ agentic capabilities since then, we regenerated its reasoning with Qwen3.5-27B to use a more recent, state-of-the-art teacher.

All curation variants derive from this same regenerated corpus and are evaluated against the same reference checkpoint. Protocol conversion and initial reasoning regeneration are therefore shared across conditions; the experimental comparisons concern the subsequent edits or selection of demonstrations.

Re: "Appendix B.2 fixes all conditions at 6 epochs / 132 updates, while B.4 says the selection baselines use a 'shared compute budget' — for the smaller corpora, which was held fixed, updates or epochs? Please add a compute-matched control." and "the training configuration is absent from the main text and ambiguous for the selection baselines" (weakness 3, question 1)

Thank you for pointing out the inconsistency. The “6 epochs” entry in Table 7 described the reference run but was inaccurate as a shared schedule. For the reported comparisons, we used a common budget of up to 132 optimizer updates, with identical batch size and learning rate across conditions. Within this budget, we selected the checkpoint with the highest mean avg@3 across the two development sets, following Section 4.4.

The smaller selection corpora received more passes through the retained data within this update budget. Error-mode-filtered SFT retained 4,207 rows and used 17 updates per epoch, so the ceiling corresponded to approximately 7.8 epochs. LLM-scored SFT retained 3,553 rows and used 14 updates per epoch, corresponding to approximately 9.4 epochs. These epoch counts describe the full 132-update training budget, but the checkpoint with the best development performance could be reached earlier.

Equal updates did not imply equal compute because trajectory lengths differed. We estimated training FLOPs from Nemotron-tokenized trajectory lengths, with calibrated chat-template overhead, accounting for both token-linear operations and quadratic attention costs. The table reports estimated tokens and FLOPs at 132 updates.

ConditionTraining rowsTokens at 132 updates (B)Relative FLOPs at 132 updates
RedSearcher reference5,2981.551.00
LLM-scored SFT3,5531.470.94
Error-mode-filtered SFT4,2071.540.99
TE15,2981.190.74
TE25,2981.330.84
TE35,2981.540.99
AC15,1211.551.00
AC25,1211.551.00

Error-mode-filtered SFT, TE3, and both answer-compression variants were within 2% of the reference’s estimated compute, which we accepted as compute-matched. LLM-scored SFT, TE1, and TE2, however, used substantially shorter training trajectories and therefore required less compute at 132 updates.

To address this difference, we extended LLM-scored SFT, TE1, and TE2 training to cover the reference’s estimated FLOP budget. The corresponding targets were approximately 179 updates (8.1 epochs) for TE1 and 158 updates (7.2 epochs) for TE2, based on their relative per-update costs and 22 updates per epoch. For LLM-scored SFT, the target was approximately 141 updates (10.1 epochs). For each extended run, we evaluated the final checkpoint and any epoch-boundary checkpoints beyond update 132 on the two development sets. None of these checkpoints exceeded its variant’s best mean avg@3 within the first 132 updates. This compute-matched check therefore left the checkpoints evaluated in the paper unchanged.

We will correct the epoch description and clarify the training budget, compute estimates, and checkpoint-selection criteria in the main text.

Reviewer_S7wUOpenReview ↗
Rating
3(+1)
Confidence
3
Informativeness
2
Presentation
3
Soundness
2
Read review
Paper summary

DiaCurate uses 9-axis behavioral diagnosis to guide edits to successful agentic SFT demonstrations and detect collateral regressions after retraining. Direct truncation harms accuracy despite improving search proxies; TE3 adds evidence-use reasoning while retaining research actions. TE3 raises avg@3 from 33.5 to 35.2 on HLE-500 and from 33.3 to 34.1 on BrowseComp-400.

Critical strengths

TE1/TE2 reveal an informative failure mode: truncation improves process scores but hurts downstream task accuracy. Evaluating edited demonstrations through the retrained policy exposes the mismatch between process proxies and policy quality, motivating checks for collateral regressions in data curation.

Critical weaknesses
  1. Search and verbosity proxies lack human validation and may penalize necessary verification or explanation. Since TE3 teaches explicit evidence references, better proxy scores need not indicate improved search or stopping; actual tool/token costs and latency are unreported. This weakens the central behavioral interpretation.
  2. TE3 changes teacher supervision/editor, compression changes the training population and validation split, and selection baselines retain fewer examples. The 1.7/0.8-point avg@3 gains also lack uncertainty estimates and independent training repeats. These gaps prevent convincing attribution of reliable gains to targeted curation.
Questions for authors
  1. Metric validation: Do blinded human audits and actual tool/token costs or latency confirm search/stopping improvements beyond lexical reuse? Please report component scores and eligible counts.
  2. Attribution: Does TE3 outperform matched teacher-rewrite/candidate-mention controls, and does compression help against an identically split, unedited reference?
  3. Statistical reliability: Please report paired question-level confidence intervals and uncertainty in development acceptance decisions. Were independent training repeats performed?
4 comments

Thank you for the thoughtful review.

Re: Verbosity proxies "lack human validation and may penalize necessary ... explanation." (weakness 1)

We conduct a blinded audit of 80 answers exceeding twice the reference length across both development sets on the RedSearcher SFTed model checkpoints. The additional content was unrequested and non-helpful explanations in 76/80 cases (95%). For the remaining 4 cases, the questions explicitly ask for reasoning/explanation so our annotator finds the excessive explanation could still be helpful but not necessary.

Most of the audited longer answers add unrequested details that burdens reading/answer extraction. For example, a BrowseComp-dev question asks, “I want you to find the name of the performer from the following information.” The requested output is only the performer’s name, and the reference is “Brad Pitt” (two tokens). The SFT checkpoint trained on RedSearch (reference run) gives this name, but also adds explanations on fourteen clues to justify why this is the answer, making it hard to read from a human perspective.

Moreover, the table below shows that extreme verbosity also coincides with poorer evaluation outcomes. RedSearcher rollouts are grouped by final-answer/reference length ratio. The >16× group comprises 34% and 41% of the full rollout population, respectively, and has the lowest accuracy and highest answer-extraction failure rate on both sets. This corroborates with the human audits that diagnosed expansion is predominantly unrequested in the audited sample, and its extreme form accompanies measurable evaluation difficulties.

SetLength rationAccuracy (%)Judge extracted no answer (%)
HLE-dev≤2×44945.90.7
HLE-dev>16×43835.62.7
BrowseComp-dev≤2×23253.00.0
BrowseComp-dev>16×28950.53.1

Re: "TE3 ... better proxy scores need not indicate improved search or stopping; actual tool/token costs and latency are unreported." and "Do blinded human audits ... confirm search/stopping improvements beyond lexical reuse? Please report component scores and eligible counts." (weakness 1, question 1)

Thank you for highlighting the need to clarify TE3. Section 5.2 describes it as making “the connection between retrieved evidence and answer commitment explicit.” Our description did not clearly explain the search cuts, and the statement that TE3 “removes no research turns” is incorrect and will be corrected. TE3 actually combines targeted removal of unproductive search turns with an added sentence teaching how to use existing evidence to commit to an answer. Whereas TE1/TE2 truncate trajectories at a judged-sufficient point, TE3 removes individual zero-result searches and repeated or near-duplicate queries that add no new URLs, while retaining the remaining research, final answer, and verification.

To assess changes beyond evidence-citing wording, we measured actual tool use over 1,200 BrowseComp-400 rollouts (k=3), with the same source restrictions, tools, and sampling settings. The table compares the base model, the reference SFT checkpoint trained on RedSearch, and TE3; resource measurements are per-rollout means.

BrowseComp-400 metricBaseReference SFTTE3
Searches23.669.064.5
Zero-result searches2.29.85.1
Page visits6.114.614.2
Tool calls (all)30.883.878.9
Model turns45.085.680.8
Search + visit service time (s)75164149

TE3 reduces searches by 7%, total tool calls by 6%, and search and page-visit service time by 9%; zero-result searches fall by 48%. The development set shows the same pattern: searches fall from 67.2 to 62.4 and zero-result searches from 9.1 to 5.7 per rollout. These counts directly measure the model’s search actions, so the reductions show lower actual tool use beyond any change in how the model refers to retrieved evidence.

To address your request for a blinded human audit, we had an annotator manually inspect the TE3 search cuts in 150 rollouts from 50 questions across both corpora. The table reports the composition of the audited search cuts.

Removed search typeShare of audited cuts
Zero-result query with new wording92.8%
Rephrased query returning only previously seen URLs7.0%
Exact repeat of an earlier query (either kind)0.2%

The zero-result queries were mostly over-constrained exact-phrase searches, such as quoting a long phrase from the question verbatim. Cutting these queries removes inefficient searches that return no evidence.

The resource measurements show that TE3 makes fewer search calls and reduces search and page-visit service time, while the human audit shows that the cuts target queries yielding no new evidence. Together, these findings support improved search efficiency beyond changes in evidence-citing wording.

Re: TE3 teacher supervision and matched teacher-rewrite control (weakness 2, question 2)

Thank you for raising this concern. We would like to clarify that TE3 does not switch the teacher used to generate the training trajectories: it uses the same Qwen3.5-27B traces as the reference, with targeted edits.

TE1/TE2 showed that truncating trajectories to remove redundant search can also compromise commitment. TE3 instead removes individual zero-result searches and repeated or near-duplicate queries that add no new URLs, retaining the remaining research. It pairs these targeted cuts with a few sentences generated by GPT-5.6 that explicitly teach the model to use retrieved evidence to commit to an answer. GPT supplies these brief additions without regenerating the trajectories. These additions account for only 0.7% of supervised assistant tokens in the median edited trajectory and 0.5% across the training corpus.

We acknowledge that even these brief additions from a different editor could affect performance. To address this, we ran a matched teacher-rewrite control for the brief additions: we kept the original Qwen3.5-27B trajectories and targeted search edits fixed, and used Qwen3.5-27B itself in place of GPT-5.6 to generate the added sentences. This comparison tests whether TE3's improvement depends on introducing GPT as the editor. The table compares avg@3 for the reference, original TE3, and this control.

The control retains improvements over the reference on both benchmarks, with average performance across the two benchmarks essentially unchanged from the original TE3. This supports that the observed gains are not specific to GPT-generated additions.

Training corpusHLE-500 avg@3 (%)BrowseComp-400 avg@3 (%)
Reference33.533.3
TE3 (paper)35.234.1
TE3 (Qwen3.5-27B rewrite control)34.734.8

Re: "compression changes the training population and validation split, and selection baselines retain fewer examples ... does compression help against an identically split, unedited reference?" (weakness 2, question 2)

We apologize for the confusion caused by our use of “validation split.” To clarify, this split is not used to select curation variants or measure their gains. It is a held-out set of SFT demonstrations used to monitor next-token prediction loss and check whether the SFT run is behaving normally, including whether validation loss decreases and whether overfitting emerges. This is an upstream training check for detecting bugs and flagging problematic runs. Our gain claims instead come from downstream evaluations shared across all conditions: checkpoint selection, behavioral diagnosis, and acceptance decisions use the fixed Research-Reasoning-Dev and Web-Discovery-Dev sets, while benchmark performance is measured on the same HLE-500 and BrowseComp-400 questions. We will clarify these distinct roles in Appendices B.2 and C.3.

Although the AC corpora in the paper contain slightly fewer training examples than the reference, both AC variants and the reference process approximately 1.55 billion tokens over 132 optimizer updates, with estimated FLOPs also matched within 2%. The common 132-update budget therefore provides comparable training compute despite the difference in row count, as detailed in our response to Reviewer yDQT on training settings and compute matching (weakness 3, question 1).

To address the difference in training population directly, we additionally trained on the full AC1 corpus using the same 5,298 source trajectories as the reference, with only the final-answer compression edits applied. The table compares its avg@3 with the reference and the AC1 results reported in the paper.

Training corpusTraining tracesHLE-500 avg@3 (%)BrowseComp-400 avg@3 (%)
Reference5,29833.533.3
AC1 (paper)5,12132.934.6
AC1 (full corpus)5,29833.134.4

Re: "The 1.7/0.8-point avg@3 gains also lack uncertainty estimates and independent training repeats. Please report paired question-level confidence intervals and uncertainty in development acceptance decisions. Were independent training repeats performed?" (weakness 2, question 3)

Thank you for suggesting a more complete assessment of statistical reliability. We assess the reliability of the benchmark gains through independent training repeats, and uncertainty in development acceptance decisions through a paired question-level bootstrap.

Benchmark gains across training seeds. We additionally perform two independent training runs for both the reference and TE3 with seeds 2026 and 2027, supplementing the original runs with seed 42 under the same settings. The table reports avg@3 as mean ± SD across these three training seeds. TE3 consistently outperforms the reference on HLE-500 and BrowseComp-400, accounting for variation across training seeds.

ConditionHLE-500 avg@3 (%)BrowseComp-400 avg@3 (%)
Reference33.4 ± 0.232.5 ± 1.1
TE334.8 ± 0.436.5 ± 2.5

Uncertainty in development acceptance decisions. We assess uncertainty in ΔT and ΔE using a paired question-level bootstrap (4,000 resamples) on both development sets. The table reports standard errors, 95% percentile confidence intervals, and bootstrap support for the prespecified acceptance thresholds, ΔT ≤ −3 and ΔE ≤ 3 (Section 4.4).

The bootstrap results support the original point-estimate decisions on both development sets. For every accepted variant, each acceptance criterion receives at least 70% bootstrap support, with most values near 1. For TE1/TE2, bootstrap support for satisfying the error tolerance is at most 20%, supporting their rejection. These results favor the original decisions while showing that some acceptance thresholds carry greater uncertainty. We will add these estimates to Section 4.4 and Appendix Table 11.

EditDevelopment setΔT ± SEBootstrap P(ΔT ≤ −3)ΔE ± SE [95% CI]Bootstrap P(ΔE ≤ 3)Result
TE1Research-Reasoning-Dev−10.9 ± 1.7>0.99+4.4 ± 1.4 [+1.8, +7.2]0.16reject
TE1Web-Discovery-Dev−11.0 ± 1.7>0.99+14.6 ± 1.6 [+11.6, +17.6]0.00reject
TE2Research-Reasoning-Dev−6.8 ± 1.70.99+4.1 ± 1.3 [+1.6, +6.8]0.20reject
TE2Web-Discovery-Dev−7.4 ± 1.6>0.99+11.8 ± 1.6 [+8.7, +15.1]0.00reject
TE3Research-Reasoning-Dev−4.5 ± 1.60.82+0.3 ± 1.2 [−2.1, +2.6]0.99accept
TE3Web-Discovery-Dev−4.2 ± 1.50.79+0.4 ± 1.5 [−2.5, +3.3]0.96accept
AC1Research-Reasoning-Dev−9.4 ± 1.3>0.99+2.4 ± 1.2 [−0.1, +4.8]0.70accept
AC1Web-Discovery-Dev−16.0 ± 1.5>0.99+0.7 ± 1.3 [−1.9, +3.3]0.96accept
AC2Research-Reasoning-Dev−3.7 ± 1.10.73−0.7 ± 1.2 [−3.1, +1.7]>0.99accept
AC2Web-Discovery-Dev−9.9 ± 1.4>0.99+2.0 ± 1.5 [−0.8, +4.9]0.75accept

Thank you again for your helpful feedback. Please let us know if you have any additional questions or remaining concerns about our responses and new results. We would be happy to address them.

Comments on the paper

2 comments

Dear Reviewers,

The authors have submitted their rebuttal. Please take a look at their responses and actively participate in the discussion, especially regarding any remaining concerns or disagreements. Please consider updating your evaluations if appropriate.

Thank you for your time and effort!

Best regards, Area Chair

We thank the AC for facilitating the discussion and the reviewers for their helpful feedback. Below, we summarize the key points and new evidence from the discussion period and how they strengthen the paper.

Novelty and contribution. DiaCurate shows that even outcome-correct demonstrations contain behavioral deficiencies, and that diagnosing and correcting these deficiencies improves the trained agent. We introduce a curation loop that links policy diagnosis to targeted edits of successful demonstrations, then re-diagnoses the policy after retraining to expose collateral failures and guide intervention redesign.

During the rebuttal, we added experiments and validation addressing the reviewers’ main shared concerns. These additions strengthen the evidence that DiaCurate improves the behaviors learned from successful demonstrations and that these improvements translate into better task performance.

Broader applicability across models and domains. We extended the study to Qwen3-4B-Thinking-2507 and to the interactive shopping domain WebShop. Compared with the corresponding outcome-filtered SFT reference, diagnosis-guided answer compaction improves Qwen’s BrowseComp-400 avg@3 from 29.3% to 31.2%, and targeted search editing improves WebShop-Test avg@3 from 41.9% to 44.1%. Both interventions also outperform the tested selection-only baseline. These studies extend the paper beyond its original model and deep-research setting, showing that the curation loop transfers with diagnostics and edits adapted to the task.

Stronger evidence for the reliability and attribution of the gains. We added two independent training repeats for both TE3 and the reference. Across three seeds, TE3 achieves 34.8 ± 0.4% versus 33.4 ± 0.2% on HLE-500 and 36.5 ± 2.5% versus 32.5 ± 1.1% on BrowseComp-400. A matched control using the original trajectory teacher to generate TE3’s brief guidance additions retains improvements on both benchmarks, supporting that the gains are not specific to the GPT editor. Additional compute-matched training checks leave the selected checkpoints unchanged, while question-level bootstrapping supports the development acceptance decisions and quantifies their uncertainty. These results strengthen the experimental basis for the paper’s conclusions.

Human validation and measurable search efficiency. We added human audits of verbosity, search cuts, and truncation sufficiency. The verbosity audit found unrequested, non-helpful explanations in 76 of 80 sampled long answers, supporting compaction that retains requested answer content while reducing distractions that hinder readability and answer extraction. Inspection of TE3’s cuts across 150 rollouts found that they target searches yielding no new evidence, supporting reductions in wasted search effort while preserving useful evidence. Overall, TE3 improves search behavior and efficiency: compared with the outcome-filtered SFT reference, it uses 7% fewer searches, 48% fewer zero-result searches, and 6% fewer total tool calls, with 9% lower search and page-visit service time. For truncation, the human audit judged the retained evidence sufficient at 49 of 50 cut points (98%), supporting the reliability of the sufficiency judgments. The subsequent policy regressions also highlight why DiaCurate evaluates edits through retraining and re-diagnosis. Together, these results support that simple rules and statistics of agentic traces can guide edits that align with human judgments and lead to meaningful behavioral improvements.

From OpenReview, CC BY 4.0. Authors as signed there; shown unedited.

Related papers

Loading the map…

Discussion (0)

Sign in to comment.