- Rating
- 2
- Confidence
- 3
- Informativeness
- 2
- Presentation
- 2
- Soundness
- 2
Read review
Paper summary
This paper identifies a behavioral blind spot in outcome-based filtering for agentic supervised fine-tuning (SFT): such filtering constrains only whether the final answer is correct, while SFT learns from the entire trajectory, so successful demonstrations can still teach undesirable behaviors such as redundant search and excessive verbosity. The authors propose DiaCurate, a diagnose-and-curate framework that profiles the trained policy with a nine-axis behavioral diagnostic, applies targeted edits to demonstrations, and retrains and re-diagnoses the resulting policy to detect collateral regressions. The framework is instantiated on a deep-research agent trained from a 5,298-example corpus and evaluated on HLE and BrowseComp. The key findings are that outcome-filtered SFT improves accuracy while substantially worsening search inefficiency and verbosity; that shortening trajectories to reduce over-search increases premature commitment and degrades accuracy; and that re-diagnosis motivates an evidence-use refinement (TE3) that reduces search inefficiency while preserving accuracy.
Critical strengths
- Interesting, clearly quantified findings about outcome-filtered SFT.
The paper concretely demonstrates that outcome-filtered SFT improves task accuracy while substantially increasing search inefficiency and verbosity, making the behavioral blind spot tangible rather than speculative. This is a valuable and timely empirical observation.
Critical weaknesses
- Limited technical novelty.
The underlying components, including behavioral diagnosis and feedback-driven filtering and editing, are established. The paper therefore contributes a framework and an empirical demonstration rather than a new method, which limits its contribution for a top-tier venue.
Questions for authors
-
Generality across models. The study uses a single model (Nemotron 3.5 Lightning). Do the behavioral blind spot of outcome filtering and the collateral-regression pattern generalize to other model families and scales, and can you provide evidence across additional models?
-
Generality across domains. The study focuses on a single agent domain, deep web research. Do the findings extend to other agent domains, such as coding or general tool-use agents, where the notions of "sufficient evidence" and "over-search" may differ considerably?
-
Reliability of the truncation point. The termination edits rely on an LLM-judge-based sufficiency judgment to decide where each trajectory is truncated, and the paper itself notes that this is not a proof of sufficiency. How accurate is this truncation point, and can you validate it with human evaluation of the sufficiency judgments?
5 comments
Re: "Generality across domains. The study focuses on a single agent domain, deep web research. Do the findings extend to other agent domains, such as coding or general tool-use agents, where the notions of 'sufficient evidence' and 'over-search' may differ considerably?" (question 2)
We have extended the study to WebShop [3], a widely used language-agent benchmark evaluated in ReAct [4] and Language Agent Tree Search [5]. It changes both the domain and the execution requirements: the agent navigates product pages, selects options, and purchases an item satisfying the user’s constraints. Actions change the interface state and available choices, and success is evaluated on the purchased item. This tests interactive shopping beyond evidence gathering and answer generation. We apply the same curation loop with the same teacher and student, adapting the diagnostics and edits to shopping.
We first diagnose fresh policy rollouts on the development set using six axes: action violation (actions refused by the current page), redundant searches (repeated queries), product revisits (reopening an already visited product), dead searches (leaving search results without opening a product), action-budget exhaustion (reaching the action limit without purchasing), and premature purchase (buying with fewer options selected than the instruction requests). We report the mean across these axes as F. Successful demonstrations retain redundant and dead searches, and these behaviors persist in the trained policy.
We target the search-related failure modes redundant searches and dead searches, defining T as their mean. As in the paper, acceptance requires T to decrease by at least 3 points and E to increase by no more than 3 points relative to the outcome-filtered SFT reference. Here E is the percentage of rollouts in which no opened product page displays all requested options within the price limit, guarding against evidence-acquisition failures that cutting search behavior could inadvertently induce.
We apply search editing (SE1) to successful teacher and policy demonstrations. For repeated queries, we remove the intervening detour that returns to the same search state; for dead searches, we remove the search and its unproductive continuation before the next search. We retain the remaining exploration and purchase steps, keeping the edited trajectory outcome-correct and internally consistent.
After retraining, we re-diagnose fresh rollouts on WebShop-Dev and apply this acceptance rule. The accepted policy achieves higher performance on the held-out WebShop-Test benchmark than the outcome-filtered reference and selection-only baseline.
These results support the framework’s applicability beyond deep research using task-specific diagnostic axes. We will add the WebShop study as an additional main section of the paper.
| Method | Version | WebShop-Dev F ↓ | WebShop-Dev ΔT | WebShop-Dev ΔE | Decision | WebShop-Test avg@3 (%) ↑ | WebShop-Test pass@3 (%) ↑ |
|---|---|---|---|---|---|---|---|
| Base model | – | 30.1 | – | – | – | 31.9 | 44.2 |
| Outcome-filtered SFT | (reference) | 17.7 | ref. | ref. | – | 41.9 | 52.2 |
| Error-mode-filtered SFT | – | 13.3 | −6.1 | +0.5 | – | 42.2 | 53.4 |
| Search editing | SE1 (ours) | 9.4 | −11.1 | −1.5 | accept | 44.1 | 54.2 |
Re: "Reliability of the truncation point. The termination edits rely on an LLM-judge-based sufficiency judgment ... How accurate is this truncation point, and can you validate it with human evaluation of the sufficiency judgments?" (question 3)
Thank you for raising this point. To ensure reliable truncation decisions, we have already applied a conservative grounding guardrail: for each trajectory judged sufficient and proposed for truncation, we verify by exact matching that the judge’s quoted evidence appears in the retained prefix. If this check fails, we reject the edit and keep the original trajectory. We will clarify this guardrail in the paper appendix.
We additionally assess truncation-point reliability through repeated LLM judge runs on 250 randomly sampled edited TE1/TE2 trajectories. The original cut was within three tool-call turns of the median across repeated judgments in 97% of trajectories, indicating high consistency of the cut location. We also perform human audits of 50 TE1/TE2 cut points. The annotator judges the retained evidence is sufficient in 49/50 cases (98%). In the remaining case, a candidate is named but one question constraint is not fully verified. This supports the sufficiency check’s reliability in the audited sample while revealing a residual failure mode: traces cut before all constraints are verified could teach the model to commit prematurely. This possible failure mode is consistent with the premature commitment and performance degradation observed for TE1/TE2. These observed failures motivated TE3, which combines targeted removal of unproductive search turns with an added sentence teaching how to use existing evidence to commit to an answer, while retaining the remaining research and verification.
We will add these reliability and human-audit results to the paper appendix.
Reference for the whole response:
[1] Where LLM Agents Fail and How They Can Learn From Failures.
[2] CurateEvo: Data-Curation Evolving for Agentic Post-Training.
[3] WebShop: Towards Scalable Real-World Web Interaction with Grounded Language Agents.
[4] ReAct: Synergizing Reasoning and Acting in Language Models.
[5] Language Agent Tree Search Unifies Reasoning, Acting, and Planning in Language Models.
Thank you for the thoughtful review.
Re: "Limited technical novelty. The underlying components, including behavioral diagnosis and feedback-driven filtering and editing, are established. The paper therefore contributes a framework and an empirical demonstration rather than a new method." (weakness 1)
We acknowledge that components such as LLM-based error-mode diagnosis and teacher-trace editing have been studied in prior work, including AgentDebug [1] and CurateEvo [2]. DiaCurate builds on these components with two contributions.
First, we introduce an iterative framework that connects behavioral diagnosis, targeted editing, and policy evaluation to improve the behavior learned from outcome-correct demonstrations. Diagnosis identifies which behaviors to change; targeted edits modify their supervision; retraining and re-diagnosis reveal both the intended improvements and collateral failures. A predefined gate requires improvement in the target behavior while keeping the increase in checkable error within tolerance, and rejected edits are revised using the new diagnosis. The TE1-TE3 progression shows why this integration matters: TE1/TE2 reduce search inefficiency but induce premature commitment and fail the gate. Re-diagnosis motivates TE3, which combines localized search edits with explicit evidence-use guidance. TE3 passes the gate on both development sets and improves held-out HLE/BrowseComp avg@3 from 33.5/33.3 to 35.2/34.1. This connects the framework to a concrete accuracy improvement through diagnosis-guided intervention redesign.
Second, we establish a scientific finding: outcome-correct agentic traces retain behavioral inefficiencies that provide headroom for improving downstream task performance. SFT on these successful traces improves accuracy while substantially worsening search inefficiency and verbosity. Our interventions show that this remaining behavioral deficiency is actionable: changing the supervision in already successful demonstrations can improve the resulting policy beyond the tested selection-only baselines. The gate also separates successful from harmful edits in our experiments: all accepted variants improve pass@3 on both benchmarks, while the rejected termination variants reduce avg@3 on both (Table 3). These findings link behavioral deficiencies within successful training traces, their targeted correction, and measurable improvements in the trained agent.
Re: "Generality across models. The study uses a single model (Nemotron 3.5 Lightning). Do the behavioral blind spot of outcome filtering and the collateral-regression pattern generalize to other model families and scales, and can you provide evidence across additional models?" (question 1)
To test whether DiaCurate generalizes across model families, we repeated the DR study with Qwen3-4B-Thinking-2507, following the same procedure as in the paper.
Outcome-filtered SFT again improves accuracy while worsening behavioral quality, and further filtering does not resolve this gap. The policy trained with our answer-compaction edit (AC1), accepted based on development diagnostics, improves both avg@3 and pass@3 on BrowseComp-400 over the outcome-filtered SFT baseline. These results support the transfer of diagnosis-guided curation to a second model family, with an accepted behavioral edit translating into better held-out benchmark performance.