acceptodds
Under review as a conference paper at ICLR 2027

From Knowledge to Decisions: Repairing Evidence Seeking after Full-Evidence Adaptation

Abstract

Full-evidence adaptation can transfer non-uniformly across evidence-conditioned competence and interactive evidence acquisition. On HotpotQA, it raises Oracle exact match from 58.6% to 68.0% but lowers Interactive exact match from 40.6% to 32.0%. Restoring inherited Base acquisition while retaining the adapted reader raises Interactive EM to 46.1%, showing that useful reader competence survives independently of the degraded adapted acquisition behavior. We call this a competence–policy transfer gap: adaptation improves evidence-conditioned competence, but that competence is not reliably translated into the decisions required to acquire useful evidence. Functional decoupling does not make inherited acquisition optimal. K2D-ALIGN (Knowledge-to-Decision Alignment) addresses the residual decision problem: when should retained reader competence justify a beneficial deviation from inherited evidence seeking? Matched continuation identifies Base-relative opportunities for learning root proposals; deployment-time admission authorizes replacement or executes exact Base fallback. Admission compares candidate and Base terminal shadow continuations before execution, trading additional model-side inference for conservative replacement. On HotpotQA, K2D-ALIGN improves Interactive EM from 46.1% to 52.3%; under matched initialization, Base-relative supervision contributes a further 3.9 points over the zero-update proposal. On 872 held-out cases, the frozen system gains 2.3 EM points over Base+FE and exceeds an approximately compute-matched Base Best-of-2 control by 3.1 points. Across four evidence-seeking benchmarks, it improves macro-average EM/F1 from 30.7/41.5 to 32.6/43.4. These results connect functional diagnosis to localized acquisition repair while preserving the reader competence that already transferred.

Primary area:
transfer learning, meta learning, and lifelong learning
Keywords:
Tool-Using Agents, Agent Adaptation, Interactive Retrieval, Evidence Seeking, Functional Transfer

Reviews

Among:
Shape:
1234this paper, Bottom 10%
Figure 1. Mean review rating of the reviewed papers at ICLR 2027. This paper averages 1.67 over 3 reviews, above 10% of the others.
Reviewer_g59YOpenReview ↗
Rating
2
Confidence
3
Informativeness
2(+1)
Presentation
2
Soundness
1
Read review
Paper summary

This paper examines the "competence-policy transfer gap" in tool-using retrieval-augmented generation (RAG): fine-tuning an LLM on full gold evidence (Full-Evidence SFT) improves evidence-conditioned reader accuracy but harms its interactive query formulation policy when deployed in an interactive search environment. The authors observe that simply pairing the inherited unadapted Base acquisition policy with the adapted reader (Base+FE) substantially recovers interactive performance.

To improve upon this baseline, the authors introduce K2D-ALIGN (Knowledge-to-Decision Alignment). K2D-ALIGN learns a specialized first-step retrieval action (root proposal) via Base-relative supervision and implements a deployment-time admission controller that executes parallel "shadow continuations" (evaluating both candidate and Base trajectories) before deciding whether to admit the proposed action or fall back to the Base policy. The method is primarily evaluated on Qwen3-4B using BM25 retrieval over Wikipedia across HotpotQA, 2WikiMultiHopQA, MuSiQue, and Bamboogle.

Critical strengths

Empirical Documentation of Reader-Acquisition Decoupling: The paper systematically documents that full-evidence SFT degrades interactive tool-use while improving reader competence, and clearly quantifies that simple functional decoupling (using Base acquisition with an FE reader) serves as a surprisingly effective and compute-free recovery baseline.

Critical weaknesses
  1. Conceptual Obsolescence Relative to Modern Agentic RL Paradigms: The fundamental motivation of this paper is severely dated. Modern interactive information seeking and multi-step tool use have largely transitioned to unified Agentic Reinforcement Learning (e.g., Search-R1, ReSearch, Tool-RL), where search, reasoning, and reflection are jointly optimized end-to-end. Trying to patch an SFT-induced policy degradation via an ad-hoc heuristic pipeline—freezing an adapted reader, falling back to base acquisition, and performing dual-branch shadow rollouts solely to modify the first search action—offers minimal scientific or practical insight to the modern research community. Operating strictly in a toy BM25 retrieval environment over legacy 2018 multi-hop QA benchmarks (HotpotQA) further widens this gap from practical web-agent and dense-retrieval realities.

  2. Severe Empirical Flaws, Missing Call Traces, and Disproportionate Compute Overhead: The submission exhibits critical experimental compromises and poor efficacy:

  • Missing Traces and Unreliable Reporting: In Table 3 and Figure 2d, the authors explicitly state: "The 52.34-EM K2D run lacks a complete call trace; Figure 2d uses a separate 50.78-EM audit." Submitting a primary claim where the corresponding compute trace was lost, substituting a lower-performing run to plot compute-quality trade-offs, and leaving baseline quantities blank ("unrecorded quantities are left blank, not imputed") is unacceptable for a top-tier venue.
  • Disproportionate Cost for Marginal/Negative Gains: K2D-ALIGN triples inference compute (model calls jump nearly threefold from 6.08 to 17.71), yet the macro-average EM across four benchmarks improves by only 1.9 points over the compute-free Base+FE baseline (30.7 to 32.6). Crucially, on the most challenging multi-hop benchmark, MuSiQue, K2D-ALIGN actually regresses performance from 13.2 to 12.6 EM (-0.6 points). Combined with tiny sample slices (root supervision from merely 24 pairs; Hermes evaluated on 32 cases with 0.00% gain) and failure to replicate on Qwen2.5-7B, the empirical claims are fragile.
Questions for authors
  1. Given that modern multi-hop retrieval and information seeking have largely transitioned to end-to-end Agentic RL (e.g., Search-R1, ReSearch) and dense/hybrid retrieval, what foundational insights or practical advantages does this piecewise SFT decoupling and dual-branch heuristic patch offer over integrated RL approaches?
  2. How can the empirical validity of K2D-ALIGN be guaranteed when the primary 52.34-EM run lacks a complete call trace, trade-off curves are plotted using a separate lower-performing run, and the method exhibits negative transfer on MuSiQue (-0.6 EM) despite tripling inference compute?
6 comments

Thank you for recognizing the compute-free Base+FE recovery baseline and raising direct questions on agentic-RL positioning, trace provenance, and compute–quality trade-offs.

1. Foundational Insight Relative to End-to-End Agentic RL (Q1, W1)

End-to-end RL (Search-R1, ReSearch, ToolRL, SearchGym) optimizes a scalar return coupling evidence acquisition (π\pi) and evidence-conditioned reading (rr). Scalar return changes alone cannot identify which sub-function improved or degraded after domain post-training, distillation, or continual updates.

Component substitution isolates both interfaces at zero extra inference compute: holding evidence fixed measures reader transfer (Oracle EM: 58.6 → 68.0 on HotpotQA), while holding the adapted reader fixed isolates acquisition loss (32.0 EM) and recovers 46.1 EM at 5.52 calls/case (Base+FE, Table 2 [orig. Table 3]; 26.1 → 30.7 macro EM across four benchmarks at 6.08 calls/case, Table 4 [orig. Table 5]).

K2D-ALIGN then formulates Base-relative credit assignment: a candidate root is supervised only when it improves terminal correctness over the inherited action from the same state under a shared reader and continuation (ΔEMi>0\Delta\mathrm{EM}_i > 0). Under matched initialization, 24 states, 48 updates, and identical admission (Table 1 [orig. Table 2]), K2D-positive rises from 48.44 (zero-update) to 52.34 EM (5 repairs, 0 regressions; paired 95% CI [+0.78, +7.81]), whereas Absolute-outcome (rewarding actions merely co-occurring with success), Direct, and Relation stay at 48.44. Functional attribution and Base-relative credit thus complement joint RL.

Manuscript: Sections 1–2; Table 1; Appendix B (Table 7).

2. Rebuttal Controls: Visible Reasoning & Modern Interfaces (W1)

Unconstrained visible-reasoning ReAct and IRCoT (new rebuttal results; prompt protocol in Appendix H). Removing scratchpad suppression, we ran 256 native-interface evaluations (128 × 2) allowing two explicit reasoning sentences per turn (384-token cap) retained across turns under the shared Qwen3-4B backbone, FE reader, BM25 index, and 5-search cap (all invalid actions retained):

Configuration (128 HotpotQA dev)EMF1InvalidCalls / case
ReAct (visible reasoning)37.5047.120/1284.33 (logged)
IRCoT (visible reasoning)40.6351.234/1285.27 (logged)
K2D-ALIGN (dev operating point)52.3464.410/12816.34 (reconstructed)

Descriptive paired EM differences on this roster are +14.84 pp (95% CI [+7.03, +22.66]) and +11.72 pp ([+5.47, +18.75]), testing scratchpad visibility under matched search caps with K2D's call count shown explicitly. A blinded LLM-judge check across all 48 EM-discordant pairs (without method labels) attributes 22 wins / 1 loss vs. ReAct and 16 wins / 0 losses vs. IRCoT to substantive correctness rather than formatting.

Dense/hybrid & RL-agent diagnostics (rebuttal-only). Across 48 BrowseComp-Plus questions under BM25, official dense, and hybrid retrieval with byte-identical Base/FE reader swaps (288 runs), plus 48 bounded 8-action episodes and 96 identical-evidence reader replays on RL-trained SearchGym-Qwen3-4B, the substitution diagnostic executes cleanly across modern retrievers and a trained search agent.

3. Same-Branch Cost Provenance for 52.34 EM (Q2, W2)

We removed the 50.78-EM substitution and incomplete H872 rows. Replaying the exact saved terminal branches yielding 52.34 EM / 64.41 F1 gives 13.41 pre-decision plus 2.94 selected-path calls—16.34 model calls and 4.63 backend requests alongside 1.47 committed searches per case—verified case-by-case against 128 instrumented traces under deterministic read-only continuation. Unmeasured token/latency cells remain N/A.

Manuscript: Table 2 [orig. Table 3]; Appendix D.3 (Tables 15–16, Figure 3).

4. MuSiQue Corpus Condition & Compute Trade-off (Q2, W2)

In the 1,625-case suite (Table 4 [orig. Table 5]), all benchmarks query Wikipedia-HotpotQA—an out-of-index test where missing MuSiQue paragraphs cap every method at 9.0–13.2 EM (K2D: 12.6 EM / 20.8 F1 vs. Base+FE: 13.2 / 20.4, while preventing unguarded Scaled Root-DPO's drop to 11.4 EM and 26.2 macro EM via Base fallback).

On MuSiQue's native paragraph-pool index (128 cases, Table 24), K2D-ALIGN raises Base+FE from 19.53 to 26.56 EM (+7.03 pp, [+1.56, +12.50]; 11 repairs, 2 regressions), alongside 40.83 → 43.12 EM on H872 (+2.29 pp, [+0.92, +3.78]; 31 repairs, 11 regressions, Table 3).

Base+FE gives compute-free recovery; K2D-ALIGN trades ~2.7–2.9× calls for selective repair.

Manuscript: Sections 3.4, 4.5–4.6; Tables 3–4, 8, 24; Appendices A.1, C.

Reviewer_g59YOpenReview ↗

Thanks For Reply

Thank you for the detailed rebuttal and for running the additional controls (visible-reasoning ReAct/IRCoT, the native-index MuSiQue slice, and the dense/hybrid and SearchGym diagnostic checks). In recognition of these added analyses and the removal of the 50.78-EM substitution, I have raised my Informativeness score from 1 to 2.

However, my core concerns remain unresolved, so I am maintaining my overall rating of 2 (Weak Rejection):

  1. Trace Provenance & Compute–Quality Trade-off: While removing the mismatched 50.78-EM substitution improves clarity, the 16.34 calls/case for the 52.34-EM run is still a post-hoc reconstruction from deterministic replay rather than original runtime logs, and token/latency measurements remain N/A. Meanwhile, across the full 1,625-case benchmark suite, K2D-ALIGN still requires ~2.7–2.9× model calls for a +1.9 macro EM gain, while its larger improvements rely on small slices (e.g., 24 states / 5 repairs on HotpotQA dev, 128 cases on native MuSiQue).
  2. Positioning Relative to Agentic RL: Characterizing end-to-end Agentic RL as an unattributable scalar-return black box is not convincing. Separating retrieval hit/recall from reader accuracy, reward shaping, and reference-policy/counterfactual advantage estimation are standard in RL. Moreover, a zero-cost component-substitution diagnostic does not justify paying ~3× inference compute for an inference-time shadow-rollout patch—especially since SearchGym-Qwen3-4B was only tested for diagnostic compatibility on 48 cases without reporting a direct accuracy and cost comparison against K2D-ALIGN.

Thank you for the follow-up.

Cost and provenance. The 16.34 figure is a reconstructed model-call count for the historical operating point; token and latency costs remain unmeasured. The full-suite result therefore describes an accuracy–inference trade-off. The smaller matched study addresses learning attribution within its evaluated setting. The component-substitution recovery itself uses Base acquisition with the adapted reader; the shadow-rollout cost belongs to the subsequent repair experiment.

Relation to RL. Retrieval metrics, reward shaping and counterfactual advantages are established tools. Our contribution concerns a specific adaptation-induced transfer phenomenon and its identification through controlled component substitution. K2D tests whether intervening on the identified acquisition component can realize additional competence. The 48-case SearchGym experiment extends the diagnostic to a trained agent’s evidence interface.

Reviewer_CZnuOpenReview ↗
Rating
2
Confidence
2
Informativeness
2
Presentation
1
Soundness
2
Read review
Paper summary

This paper investigates why fine-tuning a model with complete evidence can improve its answering ability while hurting its ability to search for evidence. The authors show that restoring the original retrieval policy while keeping the adapted reader can recover much of the lost performance. Based on this observation, they propose K2D-ALIGN, which learns to improve the initial retrieval action through comparisons with the original policy. During inference, the method evaluates candidate and original actions through additional rollouts before deciding which one to execute. The approach improves interactive QA performance on HotpotQA and several other benchmarks.

Critical strengths
  1. The observation that better evidence-conditioned answering does not necessarily lead to better retrieval behavior is interesting. The component substitution experiments provide a reasonable explanation for the performance drop after full-evidence fine-tuning.

  2. The experimental section includes several useful controls, particularly the zero-update baseline and compute-matched Best-of-2 comparison. These help distinguish the contribution of the learned policy from the benefit of additional inference.

Critical weaknesses
  1. The inference cost is quite high relative to the improvement. On the held-out HotpotQA evaluation, K2D-ALIGN increases EM from 40.83 to 43.12, while the number of model calls increases from 6.1 to 16.1 per example. I am not sure this trade-off is attractive for practical search agents.

  2. The method only modifies the first retrieval action and leaves subsequent search decisions unchanged. In multi-step search tasks, later decisions can be equally important. It is unclear how much the proposed method can help when the main problem occurs later in the trajectory.

  3. The evidence for the learned policy itself could be stronger. The main controlled supervision experiment uses only 24 retained positive action pairs. On the held-out evaluation, the trained method improves EM by just 1.37 points over the zero-update admission baseline, without a separate confidence interval for this difference. More training data and repeated experiments would help establish how much improvement actually comes from learning better retrieval actions.

Questions for authors

Weaknesses

2 comments

Thank you for recognizing the competence–policy transfer finding and our controlled baselines (zero-update and Best-of-2). Below we address practical inference cost, root-only intervention in multi-step search, and evidence for the learned policy.

1. Inference Cost and Practical Two-Tier Deployment (W1)

Our findings establish two explicit operating points for practitioners:

  1. Zero-overhead recovery (Base+FE): Pairing frozen Base acquisition with the adapted reader recovers Interactive EM from 32.0 → 46.1 on HotpotQA (5.52 model calls/case, Table 2 [orig. Table 3]), 32.34 → 40.83 on H872 (Table 3 [orig. Table 4]), and 26.1 → 30.7 macro EM across four benchmarks (6.08 calls/case, Table 4 [orig. Table 5]) with no extra model calls.

  2. Selective higher-compute repair (K2D-ALIGN): When answer accuracy outweighs model calls—or when external search APIs are metered—dual-shadow admission trades extra inference for conservative improvement, raising H872 EM from 40.83 to 43.12 (+2.29 pp, 95% CI [+0.92, +3.78]; 31 repairs, 11 regressions) while reducing committed live searches (1.76 → 1.47 on dev; −0.196 on H872). On the 128-case audit (Table 15), compute-matched Base Best-of-2 (15.85 calls) reaches 43.8 EM versus K2D-ALIGN's 52.3 EM.

Tables 15–16 and Figure 3 now pair 52.34 EM directly with a source-derived ledger from the same saved terminal branches: 16.34 model calls (13.41 pre-decision + 2.94 execution) and 4.63 backend retrieval requests alongside 1.47 committed searches per question, verified against 128 instrumented traces. All 872 held-out executions match their selected stored shadows (Appendix A.1).

Manuscript: Section 3.5; Tables 2–4, 15–16; Figure 3; Appendices A.1, D.3.

2. How Root Repair Shapes Later Multi-Step Decisions & Its Boundary (W2)

Candidate and Base roots start from the identical state s0s_0, enabling matched comparison under a shared reader and continuation (Figure 1). Although πBase(at∣st)\pi_{\text{Base}}(a_t \mid s_t) is frozen for t>0t>0, replacing a0a_0 changes retrieved evidence E1E_1 and subsequent states (s1,s2,… )(s_1, s_2, \dots), causing the fixed conditional policy to make different follow-up search and stopping decisions along the altered trajectory (Section 3.2).

We also quantify where root-only repair reaches its limit: on hop-stratified MuSiQue (Table 5a), K2D-ALIGN improves over Base+FE at 2 hops (30.95 → 35.71) and 3 hops (20.93 → 27.91; overall +3.91 EM, 95% CI [+0.78, +7.81]), while plateauing at 4 hops (2.33 → 2.33) where independent downstream drift dominates. Appendix F (Table 18 [orig. Appendix E]) scopes the 16-case saved-state pilot as zero-shot prefix transfer of the root-trained adapter rather than dedicated intermediate-state training.

Manuscript: Figure 1; Sections 3.2, 4.3, 4.6; Table 5a; Appendix F.

3. Policy Learning Attribution, Larger Training Sets & Repeated Runs (W3)

  • Matched-gate attribution on the 24 states (Table 1 [orig. Table 2]): Holding initialization, reader, continuation, 24 states, 48 updates, and dual-shadow admission identical, K2D-positive supervision raises EM from 48.44 (zero-update + admission) to 52.34 (+3.91 pp; 5 repairs, 0 regressions; paired 95% CI [+0.78, +7.81], Pboot=0.994P_{\mathrm{boot}}=0.994), whereas Direct, Relation, and Absolute-outcome updates remain at 48.44.

  • Why H872 reports complete-system transfer (Table 3): Because archived H872 zero-update and K2D runs did not share identical gate/manifest records (Section 3.4, Appendix A.1), H872 (+2.29 EM, 95% CI [+0.92, +3.78]) measures the combined proposal-and-admission intervention rather than an isolated 24-pair held-out delta.

  • Larger preference data and repeated evaluations: Beyond the 24-pair attribution slice, we trained on an independently collected 192-pair balanced preference set (96 candidate-winning, 96 Base-winning; Appendix D) and evaluated across 256 disjoint sealed cases (Section 4.5), improving EM from 41.41 to 45.31 (+3.91 pp) and F1 from 55.07 to 58.59 (+3.52 pp) with −0.332 searches/case (Pboot=0.956P_{\mathrm{boot}}=0.956). Four seed–budget replications (Table 6), native-corpus MuSiQue (19.53 → 26.56 EM, +7.03 pp, 95% CI [+1.56, +12.50], Table 24), and 256 new visible-reasoning ReAct/IRCoT rebuttal evaluations (37.50/40.63 EM at 4.33/5.27 calls vs. K2D's 52.34 at 16.34 calls, Appendix H protocol; blinded equivalence audit on all 48 discordant pairs) further support robustness.

Manuscript: Sections 4.4–4.6; Tables 1, 3, 6, 24; Appendices D, H.

Thank you again for your thoughtful feedback. We have added experiments with a larger preference set and repeated evaluations, clarified the effects and limitations of root-only intervention, and expanded the cost accounting. We also distinguish the zero-overhead Base+FE recovery from the additional accuracy–compute trade-off of K2D-ALIGN.

We would appreciate your assessment of whether these additions address your concerns, particularly regarding the evidence for policy learning and the practical trade-offs. We would be happy to provide any further clarification.

Thank you for your time and consideration.

Reviewer_m74aOpenReview ↗
Rating
1
Confidence
3
Informativeness
2
Presentation
1
Soundness
1
Read review
Paper summary

This paper investigates a competence–policy transfer gap in retrieval-based agents: fine-tuning on complete evidence can improve answering ability while degrading the search decisions needed to obtain useful evidence. On HotpotQA, full-evidence adaptation increases oracle exact match (EM) from 58.6% to 68.0%, but reduces interactive EM from 40.6% to 32.0%. Restoring the original acquisition policy while retaining the adapted reader raises interactive EM to 46.1%, isolating the acquisition-side failure. The authors propose K2D-ALIGN, which repairs only the first retrieval decision. During training, candidate actions are compared with the original action under a shared reader and frozen continuation policy. Candidates that improve terminal correctness provide preference supervision. At deployment, the system evaluates both branches through shadow rollouts and uses reader-based admission to select a candidate or fall back to the original action. On 872 held-out HotpotQA examples, K2D-ALIGN improves EM from 40.83% to 43.12%, outperforming an approximately compute-matched Best-of-2 baseline. Four-benchmark results show modest, non-uniform gains. Overall, the study supports separating evidence use from evidence acquisition and applying localized repair, while highlighting substantial inference overhead and limitations in admission reliability and transfer.

Critical strengths
  1. The paper addresses a meaningful problem.
    It shows that improved performance with supplied evidence does not necessarily translate into better autonomous retrieval, highlighting a useful distinction in agent evaluation.

  2. The component-substitution experiment is compelling.
    Restoring Base acquisition while retaining the adapted reader raises interactive EM from 32.0% to 46.1%, clearly demonstrating that useful reading ability survives the acquisition degradation.

  3. The training signal targets incremental improvement.
    Candidates are compared against Base actions under a shared reader and continuation policy, helping distinguish beneficial actions from actions that merely occur in successful trajectories.

  4. The evaluation includes useful controls.
    Held-out testing, zero-update ablations, and an approximately compute-matched Best-of-2 baseline help assess whether gains extend beyond development-set selection and additional inference computation.

Critical weaknesses
  1. The diagnosis does not clearly justify the repair.
    The paper identifies acquisition failures, including premature stopping, but does not explain why changing only the first search adequately addresses them.

  2. The benefit of training remains unclear.
    Gains may partly come from additional rollouts and selection. On held-out data, training adds only 1.37 EM points over the zero-update system, without a paired confidence interval.

  3. The intervention-location study is inconclusive.
    It uses only 16 questions and tests a first-step-trained adapter at later steps, which does not fairly assess separately trained later-step interventions.

  4. Admission is unreliable and costly.
    On Hermes, admission fails to capture the benefit of better candidates. On HotpotQA, model calls increase roughly 2.6-fold for a 2.29-point EM gain.

  5. Generalization and baseline comparisons are limited.
    Strongest results come from Qwen3-4B on HotpotQA. Gains elsewhere are inconsistent, and comparisons with trained search agents are lacking.

Questions for authors
  1. “Repairing only the first retrieval step” can understate what the method actually does.
    Section 3 emphasizes replacing only the first action, but Sections 3.4–3.5 gradually reveal that deciding whether to replace it requires completing both branches to their final answers, comparing them, and then executing the selected action. Intervening only at the first step is therefore very different from performing computation only at that step. This distinction should be clarified earlier.

  2. The sample-selection criterion and training weights are easy to confuse.
    Section 3.2 first defines a multi-component advantage AiA_i, which readers may naturally assume determines which actions are selected for training. Section 3.3 then explains that inclusion depends on improved terminal EM, while AiA_i only weights the retained samples. This distinction is central to understanding the method, yet requires cross-referencing multiple equations.

  3. The admission rule is considerably more complex than “replace when better.”
    Section 3.4 allows admission under three conditions: the branches produce the same answer, the pairwise comparison exceeds a threshold, or the Base answer lacks support while the candidate answer is supported. Furthermore, pairwise comparison and pointwise support assessment both use A/B/C labels with different meanings. This notation reuse adds unnecessary cognitive load.

  4. Different scores for the same method are explained in scattered locations.
    The main development result is 52.34 EM, whereas the computational audit corresponds to a separate run achieving 50.78 EM. Although the authors acknowledge this distinction, readers could easily associate the 52.34 score with the costs measured in the other run. Figure 2, Table 3, and the appendix should distinguish these runs more clearly.

  5. Different evaluation conditions for the same dataset make results difficult to interpret.
    For example, MuSiQue uses the HotpotQA retrieval corpus in the four-benchmark evaluation, its own context-based corpus in another experiment, and a separate hop-stratified sample in an additional analysis. These experiments address different questions, but their protocol details are scattered across the main text and appendices, making it difficult to determine which results are directly comparable.

2 comments

Thank you for recognizing the component-substitution diagnosis, Base-relative signal, and ablations, and for pinpointing every structural and notational ambiguity.

1. Premature Stopping, Root Repair & Location Pilot (W1, W3)

Addressing the diagnosed failure proceeds in two steps:

  1. Decoupling (Base+FE) resolves adapter-induced premature stopping at zero extra cost: FE-SFT stops after one query on 78.9% of cases (32.0 EM). Restoring frozen Base acquisition with the adapted reader restores Base stopping (46.9% one-query stops, Table 17b [orig. Table 1b]) and recovers 46.1 EM (Table 2 [orig. Table 3]).

  2. Root repair (K2D-ALIGN) reshapes trajectories on top of Base+FE (46.1 → 52.3 EM): Candidate and Base roots share state s0s_0. Although πBase(at∣st)\pi_{\text{Base}}(a_t \mid s_t) is frozen for t>0t>0, replacing a0a_0 changes retrieved evidence E1E_1 and later states (s1,s2,… )(s_1, s_2, \dots), altering when the conditional policy chains queries or stops. Deeper-hop independent errors remain bounded (+0.00 EM at 4-hop MuSiQue, Table 5a).

16-question pilot (W3; Appendix F, Table 18 [orig. Appendix E]). Scoped explicitly as zero-shot transfer of the root-trained adapter to multi-turn prefixes—where prefix shift confounds t1,t2t_1, t_2—not separately trained later-step adapters.

Manuscript: Sections 1, 3.1–3.2, 4.3; Table 17b; Appendix F.

2. Isolating Learned Supervision from Rollouts & Selection (W2)

Matched-gate attribution (Table 1 [orig. Table 2]). All rows share initialization, FE reader, retriever, continuation, 24 states, 48 updates, and dual-shadow admission:

Proposal recipe (matched init + admission)EMF1vs. Base+FEvs. Zero-update
Zero-update (rollouts + admission only)48.4462.41+2.340.00
Direct / Relation / Absolute-outcome48.4462.41–62.93+2.340.00
K2D-positive (ΔEM>0\Delta\mathrm{EM}>0, weighted DPO)52.3464.41+6.25+3.91 ([+0.78, +7.81])

Holding rollouts and admission fixed isolates +3.91 EM (5 repairs, 0 regressions, Pboot=0.994P_{\mathrm{boot}}=0.994); Uniform-weight DPO and Root-action SFT ablations are in Table 13b.

Held-out system evidence (Table 3 [orig. Table 4]). Because archived H872 zero-update and K2D runs did not share identical gate/manifest states (Section 3.4, Appendix A.1), H872 (40.83 → 43.12 EM, +2.29 pp, [+0.92, +3.78]; 31 repairs, 11 regressions) evaluates the fixed system rather than an isolated 24-pair delta, supported by four seed–budget replications (Table 6) and an independent 192-pair balanced collection on 256 sealed cases (41.41 → 45.31 EM, +3.91 pp; −0.332 searches/case, Section 4.5).

3. Full-Shadow Cost, Same-Branch 52.34 Ledger & Fidelity (W4, Q1, Q4)

  • Early disclosure (Q1): The Abstract, Section 1, and Figure 1 state upfront that replacing a0a_0 completes both terminal shadows before execution.

  • Same-branch 52.34 ledger (Q4): Removing the 50.78-EM run, we reconstructed 16.34 model calls (13.41 pre-decision + 2.94 execution) and 4.63 backend requests alongside 1.47 committed searches from the exact saved branches yielding 52.34 EM / 64.41 F1 (Tables 2, 15–16; Figure 3), verified on 128 instrumented traces.

  • Fidelity vs. transfer (W4): All 872 held-out answers, traces, and first retrievals match selected shadows (Appendix A.1); Hermes (7/32 admitted, +0.00 realized vs. +15.63 forced opportunity, Table 5b) remains the admission boundary.

4. Additional Baselines & Generalization (W5)

On native-corpus MuSiQue (128 cases, Table 24), K2D-ALIGN raises Base+FE from 19.53 to 26.56 EM (+7.03 pp, [+1.56, +12.50]; 11 repairs, 2 regressions).

In new rebuttal controls with retained explicit reasoning (Appendix H protocol; two sentences / 384 tokens), ReAct and IRCoT reach 37.50 (4.33 calls) and 40.63 EM (5.27 calls) on the 128 dev cases vs. K2D's 52.34 (16.34 calls; CIs [+7.03, +22.66], [+5.47, +18.75]).

Rebuttal-only BrowseComp-Plus (288 runs) and RL-trained SearchGym-Qwen3-4B (144 runs) verify diagnostic portability.

5. Presentation & Protocol Clarifications (Q2, Q3, Q5)

  • Inclusion vs. weight (Q2): Section 3.3 (Eq. 3) puts discrete inclusion (ΔEMi>0,Vi=0,ai+≠aiBase\Delta\mathrm{EM}_i>0, V_i=0, a_i^+\neq a_i^{\text{Base}}) before Ai(ai+)A_i(a_i^+) DPO weighting (Figure 1b; closed-form rewards in Eq. 10).

  • Critic notation & gates (Q3): Section 3.4 uses descriptive pairwise (Base/candidate/neither; Eq. 4) and pointwise (supported/contradicted/insufficient; ρib\rho_i^b, Eq. 5) labels, separating dev support-crossing (Γicross\Gamma_i^{\mathrm{cross}}, Eq. 6) from H872 agreement/pairwise (Γihist\Gamma_i^{\mathrm{hist}}).

  • Protocol map (Q5): Table 8 and Appendix C consolidate all MuSiQue, Bamboogle (109 vs. 125), and community protocols.

Thank you again for your detailed feedback. We have supplemented the learning evidence with controlled comparisons and repeated evaluations, clarified the connection between acquisition diagnosis and root-level repair, and revised the presentation of admission, evaluation protocols, and inference costs.

We would appreciate your assessment of whether these revisions address your concerns, particularly regarding the contribution of learning beyond rollouts and selection, and the scope of root-only intervention. We would be happy to clarify any remaining questions.

Thank you for your time and consideration.

From OpenReview, CC BY 4.0. Authors as signed there; shown unedited.

Related papers

Loading the map…

Discussion (0)

Sign in to comment.