acceptodds
Under review as a conference paper at ICLR 2027

Seek What You Ignorant: Boundary-Aware Behavioural Calibration for Search Agents

Abstract

Reliable agentic search requires large language models to calibrate trust in paramet- ric memory and retrieved evidence, deciding when to use either source, integrate both, or abstain. Yet outcome rewards, whether binary or graded, provide only sparse feedback on completed trajectories and limited direct guidance for these intermediate decisions. Moreover, appropriate reasoning patterns vary across knowledge states. We propose Knowledge boundary Self-Distillation (KbSD), a quadrant-adaptive self-distillation framework that combines dense token-level supervision with GRPO-based reinforcement learning. KbSD estimates parametric certainty and response stability from repeated retrieval-free samples, and retrieval quality from the relevance of retrieved passages. These signals, together with quadrant labels and ground-truth answers, form privileged hints for a teacher that shares the student’s model parameters. Conditioned on these hints and the student’s observed history, the teacher provides token-level guidance to the hint- free student. This information asymmetry enables dense supervision without a larger or separately trained teacher, and the student requires no privileged hints at inference. To accommodate differences in reasoning diversity across the four knowledge-boundary quadrants, KbSD uses mode-seeking reverse KL for concen- trated evidence integration, mass-covering forward KL for diverse refusal patterns, and Pareto-optimal bidirectional KL for states that require both precise source selection and broad coverage. Experiments across nine benchmarks show lower unreliability than the evaluated baselines. We report task F1 alongside refusal rates to characterize the accuracy–refusal trade-off and interpret reliability gains in the context of answer coverage.

Primary area:
foundation or frontier models, including LLMs
Keywords:
retrieval-augmented generation, agentic search, large language models

Reviews

Among:
Shape:
1234this paper, Bottom 49%
Figure 1. Mean review rating of the reviewed papers at ICLR 2027. This paper averages 2.25 over 4 reviews, above 49% of the others.
Reviewer_Z79UOpenReview ↗
Rating
2
Confidence
2
Informativeness
2
Presentation
2
Soundness
2
Read review
Paper summary

The paper proposes Knowledge boundary Self-Distillation (KbSD), a framework for improving calibration in search agents. Building on the four knowledge-boundary quadrants, KbSD employs a privileged teacher that shares the student’s model parameters but has access to privileged information. The student is trained through quadrant-adaptive self-distillation combined with GRPO, using different KL depending on the state. Experiments across nine benchmarks show substantial reductions in the proposed unreliability metric.

Critical strengths
  1. The proposed privileged self-distillation framework provides a reasonable approach to converting knowledge-boundary information into token-level training signals without requiring a larger, separately trained teacher.

  2. The paper evaluates the method across multiple benchmarks and model scales, which demonstrates its effectiveness.

Critical weaknesses
  1. The reported reliability improvements are heavily influenced by increased refusal rates. The primary unreliability metric is defined as 1 − (F1 + Refusal Rate), meaning that a higher refusal rate directly reduces the reported unreliability. As shown in Table 2, KbSD improves average F1 from 20.50 to 22.10 compared with KbPO, while the refusal rate increases substantially from 15.18% to 30.30%. It remains unclear to what extent the method improves actual answer quality rather than encouraging more conservative refusal behavior.

  2. The reliability and stability of query-level knowledge-boundary classification are not sufficiently justified. How is query-level retrieval quality determined from multi-turn search trajectories? Is this estimation sufficiently robust and consistent across different rollouts? Moreover, since the knowledge quadrants are determined before training, could the assigned labels become outdated as the model's knowledge and retrieval capabilities evolve during training?

Questions for authors

Does the privileged teacher improve intermediate search decisions? Since the teacher has access to ground-truth answers, it is unclear whether distillation improves retrieval and evidence assessment or simply guides the student toward known answers.

4 comments
AuthorsOpenReview ↗

Rebuttal for W1

W1: Answer Quality and Selective Abstention

We thank the reviewer for analyzing Unreliability and refusal behavior. We agree that its reduction is largely driven by increased refusal rates and cannot alone distinguish better answering from conservative abstention. We clarify this through the training objective, complementary metrics, and paired behavioral analysis.

1. Training Objective.

Unreliability is an evaluation metric, not KbSD's training objective. Eq. (11) explicitly penalizes refusal in the Integrated, External, and Internal quadrants, where answering is expected. Thus, KbSD encourages decisions based on the reliability of internal knowledge and external evidence rather than indiscriminate refusal.

We additionally report Truthful Helpfulness Score (THS), introduced by CRaFT [1] and adopted by GRAIT [2]. THS considers changes in correct and incorrect response rates relative to a reference model, without treating refusal as correctness. Using KbPO as reference, KbSD achieves THS = +4.46, providing evidence beyond Unreliability.

2. Answer Quality Beyond Increased Refusal.

Table 2 shows that KbSD improves overall F1 from 20.50 to 22.10 (+1.60 points), while Refusal Rate increases from 15.18% to 30.30%. Since overall F1 includes all queries, with refusals assigned zero, this improvement is not due to excluding unanswered queries.

MetricKbPOKbSD
Overall F1 (%) ↑20.5022.10
Refusal Rate (%)15.1830.30
THS (KbPO reference) ↑0.00+4.46
Conditional F1 (G.1)BaselineBetter on 8/9
Matched-Coverage F1 (G.2)Ranked baselineBetter on 8/9

Appendix G.1 reports higher Conditional F1 on 8/9 benchmarks. At matched coverage (Appendix G.2), KbSD also outperforms consistency-ranked KbPO on 8/9 benchmarks, suggesting its gains are not solely attributable to reduced coverage. However, matched coverage does not guarantee identical answered queries and cannot fully separate reasoning gains from answer selection.

3. Query-Level Behavioral Transitions.

We further compare KbPO and KbSD on identical queries. On Qwen2.5-3B/7B, KbSD refuses 26%/26% of queries previously answered incorrectly by KbPO, versus 7%/6% of previously correct queries. Thus, refusal is 3.7/4.3 times more frequent on baseline-incorrect than baseline-correct responses.

Moreover, some queries previously answered incorrectly or refused become correctly answered under KbSD. For 3B, these gains exceed losses from previously correct responses becoming incorrect or refused, yielding a net increase in correct answers.

These results suggest that KbSD both avoids some baseline errors and recovers correct answers. However, baseline-incorrect queries are not necessarily unanswerable, so these transitions do not directly measure correct-refusal rates.

In summary, although increased refusal contributes substantially to lower Unreliability, the explicit refusal penalty, higher overall F1, positive THS (+4.46), coverage analyses, and paired transitions provide evidence beyond aggregate refusal rates. We will clarify this trade-off and avoid overstating conclusions based solely on Unreliability.

References

[1] Zhu et al. (2025). Utilize the Flow Before Stepping into the Same River Twice: Certainty Represented Knowledge Flow for Refusal-Aware Instruction Tuning. AAAI 2025. https://ojs.aaai.org/index.php/AAAI/article/view/34812

[2] Zhu et al. (2025). GRAIT: Gradient-Driven Refusal-Aware Instruction Tuning for Effective Hallucination Mitigation. Findings of NAACL 2025. https://aclanthology.org/2025.findings-naacl.223/

AuthorsOpenReview ↗

Rebuttal for W2

W2: Reliability and Stability of Knowledge-Boundary Estimation

We thank the reviewer for raising concerns about the reliability of query-level knowledge boundaries. We clarify how retrieval quality is estimated from multi-turn trajectories, its consistency across rollouts, and the stability of initial labels during training.

1. Query-Level Retrieval Quality

As described in Appendix B.1, KbSD evaluates the accumulated evidence from the student's executed search trajectory, rather than scoring individual passages or averaging retrieval-turn scores:

\begin{cases} \operatorname{clip}(J(q,C_{<t},u_t),0,1), & \text{valid score},\\ 0, & \text{otherwise}. \end{cases}$$ Here, $C_{<t}$ denotes observed retrieved content and $u_t$ the current summary. We concatenate retrieved evidence in observation order (up to 8,000 characters) and include the summary (up to 2,000 characters). GPT-4o (temperature = 0) assigns scores of 0 (irrelevant), 0.5 (partially relevant but insufficient), or 1 (highly relevant and useful). Thus, **$\hat{\rho}$ measures the overall quality of evidence available within a specific trajectory prefix**, including complementary evidence and missing information. The original $\hat{\rho}(q)$ notation suppresses its dependence on $C_{<t}$ and $u_t$; we will clarify this in the revision. **2. Reliability and Cross-Rollout Stability** **Cross-Judge Consistency.** On **35,583 training samples**, retrieval-quality scores from GPT-4o, DeepSeek-V3.2, and Qwen2.5-72B-Instruct exhibit pairwise Pearson correlations of **0.87–0.91**. This supports score-level consistency, although correlations alone cannot establish quadrant-label agreement. **Threshold Sensitivity.** We vary $\tau_\mu$ and $\tau_\rho$ while keeping other settings fixed. The default thresholds ($\tau_\mu=0.6$, $\tau_\rho=0.5$) achieve the best overall performance among the tested configurations. We will include these results in the revision. **Cross-Rollout Stability.** We randomly sample **2,000 queries from the natural training distribution** and generate **five independent rollouts per query** using pre-training Qwen2.5-3B, with the model, retriever, knowledge base, judge, and thresholds fixed. As shown in **Figure R2**, pairwise quadrant agreement reaches **89.32%** with **Fleiss' $\kappa=0.776$**, indicating substantial consistency beyond chance. Instability is concentrated near the retrieval-quality threshold: queries closer to the threshold exhibit lower agreement and more frequent Q3/non-Q3 transitions, while those farther away are more stable. This directly measures sensitivity to stochastic search trajectories rather than inferring label stability from cross-judge correlations. **3. Knowledge-Boundary Stability During Training** We agree that knowledge boundaries may evolve as training changes parametric knowledge, search behavior, and evidence utilization. **Initial quadrant labels provide supervision based on the initial model state; they are not assumed to remain permanently valid.** We therefore re-estimate quadrants at initial, intermediate, and final checkpoints using the same queries, retrieval setup, and classification protocol. As shown in **Figure R3**, agreement with initial labels is **91.75%** when rerunning the initial model and **88.82%** at the final checkpoint. Thus, most initial labels remain consistent, although some drift occurs. In particular, Q3-to-non-Q3 transitions increase during training, suggesting that some initially assigned Refusal labels may become outdated. However, the initial-model rerun already exhibits variability, so not all changes can be attributed to training. Cross-rollout and cross-checkpoint agreement also use different comparison protocols and cannot fully isolate training-induced drift from trajectory randomness. These findings support the **empirical applicability of static labels under our training setting**, while highlighting the potential value of periodic re-estimation for changing boundary cases. **In summary,** KbSD's retrieval-quality estimation is conditioned on observed multi-turn evidence. Cross-judge correlations (**0.87–0.91**), cross-rollout agreement (**89.32%, $\kappa=0.776$**), and final-checkpoint label retention (**88.82%**) provide complementary evidence of reliability and stability, while revealing uncertainty near thresholds and some training-stage boundary shifts. We will clarify these limitations in the revision.
AuthorsOpenReview ↗

Rebuttal for Q1

Q1: Whether the Privileged Teacher Improves Intermediate Search Decisions

We thank the reviewer for raising this important question. We agree that two issues should be distinguished: whether distillation supervision covers intermediate search processes, and whether such supervision actually improves the quality of intermediate decisions.

1. KbSD does not restrict distillation to final answers.

As shown in Eq. (3), the teacher provides conditional token-level supervision over the student's executed search histories and tool observations, rather than independently performing searches or merely generating final answers. Supervision covers valid generation positions throughout the trajectory, providing dense feedback for evidence evaluation, knowledge-source selection, and subsequent behavioral decisions, rather than only final-answer generation.

During both training and inference, the student does not receive the teacher's privileged hints but instead learns its conditional behavior through distillation. This differs from methods that imitate only final answers. Nevertheless, the teacher may still indirectly transfer answer-related information through such supervision. Therefore, covering intermediate generation positions alone does not establish that intermediate decision-making has been independently improved.

2. The Answer-Hint Ablation examines the contribution of direct reference-answer access.

Section 4.3 removes the teacher's direct reference-answer hint while retaining knowledge-boundary hints. Average Unreliability increases from 46.46% to 49.24%, a difference of 2.78 percentage points.

This result indicates that direct answer hints are beneficial, but removing them still preserves performance close to full KbSD. Thus, direct teacher access to the reference answer is not necessary to maintain most of the observed performance, and the full method's gains cannot be attributed solely to explicit access to the correct answer.

Importantly, this ablation retains knowledge-boundary signals and quadrant labels, some of which are computed using reference answers. It therefore isolates the incremental contribution of the direct answer hint rather than establishing that the entire supervision procedure is oracle-free. Moreover, Unreliability does not directly measure the correctness of retrieval choices or evidence evaluation.

3. We distinguish process-level supervision from demonstrated improvements in intermediate decisions.

The mechanism and ablation together support two conclusions: KbSD provides dense supervision at intermediate generation positions, and performance remains close to the full method even without direct answer hints. However, these results do not establish how much of the improvement comes from better retrieval decisions, evidence evaluation, refusal selection, or final-answer generation.

We will clarify this distinction in the revised manuscript and avoid equating the presence of process-level supervision with independently verified improvements in intermediate decision quality. More direct evidence would require separate process-level evaluations of search-query quality, evidence-relevance judgments, or source selection, which we identify as an important direction for future work.

Dear Reviewer Z79U,

We would like to express our gratitude for taking the time to review our paper and provide us with valuable comments. We understand that you have busy schedules and apologize for any inconvenience caused by our urging letter.

With only two days left in the discussion period, we hope to receive feedback from the reviewers: have our responses addressed your concerns, and is there anything we can further clarify or improve?

Thank you for your consideration, and we look forward to receiving your feedback soon.

Best regards,

Authors of the paper

Reviewer_ADkROpenReview ↗
Rating
2
Confidence
3
Informativeness
2
Presentation
2
Soundness
1
Read review
Paper summary

This paper proposes KbSD, a boundary-aware self-distillation framework for search agents. It estimates parametric knowledge from repeated retrieval-free samples and retrieval quality from an LLM-based relevance judgment, then assigns each query to one of four knowledge-boundary quadrants

Critical strengths
  • The four-quadrant formulation is a useful problem decomposition. Separating the reliability of parametric knowledge from the reliability of retrieved evidence exposes decisions that retrieval-trigger-only agents do not model explicitly.
  • The privileged same-parameter teacher is an interesting way to turn sparse trajectory outcomes into token-level supervision without requiring a separately trained teacher at inference time.
Critical weaknesses
  • The primary unreliability metric is degenerate with respect to refusal. Because refused outputs receive zero F1, U=1-(F1+R)=C-F1=C(1-Conditional F1), so an always-refuse model achieves the optimum U=0. Using Table 2, F1 improves by only 1.60 points while refusal rises by 15.12 points, meaning that roughly 90% of the reported improvement is attributable to increased refusal. On GAIA, F1 decreases substantially while the primary metric improves. This directly undermines the paper's central reliability claim.
  • The core distillation estimator is not defined consistently with the tool trajectory. Equation (3) says the teacher evaluates the student's observed prefix and tool outputs without searching independently, whereas Equation (4) samples a teacher-distributed sequence. Once that sequence diverges from the student trajectory, the corresponding tool observations are undefined.
  • The boundary supervision remains strongly oracle-dependent and the KL routing is not convincingly validated. The teacher receives the reference answer, while the purported no-answer variant still uses answer-derived boundary labels; retrieval quality is judged by an unspecified GPT-4o configuration using relevance rather than factual correctness
Questions for authors

See above

9 comments
AuthorsOpenReview ↗

Rebuttal for W1

W1: Unreliability, Over-Refusal, and GAIA Performance

We thank the reviewer for highlighting the limitations of Unreliability and the trade-off between answering and abstention. We agree that lower Unreliability alone does not establish better calibration. We clarify the metric, training objective, and supplementary evidence below.

1. Evaluation Metric and Training Objective.

Following KbPO, Unreliability is defined as U=1−(F1+R)U=1-(F1+R), where F1F1 is overall answer F1 and RR is the refusal rate. We acknowledge its limitation: an always-refusing model achieves U=0U=0, and replacing a correct answer with a refusal is not penalized.

To address this evaluation limitation, we additionally report Truthful Helpfulness Score (THS), following CRaFT [1] and GRAIT [2]. THS considers changes in correct and incorrect response rates relative to a reference model, rather than treating refusal as correctness. Using KbPO as the reference, KbSD achieves THS = +4.46, providing complementary evidence beyond Unreliability.

Importantly, Unreliability is an evaluation metric, not KbSD's training objective. Eq. (11) explicitly penalizes abstention in knowledge states where answering is expected:

\mathbb{I}[c^*(q_i)\in\mathcal{C}_{\mathrm{ans}}],$$ where $b(o_i)$ indicates refusal, $\phi>0$, and $\mathcal{C}_{\mathrm{ans}}=\{\mathrm{Integrated},\mathrm{External},\mathrm{Internal}\}$. Refusal in these quadrants incurs a penalty of $-\phi$, whereas refusal in Q3 does not. Together with outcome and process rewards, this encourages selective rather than indiscriminate abstention. We acknowledge that this constraint does not eliminate the metric's limitations. **2. Answer Quality and Selective Abstention.** As reported in Table 2, KbSD improves overall F1 from **20.50 to 22.10** (+1.60 points), while refusal increases from **15.18% to 30.30%**. Since refused responses receive zero F1 and all queries are included, this gain is not an artifact of excluding abstained queries. The additional **positive THS (+4.46)** further supports an improved balance between correct and incorrect responses. Appendix G.1 reports higher Conditional F1 on **8/9 benchmarks**. At matched answer coverage (Appendix G.2), KbSD also outperforms consistency-ranked KbPO on **8/9 benchmarks**. However, matched coverage does not imply identical answered queries and cannot fully separate reasoning improvements from better answer selection. We further conduct a paired response-transition analysis. On Qwen2.5-3B/7B, KbSD refuses **26%/26%** of queries previously answered incorrectly by KbPO, compared with only **7%/6%** of previously correct queries. Thus, refusal is approximately **3.7/4.3 times** more frequent on baseline-incorrect than baseline-correct responses. Moreover, some previously incorrect or refused queries become correctly answered under KbSD. For 3B, these gains exceed losses from previously correct responses, yielding a net increase in correct answers. Together, the F1 improvement, positive THS, coverage analyses, and paired transitions suggest that KbSD does not merely increase refusal uniformly. Nevertheless, baseline-incorrect queries are not necessarily unanswerable, so these transitions do not directly establish improved correct-refusal rates. **3. The GAIA Exception.** We acknowledge the reviewer's concern regarding GAIA. On Qwen2.5-3B, F1 decreases from **3.32 to 0.91**, while refusal increases from **19.48% to 41.02%** and Unreliability decreases from **77.20 to 58.07**. This reveals a substantial accuracy–abstention trade-off on challenging tasks. Although greater abstention may reduce unsupported answers, **lower Unreliability does not compensate for the observed F1 loss**. GAIA is also the exception where KbSD does not outperform KbPO in either Conditional F1 or matched-coverage comparisons. We therefore do not claim improved answering capability on GAIA and will explicitly discuss this limitation. **In summary,** we acknowledge that Unreliability alone is insufficient. However, the explicit refusal penalty, improved overall F1, **positive THS (+4.46)**, coverage analyses, and paired response transitions provide complementary evidence that KbSD's improvements are not solely attributable to indiscriminate refusal. We will clarify these trade-offs and the GAIA limitation in the revision. **References** > [1] *Zhu et al. (2025). *Utilize the Flow Before Stepping into the Same River Twice: Certainty Represented Knowledge Flow for Refusal-Aware Instruction Tuning.* AAAI 2025. https://ojs.aaai.org/index.php/AAAI/article/view/34812* > [2] *Zhu et al. (2025). *GRAIT: Gradient-Driven Refusal-Aware Instruction Tuning for Effective Hallucination Mitigation.* Findings of NAACL 2025. https://aclanthology.org/2025.findings-naacl.223/*
AuthorsOpenReview ↗

Rebuttal for W2

W2: Consistency Between the Distillation Objective and Tool Trajectories

We thank the reviewer for raising this important issue. We agree that Eqs. (3)–(4) do not sufficiently clarify the scope of teacher sampling or its relationship with tool observations. If Eq. (4) is interpreted as sampling an independent full teacher trajectory, the student's original tool responses cannot be reused once the teacher's tool calls diverge from those of the student. This ambiguity in the original formulation needs to be corrected. We clarify our trajectory-conditioned distillation procedure below.

1. The Teacher Conditions on the Student's Actual Tool History

KbSD employs trajectory-conditioned self-distillation based on student rollouts. The student first executes an agentic search rollout involving tool calls, producing an actual interaction history and its corresponding tool observations.

The teacher then conditions on this observed history, as defined in Eq. (3):

πT(⋅∣q,Ht)=sg⁡[πθ(⋅∣pT(q),Ht)],\pi_T(\cdot\mid q,H_t)=\operatorname{sg}\left[\pi_\theta(\cdot\mid p_T(q),H_t)\right],

where HtH_t contains the student's actual interaction history and tool observations, while pT(q)p_T(q) includes privileged hints beyond the original query. The teacher neither executes additional searches independently nor constructs hypothetical responses for tool calls that were never executed.

Thus, the student's executed trajectory provides the tool context for distillation, while the teacher supplies additional distributional supervision conditioned on that context.

2. Forward-KL Sampling Scope and Loss Masking

Teacher-distributed sampling in Eq. (4) should not be interpreted as generating an unconstrained full teacher rollout that may diverge from the student's tool trajectory while continuing to reuse its tool responses.

Instead, forward distillation performs local sampling within valid tool contexts established by the executed student trajectory. At positions used for distillation, the teacher and student distributions must share the same valid prefix and observed tool responses; the difference is that the teacher additionally receives privileged hints.

Crucially, the sampled continuation must remain consistent with the available tool observations; masking tool-observation tokens alone is insufficient. If local sampling generates a different tool call requiring a different tool response, the student's original response cannot be appended to that branch. Subsequent positions without corresponding executed tool observations are excluded from the valid distillation region.

Accordingly, tool-observation tokens themselves are excluded from the distillation loss, as are subsequent positions rendered invalid by divergence from the observed tool context. Loss masking defines which positions receive valid supervision; it does not make mismatched tool calls and observations consistent.

Therefore, the expectation in Eq. (4) should be restricted to trajectory-conditioned distillation over locally sampled continuations within valid tool contexts supplied by the student, rather than unconstrained sampling of complete teacher trajectories with independent tool histories.

3. Relationship Between Eqs. (3) and (4) and Planned Revision

Eq. (3) defines the teacher's conditional distribution over student-observed histories. Forward distillation uses this distribution to construct supervision within valid local sampling regions. Both rely on the same executed tool observations; no second, independently executed teacher search trajectory is involved.

However, the original Eq. (4) uses a simplified expectation over complete teacher sequences without explicitly specifying these conditioning constraints, sampling boundaries, or loss masks. This can misleadingly suggest an expectation over independent full teacher trajectories. We will revise Eq. (4) to explicitly formulate the trajectory-conditioned local distillation objective, rather than addressing this issue through textual clarification alone. We will also specify the starting prefixes, tool-interaction boundaries, and scope of loss masking to ensure consistency between the mathematical objective and the executed tool trajectories.

In summary, the reviewer's concern about undefined tool observations is valid under the interpretation of independently sampled teacher trajectories. KbSD instead employs supervision conditioned on executed student trajectories. Our revision will make the sampling and estimation scope explicit, resolving the inconsistency introduced by the original full-sequence formulation.

AuthorsOpenReview ↗

Rebuttal for W3

W3: Oracle Dependence in Knowledge-Boundary Supervision and the Effectiveness of Adaptive KL Routing

We clarify three aspects: privileged supervision during training, the reliability of knowledge-boundary estimation, and the effectiveness of adaptive KL routing.

1. Oracle Dependence and Answer-Hint Ablation.

First, we clarify that KbSD uses reference answers during training to construct knowledge-boundary supervision signals. These constitute privileged training information rather than inputs required at inference time.

Specifically, the teacher receives privileged hints containing knowledge-boundary signals, quadrant labels, and reference answers during training, whereas the student does not require these signals at inference time.

To examine whether the gains primarily arise from the teacher's direct access to reference answers, we conduct an Answer-Hint Ablation in the original paper. Removing the reference-answer hint increases average Unreliability from 46.46% to 49.24%, while still outperforming KbPO without distillation (61.92%).

This indicates that KbSD's gains cannot be attributed entirely to the teacher's direct access to reference answers.

Importantly, this ablation removes only the direct answer hint while retaining answer-derived knowledge-boundary labels. We therefore do not interpret it as an entirely oracle-free setting. Instead, it distinguishes the contribution of direct answer hints from that of knowledge-boundary-conditioned supervision.

2. Additional Validation of Knowledge-Boundary Estimation Reliability.

To address concerns about the reliability of GPT-4o-based retrieval-quality assessment, we conduct additional cross-LLM-judge consistency and knowledge-boundary threshold-sensitivity analyses.

Cross-Judge Consistency. We evaluate retrieval quality on 35,583 training samples using DeepSeek-V3.2, GPT-4o, and Qwen2.5-72B-Instruct, and calculate pairwise Pearson correlations.

The correlations are 0.89 between DeepSeek-V3.2 and GPT-4o, 0.87 between DeepSeek-V3.2 and Qwen2.5-72B-Instruct, and 0.91 between GPT-4o and Qwen2.5-72B-Instruct. These results indicate that retrieval-quality scores are highly correlated across different judges rather than depending entirely on a particular evaluation model.

Threshold Sensitivity. We further examine the sensitivity of the parametric-knowledge threshold τμ\tau_\mu and retrieval-quality threshold τρ\tau_\rho. We vary each threshold while keeping other training and evaluation settings fixed.

The original configuration, τμ=0.6\tau_\mu=0.6 and τρ=0.5\tau_\rho=0.5, achieves the best overall performance among the tested candidates, providing empirical support for the default threshold selection.

We also acknowledge the reviewer's important distinction: retrieval relevance is not equivalent to factual correctness. Our retrieval-quality scores primarily assess the relevance and usefulness of retrieved evidence for answering a query, rather than independently verifying the factual accuracy of each passage. We will clarify the scoring criteria, GPT-4o evaluation configuration, and this limitation in the revised manuscript.

3. Effectiveness of Quadrant-Adaptive KL Routing.

KbSD not only uses knowledge-boundary signals for supervision but also adapts the direction of distillation to different knowledge states. We validate this design through KL ablations and additional multi-seed experiments.

As shown in Table 5, Adaptive KL outperforms Fixed-FKL on 4 of 5 benchmarks, reducing average Unreliability from 49.13 to 46.46 (2.67 percentage points).

To further assess reproducibility, we conduct experiments with five independent random seeds. Adaptive KL achieves an average Unreliability of 46.46 ± 0.59, compared with 49.17 ± 0.88 for Fixed-FKL, yielding an average improvement of 2.71 percentage points.

These results demonstrate that the advantage of quadrant-adaptive KL over fixed Forward KL is reproducible across independent training runs, providing further support for adapting the distillation direction to different knowledge states.

Importantly, cross-judge consistency and threshold sensitivity assess the reliability of knowledge-boundary estimation, whereas the KL ablations evaluate the incremental benefit of adaptive distillation given these boundary signals. These analyses therefore provide complementary evidence for KbSD's design.

In summary, KbSD uses reference answers to construct knowledge-boundary supervision during training but does not require oracle information at inference time. The Answer-Hint Ablation shows that the gains are not solely due to direct answer access. Cross-judge consistency and threshold-sensitivity analyses further support the reliability of boundary estimation, while multi-seed experiments provide additional evidence for the effectiveness of quadrant-adaptive KL.

Reviewer_oo2XOpenReview ↗
Rating
3
Confidence
2
Informativeness
2
Presentation
3
Soundness
3
Read review
Paper summary

This paper proposes KbSD, a self-distillation method for training LLM search agents to calibrate between parametric memory and retrieved evidence. Building on a four-quadrant taxonomy, it obtains dense token-level supervision from a same-model teacher that is prompted with privileged hints, including the quadrant label and gold answer, and scores the student’s rollouts. The distillation direction is chosen per quadrant and the loss is combined with GRPO. On Qwen2.5-3B/7B across nine QA benchmarks, KbSD reports lower unreliability than baselines.

Critical strengths
  1. The paper addresses an important follow-up question of KbPO, which is a good addition to the community, and the methodologies proposed (the same-model teacher signal) to address that is well-designed.
  2. While the scale is somewhat limited, the experiment results are prominent.
  3. The paper is well-written and easy to follow. I enjoyed reading this manuscript.
Critical weaknesses
  1. Though I liked the idea of this paper, and I understand that access to high-end GPUs may not be available for many groups, the experiment scope is quite shallow (e.g., with only Qwen2.5 at 3B and 7B, one retriever, and a 2018 Wikipedia dump), which makes the work somewhat preliminary considering that ICLR is a highly prestigious venue. Especially for this kind of topic, demonstrating generalizability across wide range of tasks and different series/sizes of models is critical towards real-world impact. For this reason I cannot recommend clear acceptance despite I liked most parts of the paper.
  2. The headline metric rewards refusal as much as correctness, and most of the headline gain is refusal. So does it mean that a model that refuses every query scores zero?
Questions for authors

See above.

5 comments
AuthorsOpenReview ↗

Rebuttal for W1

W1: Experimental Coverage and Cross-Backbone Generalization

We thank the reviewer for recognizing the motivation of our work and for the constructive suggestions regarding experimental coverage. We agree that task diversity, model families and scales, and retrieval environments represent distinct dimensions of generalization and cannot substitute for one another. We address these aspects below.

1. Task Coverage in Existing Experiments

Although the original experiments primarily use Qwen2.5-3B and Qwen2.5-7B, our evaluation is not restricted to training-domain tasks or a single type of question answering.

We evaluate KbSD on 9 benchmarks, including HotpotQA and 2WikiMultiHopQA as in-domain tasks. The remaining 7 benchmarks are unseen during training, covering diverse characteristics such as multi-hop reasoning, open-domain question answering, long-tail knowledge, and complex information integration.

As shown in Table 1, under the reported configurations, KbSD achieves improvements on all seven out-of-domain benchmarks at both model scales. Thus, our results provide evidence of generalization across QA benchmarks and knowledge distributions, beyond adaptation to the training domains.

Nevertheless, cross-benchmark evaluation cannot replace validation across model families. We therefore extend our experiments to another backbone.

2. Additional Cross-Backbone Generalization Experiment

We introduce Gemma 3-4B to examine whether KbSD's effectiveness is specific to the Qwen2.5 family or its knowledge distribution. As a different model family, Gemma 3 provides an additional setting for evaluating cross-backbone generalization.

We compare KbPO and KbSD on Gemma 3-4B, as summarized below.

Table R4: Cross-Backbone Evaluation on Gemma 3-4B

MethodF1 (%, ↑)Refusal Rate (%)Unreliability (%, ↓)
KbPO19.6017.8062.60
KbSD21.1033.9045.00

As shown in Table R4, KbSD reduces Unreliability from 62.60 to 45.00, an improvement of 17.60 percentage points. Meanwhile, overall F1 increases from 19.60 to 21.10 (+1.50 points), while the refusal rate increases from 17.80% to 33.90%.

This trend is consistent with Qwen2.5-3B: on both model families, KbSD achieves higher overall F1 alongside higher refusal rates. Specifically, F1 improves by 1.60 points on Qwen2.5-3B and 1.50 points on Gemma 3-4B. Thus, the cross-backbone evidence is supported not only by reductions in Unreliability but also by improvements in answer F1 over the full evaluation set.

Despite differences in parametric knowledge, architecture, and baseline answering behavior, KbSD exhibits a consistent improvement trend. This supports the applicability of our method beyond Qwen2.5 without assuming identical knowledge boundaries across backbones.

3. Retriever and Knowledge-Base Settings

We also agree with the reviewer's concern regarding retrieval-environment diversity. The original experiments use a fixed retriever and a 2018 Wikipedia snapshot to control external knowledge sources and retrieval conditions, reducing potential confounding factors in method comparisons.

However, a controlled retrieval environment does not establish generalization across retrieval environments. Our current evidence supports cross-task and cross-backbone applicability under the evaluated retrieval setup, but does not establish equivalent benefits with other retrieval architectures, updated knowledge bases, or dynamic web environments.

We will clarify these limitations in the revised manuscript and incorporate the new cross-backbone results to ensure that our claims remain aligned with the empirical evidence.

In summary, the original evaluation covers two model scales and nine QA benchmarks, including seven out-of-domain test sets. The additional Gemma 3-4B experiment extends model-family coverage and demonstrates improvement trends consistent with Qwen2.5 in both overall F1 and Unreliability. Together, these results strengthen the empirical support for KbSD's cross-task and cross-backbone applicability, while we explicitly acknowledge the limitations of the current retriever and knowledge-base settings.

AuthorsOpenReview ↗

Rebuttal for W2

W2: The Unreliability Metric and Over-Refusal

We thank the reviewer for raising this important concern.

Mathematically, the reviewer is correct: if a model refuses every query, then F1=0F1=0 and the refusal rate R=1R=1, yielding U=1−(F1+R)=0U=1-(F1+R)=0. This demonstrates that Unreliability alone cannot distinguish appropriate abstention from over-refusal. We also acknowledge that the reduction in Unreliability in our main results is primarily attributable to increased refusal rates rather than a comparable improvement in answer accuracy.

However, it is important to distinguish the evaluation metric from the training objective. Unreliability is an evaluation metric, not the objective directly optimized by KbSD. Rather than unconditionally encouraging abstention, KbSD incorporates an explicit refusal penalty into its knowledge-boundary-aware reinforcement learning objective. Specifically, Eq. (11) defines:

ripen=−ϕ b(oi) I[c∗(qi)∈Cans],r_i^{\mathrm{pen}}=-\phi\,b(o_i)\,\mathbb{I}\left[c^*(q_i)\in\mathcal{C}_{\mathrm{ans}}\right],

where b(oi)∈{0,1}b(o_i)\in\{0,1\} indicates whether the model abstains, c∗(qi)c^*(q_i) denotes the estimated knowledge quadrant, ϕ>0\phi>0 is the refusal penalty coefficient, and Cans={Integrated,External,Internal}\mathcal{C}_{\mathrm{ans}}=\{\mathrm{Integrated},\mathrm{External},\mathrm{Internal}\} contains the three knowledge states in which an answer is expected.

Accordingly, abstention in the Integrated, External, or Internal quadrant incurs an additional penalty of −ϕ-\phi, whereas no such penalty is imposed in the Refusal quadrant. This mechanism penalizes unnecessary abstention when internal knowledge or external evidence is estimated to be sufficient for answering. Together with outcome and process rewards, it is intended to encourage selective abstention rather than simply increasing the overall refusal probability. In other words, the fact that an always-refusing model achieves the optimal evaluation score does not imply that the training objective encourages universal refusal.

More importantly, our experimental results provide evidence of improved answer quality beyond changes in refusal rates. Compared with KbPO, KbSD improves overall F1 from 20.50 to 22.10. This metric is computed over all queries, with refused responses assigned zero F1; therefore, the improvement is not an artifact of excluding abstained queries from evaluation. Furthermore, both the Conditional F1 analysis in Appendix G and the matched-coverage comparison against consistency-ranked KbPO show that KbSD performs better on 8 of 9 benchmarks, supporting the conclusion that its benefits are not limited to answering fewer questions.

In summary, we acknowledge the limitations of Unreliability as an evaluation metric, and the refusal penalty in the training objective does not eliminate these limitations. We therefore interpret our results jointly through overall F1, refusal rates, and coverage-controlled analyses, rather than treating lower Unreliability alone as sufficient evidence of better calibration. KbSD aims to reduce unreliable answers while preserving useful answering capability, rather than improving the metric through indiscriminate abstention.

Dear Reviewer oo2X,

We would like to express our gratitude for taking the time to review our paper and provide us with valuable comments. We understand that you have busy schedules and apologize for any inconvenience caused by our urging letter.

With only two days left in the discussion period, we hope to receive feedback from the reviewers: have our responses addressed your concerns, and is there anything we can further clarify or improve?

Thank you for your consideration, and we look forward to receiving your feedback soon.

Best regards,

Authors of the paper

Reviewer_iwdfOpenReview ↗
Rating
2
Confidence
2
Informativeness
2
Presentation
3
Soundness
2
Read review
Paper summary

The paper proposes KbSD, which adds same-model privileged self-distillation to KbPO (Feng et al., ACL 2026) for knowledge-boundary-aware agentic search. Training queries are assigned to four quadrants (Integrated / External / Refusal / Internal) using parametric certainty and retrieval quality. A teacher that shares the student's parameters receives these signals, the quadrant label and the gold answer as hints, and provides token-level supervision to the hint-free student. The divergence is chosen per quadrant: reverse KL for Integrated, forward KL for Refusal, and an MGDA-weighted combination for External/Internal. The distillation loss is added to KbPO's GRPO objective. On nine QA benchmarks with Qwen2.5-3B/7B, KbSD lowers the average Unreliability Rate relative to KbPO by about 16.7 points.

Critical strengths
  1. Using privileged hints to turn coarse behavioral targets into token-level supervision is a reasonable idea, and it requires no separate teacher.
  2. Training uses only HotpotQA and 2Wiki, and the evaluation includes seven held-out benchmarks.
  3. The authors are candid about several limitations and provide supplementary analyses (conditional F1, matched coverage).
Critical weaknesses

W1. Baseline provenance and a retrieval-budget mismatch in the central comparison. The paper states that "all baselines are reimplemented and retuned under a unified evaluation protocol" with at most five retrieval iterations (§4.1, Reproducibility Statement), and KbSD itself uses five (Table 4). However, many numbers in Table 1, including the baselines and KbPO (e.g., Search-R1 72.22/69.32, KbPO 64.32/57.99 at 3B/7B), are identical to those reported in the KbPO paper. That paper states that its agent operates with "maximum 2 search iterations per query". The two descriptions cannot both hold for the same numbers. The "−L_distill" column of Table 5, described as an ablation that "reduces the objective exactly to KbPO", also reproduces KbPO's reported numbers. This matters because the paper's central claim — that the gap to KbPO "isolates the contribution of dense process-level supervision" because the methods are "otherwise matched" — depends on KbPO and KbSD being run under the same budget.

W2. The headline metric rewards refusal. Under Unreliability = 1 − (F1 + Refusal Rate), a refusal contributes the same as a perfectly correct answer, whether or not the question was answerable. Refusing every query therefore attains the optimal value of 0, and over-refusal is never penalized. Table 2 shows that the improvement over KbPO comes mostly from refusal: refusal rises from 15.18% to 30.30% (+15.1), whereas F1 rises only from 20.50 to 22.10 (+1.6). Together these account for the 16.7-point drop in unreliability. Case 5 in the appendix illustrates the problem: a correct answer is replaced by "I don't know" at no cost under this metric. The matched-coverage analysis (Appendix G) helps, but it compares against a consistency-ranked KbPO only, and the authors acknowledge that it cannot separate better reasoning from better selection.

W3. The quadrant-adaptive objective is weakly supported. In Table 5, a single fixed forward KL for all quadrants (49.13) is only 2.7 points worse than the adaptive version (46.46). A fixed reverse KL (64.32) is worse than no distillation at all (61.92). The ablation covers five benchmarks with no variance reported, so the benefit of the paper's second main contribution is unclear.

Questions for authors
  1. Were the baseline and KbPO results in Table 1 re-run under the stated five-iteration protocol, or reused from the KbPO paper? Can you report KbPO and KbSD under exactly the same retrieval budget?
  2. Can you report over-refusal on answerable questions, or a metric under which refusal scores below a correct answer (e.g., risk–coverage curves)?
  3. Can you report mean and variance over multiple seeds for Tables 1 and 5?
  4. Knowledge-boundary quadrants are model-specific. How would the quadrant distribution and refusal behavior change with a more recent backbone than Qwen2.5?
7 comments
AuthorsOpenReview ↗

Rebuttal for W1&Q1

W1 & Q1: Baseline Sources and Fairness of the Retrieval Budget

We thank the reviewer for carefully checking our experimental setup and for raising this important issue regarding the fairness of the comparison. We acknowledge that the original manuscript described the baseline configurations inaccurately. We apologize for this and provide the following clarification and additional results.

1. Correcting the retrieval configuration and the interpretation of the original comparison. In Table 1, the maximum number of retrieval rounds was 2 for the relevant retrieval baselines and KbPO, but 5 for KbSD. The statement in the original manuscript that "all baselines were re-implemented and tuned under a unified five-round retrieval budget" is therefore inaccurate. We have corrected the configuration description in $\S$4.1 and Appendix B of the revised manuscript, and we will also revise the Reproducibility Statement and the relevant table captions to specify, item by item, the source of each result and its retrieval budget.

2. Additional controlled experiments under identical retrieval budgets. To examine whether the advantage of KbSD persists under the same retrieval budget, we ran additional comparisons with the maximum number of retrieval rounds set to 2 and 5, using the same retriever, knowledge base, and evaluation protocol.

Table R1: Unreliability Rate (%, lower is better, ↓\downarrow) under different retrieval budgets

ModelKbPO@2KbSD@2KbPO@5KbSD@5
Qwen2.5-3B64.3252.1854.9047.63
Qwen2.5-7B57.9945.3650.7341.15

Here, @2 and @5 denote the maximum number of retrieval rounds allowed per question, not the number of rounds actually executed.

  • KbSD outperforms KbPO under every identical budget. With a two-round budget, KbSD reduces the Unreliability Rate by 12.14 and 12.63 percentage points on the 3B and 7B models, respectively; with a five-round budget, the reductions are 7.27 and 9.58 percentage points. This shows that the advantage of KbSD remains once the retrieval cap is controlled.
  • KbSD with only two retrieval rounds still outperforms KbPO with five rounds, by 2.72 and 5.37 percentage points on 3B and 7B, respectively. The improvement of KbSD therefore cannot be explained by a larger retrieval budget alone. We also note that a larger retrieval budget helps both methods, so we no longer attribute the full gain in the original cross-budget comparison to self-distillation.

3. Clarifying the meaning of the ablation in Table 5. The reviewer's observation about Table 5 is also correct. The statement that "removing the distillation term reduces the objective to KbPO" describes a relationship at the level of the objective function; it does not mean that results obtained under different retrieval budgets constitute a strict single-variable ablation. In the original table, this column and the full KbSD differ in retrieval budget, so it cannot be used on its own to quantify the contribution of the distillation term. We will state clearly the source and budget of the results in this column, revise the statements in Table 5 and Appendix F that interpret it as a strict ablation, and use same-budget comparisons as the basis for assessing the gains of our method.

In summary, we accept and correct the inaccurate statements in the original manuscript regarding the experimental configuration and the attribution of results. The additional experiments support the following conclusion: under identical retrieval budgets, KbSD consistently reduces the Unreliability Rate, and its advantage does not come solely from additional retrieval opportunities.

We agree that, in the original table, the gap between KbPO and KbSD reflects differences in both method and budget, and therefore cannot be taken directly as evidence isolating the contribution of dense self-distillation. Accordingly, we will revise the statement in $\S$4.2 that "all other conditions being identical, this gap isolates the contribution of process-level supervision."

AuthorsOpenReview ↗

Rebuttal for Q2

Q2: Unreliability and Over-Refusal

We thank the reviewer. We agree that Unreliability alone cannot distinguish appropriate refusal from over-refusal, so we complement it with answer-quality metrics and per-question behavior analysis.

1. Metric. Following KbPO, U=1−(F1+R)U=1-(F1+R), with overall F1 and refusal rate RR on a [0,1][0,1] scale. As the reviewer notes, refusing everything gives U=0U=0, and Case 5 shows that turning a correct answer into a refusal is not penalized. A lower UU alone is therefore not sufficient evidence of better calibration, and we do not claim it as such.

2. Training objective. Unreliability is an evaluation metric, not what KbSD optimizes. Eq. (11) adds an explicit penalty:

ripen=−ϕ b(oi) I[c∗(qi)∈Cans]r_i^{\mathrm{pen}}=-\phi\,b(o_i)\,\mathbb{I}[c^*(q_i)\in\mathcal{C}_{\mathrm{ans}}]

where b(oi)=1b(o_i)=1 if the response is a refusal and Cans=[Integrated, External, Internal]\mathcal{C}_{\mathrm{ans}}=[{\text{Integrated, External, Internal}}] are the answerable quadrants. Unnecessary refusal is thus penalized, while refusal in the Refusal quadrant is not. This does not guarantee zero over-refusal, but "refusing everything optimizes the metric" is not the same as "the objective rewards refusing everything."

3. Joint evaluation. We agree with the decomposition of Table 2: refusal rate rises by 15.12 points and F1 by 1.60, so most of the Unreliability drop reflects more refusals. We therefore report:

MetricKbPOKbSD
Overall F1 (%)20.5022.10
Refusal Rate (%)15.1830.30
THS (KbPO as reference)–+4.46
Conditional F1 (App. G.1)–higher on 8/9 benchmarks
Matched-coverage F1 (App. G.2)–higher on 8/9 benchmarks
  • Overall F1 is computed over all questions with refusals counted as zero, so the gain does not come from shrinking the denominator.
  • Following CRaFT and GRAIT, we add the Truthful Helpfulness Score (THS), which jointly accounts for correct and incorrect responses relative to a reference model. With KbPO as reference, KbSD obtains +4.46+4.46, additional evidence of a better accuracy–hallucination trade-off beyond the refusal-sensitive Unreliability.
  • Conditional F1 (G.1) and matched-coverage F1 (G.2) are higher on 8/9 benchmarks. We accept that matching coverage is not matching the answered questions, so G.2 alone cannot separate better reasoning from better sample selection. Hence the paired analysis below.

4. Paired analysis. We compare KbPO and KbSD on identical questions. On Qwen2.5-3B/7B, KbSD refuses 26% of the questions KbPO answered incorrectly, versus 7%/6% of those KbPO answered correctly (denominators are the respective baseline-incorrect/correct sets), i.e., 3.7×/4.3× higher. The additional refusals are thus concentrated on baseline errors rather than spread uniformly. The "correct → refusal" cases such as Case 5 do exist, and this analysis quantifies them. Moreover, some questions that KbPO answered incorrectly or refused become correct under KbSD; on 3B, these gains exceed the losses, giving a net increase in correct answers.

This does not imply that KbPO lacks knowledge-boundary awareness; KbSD adds dense process-level supervision on top of it. Nor do we equate KbPO's errors with objectively unanswerable questions.

In summary, we agree that Unreliability should not justify refusals on its own. The evidence for KbSD does not rest on it: the objective penalizes unnecessary refusals, overall F1 and THS improve despite lower coverage, and the paired results show selective refusal alongside newly correct answers.

AuthorsOpenReview ↗

Rebuttal for W3&Q3

Q3: Effectiveness and Statistical Stability of the Quadrant-Adaptive Distillation Objective

We thank the reviewer for the insightful suggestion. We agree that, beyond average performance, we should verify stability across random seeds and clarify the independent contribution of the adaptive strategy relative to a fixed distillation direction.

1. Effectiveness of Quadrant-Adaptive KL

Quadrant-adaptive KL selects the distillation direction according to the behavioral requirements of each knowledge state. Beyond distillation vs. no distillation, we therefore ask: given dense supervision, does adaptively choosing the direction further improve performance?

Table 5 shows that Adaptive KL outperforms Fixed-FKL on four of five benchmarks, reducing average Unreliability from 49.13 to 46.46 (2.67 points), with gains of 4.85, 3.93, and 4.45 points on TriviaQA, HotpotQA, and PopQA. This comes from a direct comparison with the stronger Fixed-FKL, not merely the weaker Fixed-RKL.

Fixed-FKL is slightly better on 2WikiMQA, so we claim an overall advantage on the evaluated tasks, not a strict one on every task. Dense distillation provides process-level supervision, while the quadrant-adaptive strategy refines how it is applied per knowledge state. To test whether this extra gain is stable, we ran the following experiment.

2. Multi-Seed Stability

We repeated the main ablations of Table 5 on the same five benchmarks with 5 independent random seeds.

Table R3: Multi-seed ablation (Unreliability, %, lower is better)

MethodOriginal5 seeds (mean ± std)
No Distillation61.9261.96 ± 0.63
Fixed-RKL64.3264.38 ± 0.54
Fixed-FKL49.1349.17 ± 0.88
Adaptive KL (Ours)46.4646.46 ± 0.59

Cross-seed means differ from the original results by at most 0.06 points, and the method ranking is unchanged. Adaptive KL's average advantage over Fixed-FKL remains 2.71 points (original: 2.67), and its standard deviation (0.59) is lower than Fixed-FKL's (0.88). The advantage therefore does not rest on a single selected run, and these results supply the variance information missing from the original submission.

3. Rationale for Adaptive KL

KbSD uses Reverse KL in the Integrated quadrant (both sources reliable, concentrating on teacher-supported integration), Forward KL in the Refusal quadrant (neither reliable, preserving diverse evaluation and refusal paths), and a combination of both in the External/Internal quadrants (one source reliable). It is thus not a simple stacking of losses, but an adjustment of supervision direction based on source reliability.

Fixed-RKL is even worse than No Distillation (64.38 vs. 61.96 on average), so always using Reverse KL is not a reliable default; its uniform concentration may not suit states that require diverse plausible behaviors. We do not treat this degradation alone as proof that the quadrant assignment is correct. The key evidence is the additional gain over the stronger Fixed-FKL baseline, which holds across tasks and is preserved across seeds.

In summary, Adaptive KL outperforms Fixed-FKL on four of five benchmarks, and the five-seed experiments preserve this advantage (2.71 points) with small cross-seed fluctuation, supporting quadrant-adaptive KL as a key design component of KbSD.

From OpenReview, CC BY 4.0. Authors as signed there; shown unedited.

Related papers

Loading the map…

Discussion (0)

Sign in to comment.