- Rating
- 2
- Confidence
- 2
- Informativeness
- 2
- Presentation
- 2
- Soundness
- 2
Read review
Paper summary
The paper proposes Knowledge boundary Self-Distillation (KbSD), a framework for improving calibration in search agents. Building on the four knowledge-boundary quadrants, KbSD employs a privileged teacher that shares the student’s model parameters but has access to privileged information. The student is trained through quadrant-adaptive self-distillation combined with GRPO, using different KL depending on the state. Experiments across nine benchmarks show substantial reductions in the proposed unreliability metric.
Critical strengths
-
The proposed privileged self-distillation framework provides a reasonable approach to converting knowledge-boundary information into token-level training signals without requiring a larger, separately trained teacher.
-
The paper evaluates the method across multiple benchmarks and model scales, which demonstrates its effectiveness.
Critical weaknesses
-
The reported reliability improvements are heavily influenced by increased refusal rates. The primary unreliability metric is defined as 1 − (F1 + Refusal Rate), meaning that a higher refusal rate directly reduces the reported unreliability. As shown in Table 2, KbSD improves average F1 from 20.50 to 22.10 compared with KbPO, while the refusal rate increases substantially from 15.18% to 30.30%. It remains unclear to what extent the method improves actual answer quality rather than encouraging more conservative refusal behavior.
-
The reliability and stability of query-level knowledge-boundary classification are not sufficiently justified. How is query-level retrieval quality determined from multi-turn search trajectories? Is this estimation sufficiently robust and consistent across different rollouts? Moreover, since the knowledge quadrants are determined before training, could the assigned labels become outdated as the model's knowledge and retrieval capabilities evolve during training?
Questions for authors
Does the privileged teacher improve intermediate search decisions? Since the teacher has access to ground-truth answers, it is unclear whether distillation improves retrieval and evidence assessment or simply guides the student toward known answers.
4 comments
Rebuttal for W2
W2: Reliability and Stability of Knowledge-Boundary Estimation
We thank the reviewer for raising concerns about the reliability of query-level knowledge boundaries. We clarify how retrieval quality is estimated from multi-turn trajectories, its consistency across rollouts, and the stability of initial labels during training.
1. Query-Level Retrieval Quality
As described in Appendix B.1, KbSD evaluates the accumulated evidence from the student's executed search trajectory, rather than scoring individual passages or averaging retrieval-turn scores:
\begin{cases} \operatorname{clip}(J(q,C_{<t},u_t),0,1), & \text{valid score},\\ 0, & \text{otherwise}. \end{cases}$$ Here, $C_{<t}$ denotes observed retrieved content and $u_t$ the current summary. We concatenate retrieved evidence in observation order (up to 8,000 characters) and include the summary (up to 2,000 characters). GPT-4o (temperature = 0) assigns scores of 0 (irrelevant), 0.5 (partially relevant but insufficient), or 1 (highly relevant and useful). Thus, **$\hat{\rho}$ measures the overall quality of evidence available within a specific trajectory prefix**, including complementary evidence and missing information. The original $\hat{\rho}(q)$ notation suppresses its dependence on $C_{<t}$ and $u_t$; we will clarify this in the revision. **2. Reliability and Cross-Rollout Stability** **Cross-Judge Consistency.** On **35,583 training samples**, retrieval-quality scores from GPT-4o, DeepSeek-V3.2, and Qwen2.5-72B-Instruct exhibit pairwise Pearson correlations of **0.87–0.91**. This supports score-level consistency, although correlations alone cannot establish quadrant-label agreement. **Threshold Sensitivity.** We vary $\tau_\mu$ and $\tau_\rho$ while keeping other settings fixed. The default thresholds ($\tau_\mu=0.6$, $\tau_\rho=0.5$) achieve the best overall performance among the tested configurations. We will include these results in the revision. **Cross-Rollout Stability.** We randomly sample **2,000 queries from the natural training distribution** and generate **five independent rollouts per query** using pre-training Qwen2.5-3B, with the model, retriever, knowledge base, judge, and thresholds fixed. As shown in **Figure R2**, pairwise quadrant agreement reaches **89.32%** with **Fleiss' $\kappa=0.776$**, indicating substantial consistency beyond chance. Instability is concentrated near the retrieval-quality threshold: queries closer to the threshold exhibit lower agreement and more frequent Q3/non-Q3 transitions, while those farther away are more stable. This directly measures sensitivity to stochastic search trajectories rather than inferring label stability from cross-judge correlations. **3. Knowledge-Boundary Stability During Training** We agree that knowledge boundaries may evolve as training changes parametric knowledge, search behavior, and evidence utilization. **Initial quadrant labels provide supervision based on the initial model state; they are not assumed to remain permanently valid.** We therefore re-estimate quadrants at initial, intermediate, and final checkpoints using the same queries, retrieval setup, and classification protocol. As shown in **Figure R3**, agreement with initial labels is **91.75%** when rerunning the initial model and **88.82%** at the final checkpoint. Thus, most initial labels remain consistent, although some drift occurs. In particular, Q3-to-non-Q3 transitions increase during training, suggesting that some initially assigned Refusal labels may become outdated. However, the initial-model rerun already exhibits variability, so not all changes can be attributed to training. Cross-rollout and cross-checkpoint agreement also use different comparison protocols and cannot fully isolate training-induced drift from trajectory randomness. These findings support the **empirical applicability of static labels under our training setting**, while highlighting the potential value of periodic re-estimation for changing boundary cases. **In summary,** KbSD's retrieval-quality estimation is conditioned on observed multi-turn evidence. Cross-judge correlations (**0.87–0.91**), cross-rollout agreement (**89.32%, $\kappa=0.776$**), and final-checkpoint label retention (**88.82%**) provide complementary evidence of reliability and stability, while revealing uncertainty near thresholds and some training-stage boundary shifts. We will clarify these limitations in the revision.Rebuttal for Q1
Q1: Whether the Privileged Teacher Improves Intermediate Search Decisions
We thank the reviewer for raising this important question. We agree that two issues should be distinguished: whether distillation supervision covers intermediate search processes, and whether such supervision actually improves the quality of intermediate decisions.
1. KbSD does not restrict distillation to final answers.
As shown in Eq. (3), the teacher provides conditional token-level supervision over the student's executed search histories and tool observations, rather than independently performing searches or merely generating final answers. Supervision covers valid generation positions throughout the trajectory, providing dense feedback for evidence evaluation, knowledge-source selection, and subsequent behavioral decisions, rather than only final-answer generation.
During both training and inference, the student does not receive the teacher's privileged hints but instead learns its conditional behavior through distillation. This differs from methods that imitate only final answers. Nevertheless, the teacher may still indirectly transfer answer-related information through such supervision. Therefore, covering intermediate generation positions alone does not establish that intermediate decision-making has been independently improved.
2. The Answer-Hint Ablation examines the contribution of direct reference-answer access.
Section 4.3 removes the teacher's direct reference-answer hint while retaining knowledge-boundary hints. Average Unreliability increases from 46.46% to 49.24%, a difference of 2.78 percentage points.
This result indicates that direct answer hints are beneficial, but removing them still preserves performance close to full KbSD. Thus, direct teacher access to the reference answer is not necessary to maintain most of the observed performance, and the full method's gains cannot be attributed solely to explicit access to the correct answer.
Importantly, this ablation retains knowledge-boundary signals and quadrant labels, some of which are computed using reference answers. It therefore isolates the incremental contribution of the direct answer hint rather than establishing that the entire supervision procedure is oracle-free. Moreover, Unreliability does not directly measure the correctness of retrieval choices or evidence evaluation.
3. We distinguish process-level supervision from demonstrated improvements in intermediate decisions.
The mechanism and ablation together support two conclusions: KbSD provides dense supervision at intermediate generation positions, and performance remains close to the full method even without direct answer hints. However, these results do not establish how much of the improvement comes from better retrieval decisions, evidence evaluation, refusal selection, or final-answer generation.
We will clarify this distinction in the revised manuscript and avoid equating the presence of process-level supervision with independently verified improvements in intermediate decision quality. More direct evidence would require separate process-level evaluations of search-query quality, evidence-relevance judgments, or source selection, which we identify as an important direction for future work.
Dear Reviewer Z79U,
We would like to express our gratitude for taking the time to review our paper and provide us with valuable comments. We understand that you have busy schedules and apologize for any inconvenience caused by our urging letter.
With only two days left in the discussion period, we hope to receive feedback from the reviewers: have our responses addressed your concerns, and is there anything we can further clarify or improve?
Thank you for your consideration, and we look forward to receiving your feedback soon.
Best regards,
Authors of the paper
Rebuttal for W1
We thank the reviewer for analyzing Unreliability and refusal behavior. We agree that its reduction is largely driven by increased refusal rates and cannot alone distinguish better answering from conservative abstention. We clarify this through the training objective, complementary metrics, and paired behavioral analysis.
1. Training Objective.
Unreliability is an evaluation metric, not KbSD's training objective. Eq. (11) explicitly penalizes refusal in the Integrated, External, and Internal quadrants, where answering is expected. Thus, KbSD encourages decisions based on the reliability of internal knowledge and external evidence rather than indiscriminate refusal.
We additionally report Truthful Helpfulness Score (THS), introduced by CRaFT [1] and adopted by GRAIT [2]. THS considers changes in correct and incorrect response rates relative to a reference model, without treating refusal as correctness. Using KbPO as reference, KbSD achieves THS = +4.46, providing evidence beyond Unreliability.
2. Answer Quality Beyond Increased Refusal.
Table 2 shows that KbSD improves overall F1 from 20.50 to 22.10 (+1.60 points), while Refusal Rate increases from 15.18% to 30.30%. Since overall F1 includes all queries, with refusals assigned zero, this improvement is not due to excluding unanswered queries.
Appendix G.1 reports higher Conditional F1 on 8/9 benchmarks. At matched coverage (Appendix G.2), KbSD also outperforms consistency-ranked KbPO on 8/9 benchmarks, suggesting its gains are not solely attributable to reduced coverage. However, matched coverage does not guarantee identical answered queries and cannot fully separate reasoning gains from answer selection.
3. Query-Level Behavioral Transitions.
We further compare KbPO and KbSD on identical queries. On Qwen2.5-3B/7B, KbSD refuses 26%/26% of queries previously answered incorrectly by KbPO, versus 7%/6% of previously correct queries. Thus, refusal is 3.7/4.3 times more frequent on baseline-incorrect than baseline-correct responses.
Moreover, some queries previously answered incorrectly or refused become correctly answered under KbSD. For 3B, these gains exceed losses from previously correct responses becoming incorrect or refused, yielding a net increase in correct answers.
These results suggest that KbSD both avoids some baseline errors and recovers correct answers. However, baseline-incorrect queries are not necessarily unanswerable, so these transitions do not directly measure correct-refusal rates.
In summary, although increased refusal contributes substantially to lower Unreliability, the explicit refusal penalty, higher overall F1, positive THS (+4.46), coverage analyses, and paired transitions provide evidence beyond aggregate refusal rates. We will clarify this trade-off and avoid overstating conclusions based solely on Unreliability.
References