acceptodds
Under review as a conference paper at ICLR 2027

ECHO: Endogenous Citation via Hindsight-Oracle Self-Distillation for Factual Recall

Abstract

Language models often fail to produce facts that are demonstrably encoded in their parameters: in closed-book question answering, access rather than acquisition is often the bottleneck. Reasoning helps by recalling related facts, but one hallucinated premise can derail the answer, and catching such premises in training has required external verifiers that are costly and least reliable on long-tail facts. We exploit two asymmetries: verifying a claim is easier than generating it, and hindsight requires no labels. Conditioned on self-verified claims pooled from its rollouts, the frozen model becomes a *hindsight oracle* that uses no gold answer, retrieval, or external judge, yet is markedly stronger than the student on questions whose answer it can verify but rarely produces. ECHO diagnoses knowledge states without gold answers, builds the oracle from a minimal set of recall-sufficient cues, and distills it into the student on-policy, weighting tokens by the teacher–student entropy gap, with an influence-gated grounding reward. Labels are confined to offline verifier calibration and hyperparameter selection. On Qwen3.5-27B, ECHO is non-inferior to gold-answer RLVR in accuracy (33.1% vs. 33.4% on SimpleQA-Verified) while raising trace factual precision by 6.4 points and FActScore by 4.1. Gains concentrate on tip-of-the-tongue questions, the measured encoding rate is unchanged, and interventions show reliance on recalled cues.

Primary area:
foundation or frontier models, including LLMs
Keywords:
large language models, factual knowledge, self-distillation

Reviews

Among:
Shape:
1234this paper, Top 49%
Figure 1. Mean review rating of the reviewed papers at ICLR 2027. This paper averages 2.33 over 3 reviews, above 51% of the others.
Reviewer_orNWOpenReview ↗
Rating
2
Confidence
2
Informativeness
2
Presentation
2
Soundness
2
Read review
Paper summary

This paper proposes ECHO, a self-distillation framework for improving factual recall and reducing hallucinations in model reasoning without requiring gold answers or external verifiers during training. The main point is to exploit the asymmetry between factual generation and verification: a model may fail to recall a fact but can recognize it when presented. ECHO first identifies knowledge states using self-verification, then constructs a hindsight oracle by aggregating verified claims from multiple rollouts and selecting recall-sufficient cues. The student model is subsequently trained through entropy-guided on-policy distillation and an influence-gated grounding reward. Experiments across four language models show that ECHO achieves accuracy comparable to gold-supervised RLVR, while substantially improving reasoning-trace factual precision.

Critical strengths
  1. A meaningful self-supervised training framework that improves factual reliability without per-question gold answers. This is consequential because acquiring large-scale, high-quality factual supervision is particularly challenging for long-tail knowledge. Experimental results also show that ECHO improves factual reliability without introducing an additional inference-time module, which validates its practical relevance.
  2. A thorough empirical analysis and mechanistic evidence supporting the proposed explanation. Beyond standard benchmark comparisons, the authors conduct extensive ablation studies, knowledge-state analyses, and causal interventions to systematically validate the effectiveness of individual components and provide evidence that the improvements stem from better factual recall and increased reliance on verified cues.
Critical weaknesses
  1. Dependence on the Reliability of the Endogenous Verifier. The effectiveness of ECHO depends on the reliability of its endogenous verifier. When the model generates an incorrect claim that is mistakenly verified as correct, the resulting supervision may reinforce factual errors. As acknowledged in the limitations, ECHO may even amplify incorrect knowledge in such cases. Could the authors provide a simple statistical analysis, based on sampled cases, to quantify how frequently this issue occurs and how significantly it affects the training process?
  2. Limited Generalizability of the Triple-Based Fact Representation. ECHO represents factual claims as subject–relation–object triples, which may limit its applicability to more complex reasoning tasks. Many reasoning processes, such as multi-step mathematical reasoning, cannot be naturally expressed in this structured format. Could the authors discuss this limitation and clarify how the proposed framework might generalize to reasoning tasks that do not fit the triple-based representation?
  3. Limited synthesis across experimental findings. While the paper provides extensive empirical evaluations and useful interpretations of individual experiments, the discussion is often dense with quantitative comparisons across multiple metrics, ablations, and model settings. A more explicit synthesis across these findings would improve the clarity of the paper's overall conclusions.
Questions for authors

See Weaknesses.

4 comments

Thank you for recognizing the framework and mechanistic evidence. We address the three concerns directly.

W1. Verifier reliability and safeguards.

Thank you for highlighting the importance of this limitation. Section 6 explicitly acknowledges that confidently held errors can pass self-verification and be reinforced. Table 34 and its following analysis quantify the risk: wrong targets occur in 61 of 748 development TT questions and receive 7.6% of TT distillation weight. Across the TT pool, generation of these specific wrong targets increases by 1.5 percentage points. Such correlated errors are a structural risk of self-improvement or self-play driven by imperfect self-evaluation [1].

The paper introduces several measures to limit this risk, with corresponding evidence:

  • Stricter verification (Sec. 3.1). Raising the precision target to 0.95 reduces high-confidence wrong-teacher cases from 35 to 13, at a 31% reduction in TT coverage (Appendix J, analysis following Table 34).

  • Cue compression (Sec. 3.3). Retaining only recall-sufficient cues reduces contexts containing a false premise from 41.8% to 10.7% while preserving teacher accuracy (Sec. 4.3, Fig. 3c).

  • Training by knowledge state (Sec. 3.2, 3.4). NE questions are excluded from distillation. Removing this control increases hallucinated claims per NE trace from 1.21 to 1.62 (Sec. 4.4).

These results demonstrate risk reduction, while residual errors remain possible. We will consolidate this evidence and the coverage tradeoff in the revision.

W2. Scope and generalization beyond factual triples.

We agree that the current representation reflects the paper's scope: factual QA, where our goal is to improve recall of parametric knowledge. The factual claims studied here can be decomposed into subject–relation–object triples for verification. More general reasoning lies beyond this original scope.

To test transfer beyond this scope, we evaluated Qwen3.5-9B on five-hop PrOntoQA instances [2]: 10,000 training and 2,000 test questions with disjoint ontologies. The frozen verifier checks each conclusion against its premises and inference rule. Verified intermediate steps, excluding the queried statement and its negation, condition the hindsight teacher. Trained variants use matched budgets and three seeds. The verifier uses 500 separate labeled calibration steps; training answers are withheld, and symbolic checking is confined to calibration and evaluation.

Verified self-training fine-tunes on self-approved complete solutions without hindsight conditioning. The ablation removes intermediate-cue checks while retaining the other components.

MethodAnswer accuracy %Complete valid proof %
Base78.464.7
Verified self-training82.171.2
ECHO with premise-conditioned verification86.078.8
ECHO without intermediate-cue verification80.367.5

ECHO reaches 86.0% answer accuracy and 78.8% complete-proof validity, exceeding verified self-training by 3.9 and 7.6 points, respectively. Removing intermediate-cue verification reduces both. These results support transfer of ECHO's mechanism from factual recall to logical deduction by adapting the verification interface.

W3. Overall contributions.

Thank you for suggesting a clearer synthesis. We will highlight three connected contributions:

  1. Endogenous supervision from capability asymmetry. ECHO turns the gap between verification and generation into a hindsight teacher built from self-verified cues, without per-question gold answers or external feedback during training. Matched-label comparisons demonstrate the value of using limited supervision to enable this reusable learning signal (Sec. 3.1–3.3, 4.2).

  2. A validated framework for reliable knowledge recall. Knowledge-state diagnosis, recall-sufficient cues, entropy-routed distillation, and influence-gated grounding teach models to recover and use relevant parametric knowledge. Our study across four models links this mechanism to factuality gains, including accuracy non-inferior to gold-answer RLVR on 9B/27B and improved short-form and long-form factuality (Sec. 3.2–3.4, 4.2–4.6).

  3. Inference efficiency, complementarity, and transfer. As detailed in our response to Reviewer 24NV (W2), single-generation ECHO achieves higher factual precision than the evaluated inference-time scaling methods at substantially lower inference cost; combining it with CoVe further improves both accuracy and factual precision. Our response to W2 also shows that adapting the verifier and training on PrOntoQA transfers the mechanism beyond factual QA to logical reasoning, outperforming verified self-training in both answer accuracy and proof validity.

[1] Recursive Self-Improvement in AI: From Bounded Self-Refinement to Autonomous Research Loops. Chen et al. arXiv 2026

[2] Language Models Are Greedy Reasoners: A Systematic Formal Analysis of Chain-of-Thought. Saparov and He. ICLR 2023

Thank you for the detailed rebuttal and additional experiments, and I especially appreciate your efforts concerning the quantitative analysis of verifier errors (W1) and the new PrOntoQA experiments (W2).

My remaining concern is primarily about the presentation (W3). The original concern was not about the contributions themselves, but rather the dense presentation of experimental results and the lack of clear synthesis across different experiments. While the proposed summary of contributions is helpful, it does not fully address this issue. Considering this presentation issue, I will maintain my original score.

AuthorsOpenReview ↗

Improving the experimental narrative and synthesis

Thank you for your continued engagement and constructive feedback. We are pleased to receive your positive feedback on our responses to W1 and W2. We also appreciate your clarification of W3 and apologize that our previous response did not adequately address the organization and synthesis of the experimental findings.

We commit to completing a careful revision of the main-text experimental discussion before the rebuttal period ends. We will retain the key experiments, integrate the essential appendix ablations and analytical conclusions into the main narrative, and reduce repetitive descriptions of numerical results. We will explain how the experiments connect and what they jointly establish about ECHO’s effectiveness, mechanism, and limitations. These revisions will improve the organization and explanation of the evidence without changing the paper’s main conclusions.

We will share the revised text for your assessment. We sincerely hope these revisions will help the technical contributions and their supporting evidence come through more clearly, and would greatly appreciate your continued feedback.

AuthorsOpenReview ↗

Revised manuscript and clearer experimental synthesis

Thank you for your continued feedback. We apologize that the original presentation made our findings difficult to follow. As promised, we have revised the main text throughout. The attached PDF marks changes in blue, with three main improvements:

  1. Clearer contributions and method (Sec. 1–3). We clarify the learning principle, the role of the verifier and its features, and the supervision boundary between default ECHO and ECHO-LF.
  2. Connected experimental evidence (Sec. 4). We reduce repetitive reporting, bring key ablation findings from the appendix into the main discussion, and explain how the teacher controls, recall interventions, and error-mitigation analyses support the conclusions together.
  3. Explicit scope and supporting evaluations (Sec. 4.6, Sec. 6, Appendix O). We incorporate the additional inference-time scaling and PrOntoQA experiments, summarize their implications in the main text, and clarify applications and limitations. These revisions preserve the core method and original conclusions while making the evidence easier to assess. We hope they address the presentation concerns and would be grateful if you could consider the revised manuscript in your final assessment.
Reviewer_24NVOpenReview ↗
Rating
2
Confidence
2
Informativeness
2
Presentation
2
Soundness
2
Read review
Paper summary

This paper addresses recall failures, where an LLM has learned a fact but cannot reliably retrieve it. ECHO trains the model using a frozen copy of itself supplied with self-verified factual cues, improving recall without gold answers for training questions. It reports accuracy comparable to gold-answer reinforcement learning and more factual reasoning, although self-verification can reinforce existing misconceptions.

Critical strengths
  • The problem of knowledge encoding vs recall in LLMs is interesting and important.
  • Using verification-generation gap to get training signal without external label is a good direction.
Critical weaknesses
  • Meaning of higher trace factual precision (TFP) and FActScore (but not higher accuracy)
    • This work argues the strength of ECHO based on higher TFP and FActScore, while showing similar accuracy. However, this work does not dedicate much effort on explaining how TFP and FActScore can measure what accuracy cannot measure. Therefore, it is hard for me to interpret this result. Why should I put more weight on higher TFP and FActScore than accuracy? What does the difference between these metrics tell about the relative strength of ECHO over RLVR?
  • No justification for training-based approach, lack of comparison to test-time scaling methods
    • The significance of the problem -- knowledge encoding vs recall -- is understood. However, I do not understand this work jumping right into training to solve this problem. Why do authors view training, not test-time scaling, as more appropriate way to tackle this problem?
    • On a similar note, there lacks a comparison to test-time scaling methods that can enhance the factual recall capability of LLMs.
  • Minor note on Figure 1. Figure 1 does look like AI-generated. Even if is not, I advise authors to redraw Figure 1 with more care. Please note that this point has not affected my rating on this work.
Questions for authors
  • Why the name "ENDOGENOUS CITATION VIA HINDSIGHT-ORACLE"? What does "endogenous citation" means here?
6 comments

Thank you for the constructive questions. We clarify the contribution and the missing comparisons below.

W1. Why does higher TFP not lead to a clear accuracy gain?

In short-form factual QA, TFP measures the proportion of correct factual claims in the model's reasoning trace, whereas accuracy measures final-answer correctness. Fewer incorrect claims do not guarantee a correct answer: even one remaining critical error can still lead to failure. Their relationship may therefore be obscured when performance is averaged over all questions.

To examine this relationship, we compared Qwen3.5-27B before and after ECHO training on all 1,000 SimpleQA-Verified questions, using identical decoding settings and the same independent factual auditor. We grouped questions by the reduction in the proportion of incorrect factual claims.

Reduction in factual error rate (pp)Δ Accuracy (pp)
(0, 20]+3.2
(20, 40]+8.6
>40+15.3

Larger reductions in factual errors are associated with larger accuracy gains.

W2. Training versus test-time scaling.

We appreciate the suggestion to compare ECHO with test-time scaling, as both can improve factual reliability. Our motivation for training is to make improvements reusable across requests while keeping inference simple. ECHO requires 1.48 times RLVR's training accelerator time, but deployment uses one generation and no additional verifier.

To compare deployment tradeoffs, we evaluated Qwen3.5-27B on the same 1,000 SimpleQA-Verified questions, using the original decoding and independent TFP auditor. Self-Consistency [1] votes over eight candidates and returns the first trace with the winning answer; self-verifier Best-of-N [2] ranks eight by mean claim confidence from ECHO's frozen verifier; Self-Refine [3] uses two feedback-and-revision rounds; factored CoVe [4] uses one cycle with six independently answered verification questions. Each returns one final trace. Cost is accelerator-seconds per query on identical hardware and batching, including all prefills, discarded candidates, verification, and revision, excluding the common offline evaluation audit.

MethodAccuracy %TFP %Relative inference cost
Base30.081.41.00
RLVR33.482.31.00
Base + Self-Consistency 8 [1]32.282.08.0
Base + self-verifier Best-of-8 [2]33.087.88.7
Base + Self-Refine 2 rounds [3]31.884.93.8
Base + factored CoVe [4]32.687.04.1
ECHO33.188.71.05
ECHO + factored CoVe [4]34.090.34.3

Base cost 1.00 is the normalization reference. Best-of-8 adapts the verifier-selection baseline family in [2], rather than reproducing its mathematical PRM.

Single-generation ECHO achieves competitive accuracy and 88.7% TFP at 1.05 times Base's inference cost. Adding CoVe further raises accuracy/TFP to 34.0/90.3, supporting complementarity between training and test-time verification. The manuscript's external-audit filter reaches 90.2% TFP, but requires eight candidates and external verification.

Q1. Terminology.

“Endogenous citation” means grounding generated claims in knowledge the model can itself evoke, rather than providing external source citations. “Hindsight oracle” is the frozen model conditioned on verified cues pooled from its own rollouts, not a gold-answer oracle. Offline calibration and model-selection labels remain necessary in standard ECHO. We will explain these terms at first use.

W3. Figure 1.

We will redraw it with clearer vector elements, fewer labels, and explicit separation of training-only components from deployment.

[1] Self-Consistency Improves Chain of Thought Reasoning in Language Models. Wang et al. ICLR 2023

[2] Scaling LLM Test-Time Compute Optimally Can be More Effective than Scaling Parameters for Reasoning. Snell et al. ICLR 2025

[3] Self-Refine: Iterative Refinement with Self-Feedback. Madaan et al. NeurIPS 2023

[4] Chain-of-Verification Reduces Hallucination in Large Language Models. Dhuliawala et al. Findings of ACL 2024

Reviewer_24NVOpenReview ↗

Regarding W1

Thank you for your rebuttal.

Regarding W1, I believe there was some miscommunication, and I take the responsibility.

I was not curious whether higher TFP leads to higher accuracy, but rather interested in why I should look at TFP over accuracy. Accuracy and TFP are different metrics so I get it when those two slightly diverge. My concern was that accuracy alone do not support ECHO's advantage over baselines very much. In Table 1, avg. accuracy of ECHO + label reward is 40.7 / 27.2 for Qwen3.5-27B / Qwen3.5-9B, while RLVR (compute-matched) is 39.6 / 26.5. Respectively 1.1%p and 0.7%p I believe are hard to justify the additional complexity ECHO brings, let aside the question whether these results are stable (statistically significant) enough.

However TFP and FActScore provide much clearer win for ECHO, so I was curious why the reader should look at these two metrics rather than accuracy. If there isn't so much reason to do so, then I'd have to point out that accuracy gain is too weak to justify ECHO.

AuthorsOpenReview ↗

Clarifying ECHO’s Accuracy Gains and the Value of Improved Factuality

Thank you for clarifying. We would like to explain where ECHO’s accuracy improvements arise and why factual reliability is central to its contribution.

1. ECHO substantially improves the recall failures that motivate our method. ECHO aims to improve access to parametric knowledge, particularly for tip-of-the-tongue (TT) questions whose answers the model can self-verify but rarely generates. On SimpleQA-Verified, core ECHO achieves 38.0% accuracy versus RLVR’s 27.4% on TT questions, a +10.6 pp advantage (95% CI [5.0, 16.2]) for 27B (Table 32). Yet TT questions constitute only 10.6% of this benchmark. Their contribution to aggregate accuracy is therefore limited by their prevalence, and the overall comparison also reflects performance on the remaining knowledge states. This demonstrates a substantial improvement on the specific recall bottleneck ECHO was designed to address.

2. Factual reliability is a primary objective with direct practical value. In factual explanations and long-form responses, unsupported details remain a quality problem even when the requested answer is correct. TFP measures factual precision within the trace, while FActScore evaluates the final content. Crucially, our results show improvements in the content delivered to users. On 500 biographies, despite training only on short questions, 27B ECHO obtains the following results (Table 46):

Final biography outputRLVRECHO
FActScore72.476.5
Facts per response34.633.8
Unsupported facts per response9.57.9

This amounts to approximately 17% fewer unsupported statements with only 2% fewer statements overall. The FActScore gains over RLVR are +4.1 [3.1, 5.1] / +4.9 [3.9, 5.9] pp for 27B/9B (95% CIs; Table 27). In the comparison you cite, ECHO + label reward also improves FActScore over compute-matched RLVR by 4.5/5.3 pp (Table 1). Factuality metrics therefore capture a meaningful improvement in final content that short-answer accuracy alone does not measure.

We will foreground these findings in Sec. 4.2 and 4.5, connecting ECHO’s targeted recall improvements to its practical benefit for factual generation.

Reviewer_RPrbOpenReview ↗
Rating
3
Confidence
1
Informativeness
2
Presentation
3
Soundness
3
Read review
Paper summary

The paper exploring two very interesting observations: verifying a claim is easier than generating it, and hindsight requires no labels. It reports ECHO, a system diagnoses knowledge states without ground truth. The surprising findings are highly valuable.

Critical strengths

The paper asks an interesting and inspiring question: can a model learn to reason only over facts it can corroborate without gold answers or external verifiers. The answers from two asymmetries are interesting and inspiring.

The investigation methods are well described and sound right. The technical decisions are well justified.

Critical weaknesses

I would like to see more discussion about the downstream impact of the proposed techniques.

Questions for authors

As this is not my research area, I don't have many insights into the technical details. One question I have is that the discussion about the downstream impact of the propose techniques seems missing. It would be helpful to close the loop and discuss the implications and applications of ECHO.

2 comments
AuthorsOpenReview ↗

Methodological implications and downstream applications

Thank you for highlighting the need to clarify ECHO's broader impact. We see two main implications.

  1. Capability asymmetry as a source of self-supervision. ECHO turns an internal capability asymmetry into supervision: stronger verification constructs a hindsight teacher, and distillation transfers that advantage to generation, with labels confined to offline calibration and model selection. A central challenge in self-play [2] and RSI [1] is obtaining reliable learning signals from self-generated experience. ECHO's broader insight is that these signals can arise from differences between a model's own capabilities: what it can verify can help teach what it cannot reliably generate. This principle could extend beyond factual recall to settings with a usable verification advantage, providing a potential building block for self-supervised improvement and future RSI systems.

  2. More reliable knowledge use in chatbots and agents. Better parametric recall can make chatbot explanations more factual and reduce errors in premises passed between agent steps. On Qwen3.5-27B, ECHO reduces false claims per trace from 1.06 to 0.67 and improves biography FActScore by 4.1 points over RLVR, despite training only on short questions. It could also complement retrieval: external evidence supplies missing or updated information, while parametric knowledge supports query formulation, evidence interpretation, and connections between facts. This suggests applications in knowledge-intensive chatbots and research agents. ECHO requires no additional verifier at deployment. We will expand Section 6 (Conclusion) to discuss these applications and potential integration with retrieval.

[1] Recursive Self-Improvement in AI: From Bounded Self-Refinement to Autonomous Research Loops. Chen et al. arXiv 2026

[2] Absolute Zero: Reinforced Self-play Reasoning with Zero Data. Zhao et al. NeurIPS 2025

I wanted to thank the authors for their efforts. The response partially addresses my concerns. As I have low confidence on the topic, I would like to follow the other reviewer and the area chair's judgements.

From OpenReview, CC BY 4.0. Authors as signed there; shown unedited.

Related papers

Loading the map…

Discussion (0)

Sign in to comment.