ECHO: Endogenous Citation via Hindsight-Oracle Self-Distillation for Factual Recall

Language models often fail to produce facts that are demonstrably encoded in their parameters: in closed-book question answering, access rather than acquisition is often the bottleneck. Reasoning helps by recalling related facts, but one hallucinated premise can derail the answer, and catching such premises in training has required external verifiers that are costly and least reliable on long-tail facts. We exploit two asymmetries: verifying a claim is easier than generating it, and hindsight requires no labels. Conditioned on self-verified claims pooled from its rollouts, the frozen model becomes a *hindsight oracle* that uses no gold answer, retrieval, or external judge, yet is markedly stronger than the student on questions whose answer it can verify but rarely produces. ECHO diagnoses knowledge states without gold answers, builds the oracle from a minimal set of recall-sufficient cues, and distills it into the student on-policy, weighting tokens by the teacher–student entropy gap, with an influence-gated grounding reward. Labels are confined to offline verifier calibration and hyperparameter selection. On Qwen3.5-27B, ECHO is non-inferior to gold-answer RLVR in accuracy (33.1% vs. 33.4% on SimpleQA-Verified) while raising trace factual precision by 6.4 points and FActScore by 4.1. Gains concentrate on tip-of-the-tongue questions, the measured encoding rate is unchanged, and interventions show reliance on recalled cues.