- Rating
- 3
- Confidence
- 2
- Informativeness
- 2
- Presentation
- 3
- Soundness
- 2
Read review
Paper summary
The paper presents a benchmark, a fine-tuning method, and an analysis of the problems smaller LLMs run into when they learn new knowledge. The study uses four models and two fictional company databases, each made up of several documents (four databases in the scaling experiments). The benchmark measures four things. The first is accuracy on the new facts. The second is how free of hallucination the learning was, tested with questions whose answers cannot be known. The third is how accuracy on old knowledge compares. The fourth is whether answers to old-knowledge questions stay unrelated to the fictional companies, where a high rate means little knowledge spills over. The authors' hallucination-aware training improves on all of these metrics. It combines context distillation with hallucination-probing questions and old-knowledge regularisation, which makes the models more reliable at the boundaries of what they know.
Critical strengths
The paper is well written. The figures, tables and sections all support the argument and each one is needed. The method does more than reduce hallucinations. It focuses on the boundary of knowledge ingestion in LLMs. Rather than only asking whether new knowledge can be retrieved, the evaluation covers several angles: old-knowledge accuracy, transfer across documents, behaviour on old knowledge, and compositional accuracy. The paper also gives its prompts, including those for the LLM-judge evaluations, in a thorough appendix. I also appreciate that the authors go past their registered metrics and compare the grounding of the teacher.
Critical weaknesses
My main concern is the number of questions and repetitions. The main experiment has 20 questions for each of the three types, 10 per company across 2 companies, so each question is worth 0.05 accuracy. The paper reports no training seeds, no repeated runs and no measure of spread. With accuracy this coarse, the differences between conditions are hard to interpret. For example, the gap between KL and SFT is small compared with the weight of a single question. A related point: it is not clear to me how much the training probes and the test probes overlap. The model might simply have learned to decline questions of a familiar shape.
The related work section lists many methods and compares them with the proposed approach in conceptual terms, but none of them is evaluated against it on the paper's metrics. The main baseline the authors introduce is a variant of their own distillation.
Even with the extensive appendix, several prompts are missing: the prompts for fact extraction, question generation, and unanswerable and false-premise generation, the teacher's instructions, and the recall-first instruction that separates Grounded+ from Grounded−.
Please also check these smaller issues:
- Section 5.4 says "all three models" twice (L383, L393–394), but Fig. 2 shows four.
- The Gemma4-31B row of Table 3 adds up to 35,738, but the table gives 45,738 (L680).
Questions for authors
- How many training seeds or repeated runs are behind the reported numbers, and how much do the results vary between them? With 20 questions per type, each question is worth 0.05 accuracy, so it would help to know whether differences such as the KL vs SFT gap are larger than this spread.
- How much do the training probes and the test probes overlap? Could the model have learned to decline questions of a familiar shape instead of recognising the limits of its knowledge? Results on test probes whose form differs from the training probes would help answer this.
- Why are the methods discussed in the related work not evaluated as baselines on your metrics? Apart from the variant of your own distillation, is there an external method you could compare against?
- Could you add the missing prompts to the appendix: fact extraction, question generation, unanswerable and false-premise generation, the teacher's instructions, and the recall-first instruction that separates Grounded+ from Grounded−?
- Section 5.4 says "all three models" (L383, L393–394), but Fig. 2 shows four. Which is correct?
- The Gemma4-31B row of Table 3 adds up to 35,738, but the table reports 45,738 (L680). Is this a typo, or is a value missing from the row? Or did I mix up some numbers?
2 comments
Q3. Comparison with external baselines
Why are the methods discussed in Related Work not evaluated as baselines?
The methods discussed in our Related Work address different aspects of knowledge acquisition:
- Knowledge-editing methods typically modify isolated factual associations rather than acquire knowledge from documents.
- R-Tuning and factual-QA adaptation methods (Gekhman et al., 2024; Dang et al., 2025; Kaplan et al., 2026) primarily study uncertainty, hallucinations, or interference with existing knowledge during fine-tuning, without demonstrating the use of newly acquired knowledge on unseen questions.
- Kujanpää et al. (2024) use question-based context distillation for document-based knowledge acquisition, which is similar in spirit to the standard-training baseline in our experiments.
For methods that learn directly from document text, prior work reports standard unsupervised fine-tuning to be ineffective for knowledge acquisition (Kujanpää et al., 2024; Ovadia et al., 2024). We therefore prioritized a more recent alternative, DiSC, for an additional comparison.
Comparison with an external method: DiSC (Padmanabhan et al., 2026, ICML Spotlight)
We evaluated DiSC before submission, but omitted the results due to its very low performance. We report this comparison below to address the reviewer's request for an external baseline.
Our method vs. DiSC. DiSC trains a student model to learn document content through next-sentence prediction, without using question-answer pairs. Specifically, a frozen teacher predicts the next sentence given the preceding sentences, and the student learns to match these predictions without seeing the preceding context. In contrast, our method generates diverse questions about the document's facts and relationships, and trains the student to answer them without access to the document. We implemented DiSC on the same documents using Qwen3.8-27B, with matched training settings and compute budget (LoRA rank 64, KL distillation with temperature 2).
| Metric | DiSC | Our method |
|---|---|---|
| New-knowledge accuracy | 0.06 | 0.98 |
| Compositional accuracy | 0.07 | 0.70 |
DiSC largely fails to make newly acquired knowledge usable for question answering in our setting. One possible explanation is that sentence-level distillation does not explicitly teach the model to apply learned facts across different contexts or question formulations. For example, given "Mira Toivanen works as a quality coordinator. She maintains a library of job templates," learning the second sentence without its context does not establish who "she" refers to. Similarly, learning "Mira Toivanen is the quality coordinator" does not necessarily enable answering "Who is the quality coordinator?" (due to the reversal curse; Berglund et al., 2023). We note that this finding does not contradict DiSC's reported results, which evaluate knowledge acquisition through multiple-choice questions rather than open-ended recall or cross-document composition.
Our question-driven distillation directly addresses these challenges by generating questions that make entity references explicit and probe the same facts from different directions. This provides a plausible explanation for the large gap in knowledge acquisition and cross-document composition.
We will include this external baseline and its implementation details in the revised manuscript.
Q4: Prompts
We provide the 40 company documents, evaluation questions, model answers, and data-generation prompts in our https://anonymous.4open.science/r/fictional-companies-dataset-B3AE
Minor: Q5, Q6
Four models is correct. We will correct both to “all four models,” consistent with Figure 2 and the experimental setup. We thank the reviewer for catching this typo. The correct total is 35,738, and we will update Table 3 accordingly.
Q1: Training-seed variability and statistical reliability
We thank the reviewer for highlighting this concern. We have repeated the main comparison between standard and hallucination-aware training with three independent training seeds for Qwen3.8-27B. Please see the multi-seed results table in our response to Reviewer 2VyW (Q3).
The substantial hallucination reduction is consistent across all three seeds: hallucination-free rates improve from 0.440 ± 0.041 to 0.920 ± 0.049 (direct) and from 0.712 ± 0.024 to 0.983 ± 0.019 (reasoning). These improvements are substantially larger than the observed seed variability.
We agree that smaller differences, such as KL vs. SFT, should not be overinterpreted without additional repetitions. Importantly, our central contribution does not rely on the superiority of KL over SFT, but on the substantial and consistent reduction in hallucinations achieved by hallucination-aware training. We will include the multi-seed results in the revised manuscript.
Q2: Training/test probe overlap and generalization
The original training and test probes share an automated generation pipeline, so some similarity in question structure is possible. To investigate this concern, two authors independently wrote 20 new hallucination-probing questions each (40 total), after training. These questions were never used for training or model selection and include diverse formulations of missing information and false premises.
We evaluated three models under both training methods, using GPT-5-mini as the judge. The table reports hallucination-free rates averaged across the two manually written sets (direct / reasoning).
The improvement generalizes to independently written questions across all three models reported here, suggesting that the models are not merely learning the original probe format.
For example, when asked "When did Alvar Aalto design the famous building that now serves as the office of Aalto Harbor Systems Oy?", standard Qwen3.8-27B fabricated a year (1938), whereas hallucination-aware training correctly rejected the false premise.. Below, we show two example answers to this question.
Standard training:
Hallucination-aware training: