- Rating
- 3
- Confidence
- 3
- Informativeness
- 3
- Presentation
- 3
- Soundness
- 3
Read review
Paper summary
This paper explores the issue of factual retrieval failure in large language models through the lens of "representation collapse"—a phenomenon where semantically distinct questions occupy highly similar regions within the hidden-state space, thereby hindering accurate knowledge retrieval. The authors identify a significant negative correlation between the "crowdedness" of representations and the accuracy of factual question-answering. To address this, they propose CRAFT (Contrastive Representation Adjustment for Factuality), an unsupervised fine-tuning method. By leveraging contrastive learning and diversity regularization and without requiring ground-truth answers. CRAFT brings the representations of semantically equivalent (paraphrased) questions closer together, while simultaneously pushing the representations of distinct questions further apart.
Critical strengths
-
A novel and interesting perspective on factual retrieval failures. Compared to approaches that directly incorporate knowledge-based supervision or adjust output probabilities, this research perspective offers a degree of novelty.
-
A simple and effective method that does not require ground-truth answer supervision. The proposed CRAFT method features a concise design; it employs contrastive learning and diversity regularization to adjust the hidden representations of questions—all without the need for ground-truth answers as training labels or any modifications to the model architecture.
Critical weaknesses
1.The definition of "crowdedness" is highly dependent on the distribution of the reference questions. The authors use other questions within a fixed benchmark to approximate the local density ("crowdedness") within the space of question representations. However, this metric is simultaneously influenced by the size of the reference set, the distribution of question types, and the similarity threshold. Consequently, it remains unclear whether "crowdedness" reflects conflicts in the model’s internal knowledge representations or merely the characteristics of the question distribution within a specific dataset.
- Insufficient alignment between negative sampling and the proposed representation collapse mechanism.The proposed method uses randomly sampled questions as negatives, although its central motivation concerns representational confusion between questions associated with different factual answers. Random negatives are unlikely to represent genuinely confusable facts and may already be well separated in the model's representation space. While the authors evaluate similarity-selected negatives, this strategy does not explicitly identify questions that are semantically similar but require distinct answers.
Questions for authors
1.Is the method proposed by the authors similar to the approach of generating answers for a single problem which presented via multiple different phrasings and then using a voting mechanism? 2. See "weakness"
2 comments
Thanks for the author's response. I will maintain my score.
Response to Reviewer jjE7
We thank the reviewer for the positive assessment and the thoughtful questions.
1. Crowdedness depends on the reference set.
This is a fair concern, and we tested it directly. We pooled the questions of TriviaQA, NQ-Open and HotpotQA and computed each question's crowdedness twice: against its own dataset only, and against the combined pool of all three. The neighbour counts differ, since the pools differ in size, but the order of questions from least to most crowded is nearly the same at every size (Spearman rank correlation 0.98-0.99 on TriviaQA, 0.94-0.96 on HotpotQA and 0.89-0.92 on NQ-Open, for Qwen2.5-1.5B, 3B and 7B), and the correlation with accuracy changes by at most 0.07 (on TriviaQA, -0.26 to -0.32 with either reference set). Changing the reference set therefore leaves the results essentially unchanged (figures: 1.5B, 3B, 7B).
Three further observations point the same way:
Together, these suggest that crowdedness reflects how the model represents questions at the point of answering, not only properties of the benchmark.
2. Random negatives do not target confusable facts. We agree that random negatives do not explicitly target questions that are similar but need different answers. What we aim to reduce, however, is crowdedness, and it can be reduced in two ways: by separating close neighbours locally, or by spreading all questions apart globally. Either way, each question ends up with fewer close neighbours, so reducing crowdedness does not require negatives that target confusable pairs. As we note in the Limitations, CRAFT acts largely through global spreading. The usual risk of global spreading is that it changes the representations too much and hurts other tasks; we find that CRAFT does not reduce performance on other tasks (line 420).
Global spreading also does not leave close neighbours together. In appx B.6, lexically near-identical pairs with different answers move apart (mean similarity 0.889 -> 0.762 over 100 pairs), and each question's 5% most similar neighbours move away for every question (0.813 -> 0.561). Selecting negatives from each anchor's most similar training questions did not help (Table 10), but, as the reviewer points out, similarity alone does not guarantee that two questions are confusable. Following the reviewer's point, we also trained CRAFT on Qwen2.5-1.5B with negatives generated by a larger LLM (DeepSeek-V4.1-Flash) to be close in form to each question but to require a different answer. The results are essentially the same as with random negatives (EM / C-EM): 37.4 / 41.8 vs. 36.7 / 41.6 on TriviaQA, 12.4 / 16.5 vs. 12.7 / 16.5 on NQ-Open, and 14.9 / 16.3 vs. 14.9 / 16.3 on HotpotQA, all within the significance margins of appx B.11. This is consistent with the global spreading described above, and suggests that random negatives already provide most of the benefit.
3. Question: Is CRAFT similar to answering several paraphrases and voting?
No. The two differ in what they change, when they use paraphrases, and what they rely on: