acceptodds
Under review as a conference paper at ICLR 2027

The Knowledge Is Already There: Geometric Interventions for Factual Recall in LLMs

Abstract

Factual retrieval failures in large language models are often assumed to reflect missing knowledge. We investigate an alternative mode: representational collapse, in which questions requiring different answers occupy overlapping regions of the hidden-state space. We measure this by a question’s crowdedness, the number of other questions with highly similar representations. Across three sizes of Qwen2.5, the more crowded half of TriviaQA questions is answered correctly 22–25 percentage points less often than the rest. We propose Contrastive Representation Adjustment for Factuality (CRAFT), which reshapes this geometry through a contrastive objective over question paraphrases and negatives, without any answer supervision. Evaluated on TriviaQA, NQ-Open, and HotpotQA across three model families, CRAFT gives the best exact-match accuracy of all methods on every benchmark for all Qwen2.5 sizes, raising Qwen2.5-7B accuracy on TriviaQA from 38.4% to 58.2%. Finally, CRAFT largely removes representational collapse, consistent with many retrieval failures reflecting knowledge that is present in the model but hard to reach

Primary area:
generative models
Keywords:
Hallucination, factuality.

Reviews

Among:
Shape:
1234this paper, Bottom 27%
Figure 1. Mean review rating of the reviewed papers at ICLR 2027. This paper averages 2.00 over 3 reviews, above 27% of the others.
Reviewer_jjE7OpenReview ↗
Rating
3
Confidence
3
Informativeness
3
Presentation
3
Soundness
3
Read review
Paper summary

This paper explores the issue of factual retrieval failure in large language models through the lens of "representation collapse"—a phenomenon where semantically distinct questions occupy highly similar regions within the hidden-state space, thereby hindering accurate knowledge retrieval. The authors identify a significant negative correlation between the "crowdedness" of representations and the accuracy of factual question-answering. To address this, they propose CRAFT (Contrastive Representation Adjustment for Factuality), an unsupervised fine-tuning method. By leveraging contrastive learning and diversity regularization and without requiring ground-truth answers. CRAFT brings the representations of semantically equivalent (paraphrased) questions closer together, while simultaneously pushing the representations of distinct questions further apart.

Critical strengths
  1. A novel and interesting perspective on factual retrieval failures. Compared to approaches that directly incorporate knowledge-based supervision or adjust output probabilities, this research perspective offers a degree of novelty.

  2. A simple and effective method that does not require ground-truth answer supervision. The proposed CRAFT method features a concise design; it employs contrastive learning and diversity regularization to adjust the hidden representations of questions—all without the need for ground-truth answers as training labels or any modifications to the model architecture.

Critical weaknesses

1.The definition of "crowdedness" is highly dependent on the distribution of the reference questions. The authors use other questions within a fixed benchmark to approximate the local density ("crowdedness") within the space of question representations. However, this metric is simultaneously influenced by the size of the reference set, the distribution of question types, and the similarity threshold. Consequently, it remains unclear whether "crowdedness" reflects conflicts in the model’s internal knowledge representations or merely the characteristics of the question distribution within a specific dataset.

  1. Insufficient alignment between negative sampling and the proposed representation collapse mechanism.The proposed method uses randomly sampled questions as negatives, although its central motivation concerns representational confusion between questions associated with different factual answers. Random negatives are unlikely to represent genuinely confusable facts and may already be well separated in the model's representation space. While the authors evaluate similarity-selected negatives, this strategy does not explicitly identify questions that are semantically similar but require distinct answers.
Questions for authors

1.Is the method proposed by the authors similar to the approach of generating answers for a single problem which presented via multiple different phrasings and then using a voting mechanism? 2. See "weakness"

2 comments
AuthorsOpenReview ↗

Response to Reviewer jjE7

We thank the reviewer for the positive assessment and the thoughtful questions.

1. Crowdedness depends on the reference set.

This is a fair concern, and we tested it directly. We pooled the questions of TriviaQA, NQ-Open and HotpotQA and computed each question's crowdedness twice: against its own dataset only, and against the combined pool of all three. The neighbour counts differ, since the pools differ in size, but the order of questions from least to most crowded is nearly the same at every size (Spearman rank correlation 0.98-0.99 on TriviaQA, 0.94-0.96 on HotpotQA and 0.89-0.92 on NQ-Open, for Qwen2.5-1.5B, 3B and 7B), and the correlation with accuracy changes by at most 0.07 (on TriviaQA, -0.26 to -0.32 with either reference set). Changing the reference set therefore leaves the results essentially unchanged (figures: 1.5B, 3B, 7B).

Three further observations point the same way:

  • Threshold: tau is not a fixed value but a percentile of the model's own pairwise similarities, and we report results over several thresholds and a full (tau, pi) grid (Fig. 5), so the conclusions do not rest on one choice.
  • Datasets and models: the association holds across benchmarks with different question distributions (TriviaQA and NQ-Open, Fig. 1 and appx A.4) and across models (Qwen2.5 1.5B-32B and, in a new experiment, Qwen3-1.7B-Base, where C-EM falls from about 44% for the least crowded questions to about 8% for the most crowded; Figure).
  • Layers: crowdedness predicts failure only in the last layers, not in early layers, where surface form such as shared wording or question type dominates (Table 6). We see the same pattern on Qwen3-1.7B-Base (rho = -0.31 with accuracy and +0.30 with entropy at the final layer, against at most +0.13 for entropy at any other layer; Figure).

Together, these suggest that crowdedness reflects how the model represents questions at the point of answering, not only properties of the benchmark.

2. Random negatives do not target confusable facts. We agree that random negatives do not explicitly target questions that are similar but need different answers. What we aim to reduce, however, is crowdedness, and it can be reduced in two ways: by separating close neighbours locally, or by spreading all questions apart globally. Either way, each question ends up with fewer close neighbours, so reducing crowdedness does not require negatives that target confusable pairs. As we note in the Limitations, CRAFT acts largely through global spreading. The usual risk of global spreading is that it changes the representations too much and hurts other tasks; we find that CRAFT does not reduce performance on other tasks (line 420).

Global spreading also does not leave close neighbours together. In appx B.6, lexically near-identical pairs with different answers move apart (mean similarity 0.889 -> 0.762 over 100 pairs), and each question's 5% most similar neighbours move away for every question (0.813 -> 0.561). Selecting negatives from each anchor's most similar training questions did not help (Table 10), but, as the reviewer points out, similarity alone does not guarantee that two questions are confusable. Following the reviewer's point, we also trained CRAFT on Qwen2.5-1.5B with negatives generated by a larger LLM (DeepSeek-V4.1-Flash) to be close in form to each question but to require a different answer. The results are essentially the same as with random negatives (EM / C-EM): 37.4 / 41.8 vs. 36.7 / 41.6 on TriviaQA, 12.4 / 16.5 vs. 12.7 / 16.5 on NQ-Open, and 14.9 / 16.3 vs. 14.9 / 16.3 on HotpotQA, all within the significance margins of appx B.11. This is consistent with the global spreading described above, and suggests that random negatives already provide most of the benefit.

3. Question: Is CRAFT similar to answering several paraphrases and voting?

No. The two differ in what they change, when they use paraphrases, and what they rely on:

  • What changes: voting leaves the model unchanged and aggregates its outputs. CRAFT changes the model itself, by reshaping how it represents questions.
  • When paraphrases are used: voting needs paraphrases and several generations for every question at test time. CRAFT uses paraphrases only during training; at test time it answers once, greedily, from the original question, at the same cost as the base model.
  • What it relies on: voting depends on generating and comparing answers. CRAFT never generates or compares answers; its training signal comes only from the hidden states of the questions.

Thanks for the author's response. I will maintain my score.

Reviewer_9fS5OpenReview ↗
Rating
2
Confidence
3
Informativeness
2
Presentation
2
Soundness
2
Read review
Paper summary

The paper hypothesizes that factual QA failures in LLMs are associated with “crowdedness”, i.e., many questions having very similar final-layer representations. The authors show that more crowded TriviaQA questions are answered correctly less often, and propose CRAFT, an answer-free contrastive fine-tuning method (paraphrases as positives, random questions as negatives, plus a pairwise diversity term). CRAFT substantially improves EM on TriviaQA and transfers to NQ-Open and HotpotQA.

Critical strengths
  • I find the idea of the paper very interesting for research. Their hypothesis is plausible and worth investigating.
  • CRAFT does not use gold answers at any stage, and it is cheaper to train than the DPO-based unsupervised baselines.
  • The paper has a large appendix with many additional experiments, including controls for answer length.
Critical weaknesses
  • It is unclear on which subset of TriviaQA (and possibly the other datasets) the hyperparameters and design choices were selected, and on which subset the final results are reported (the hidden test set or the open validation set). Appendix B (“Ablations, Controls, and Additional Evaluations”, lines 1190–1191) states that all results in that section use the TriviaQA validation set, and in lines 124–125 the choice of the final layer is justified by the observation that it “gives higher accuracy”. If the main results in Table 1 are also reported on the validation set, the design choices were made on the evaluation data. This is important for judging whether the results are reliable.
  • EM conflates answer format with knowledge, especially for base models under a zero-shot prompt. The C-EM gains are much smaller than the EM gains, which suggests that a substantial part of the headline improvement comes from shorter answers.
  • It is unclear whether the effectiveness of CRAFT is consistent with the authors’ explanation of the failures (i.e., that low accuracy is caused by crowded questions). In lines 443–445 the authors write: “Lexically near-identical pairs with different answers also separate (0.889 → 0.762 over 100 pairs). However, a question’s middle and least similar neighbours move away by comparable amounts, so part of the effect is global”. The authors acknowledge this in the Limitations (lines 477–479), but if CRAFT acts largely through global spreading, its success does not support the claim that crowding causes retrieval failures.
  • Fig. 6 raises a related concern. The main text describes it as showing that "Δ is small across most of the grid" (lines 860–862) and that "at 1.5B and 3B, crowded and uncrowded questions gain by similar amounts" (lines 459–460), whereas the caption states that "for 1.5B and 3B, the most crowded questions gain less than the rest" (lines 897–898). The figure supports the caption: for the 10% most crowded questions, Δ ranges from −0.107 to −0.139 across all τ at 1.5B, and from −0.052 to −0.149 at 3B (a negative Δ means that the most crowded questions gain less from CRAFT than the remaining questions). Assuming that Fig. 6 and Table 1 are computed on the same set of questions, combining the two implies that at the 99th-percentile threshold, the most crowded 10% of questions gain roughly 4 EM points at 1.5B, versus about 17 for the remaining questions (about 10 vs. 23 at 3B). At 7B, the sign of Δ depends on τ: it is mostly negative across the grid for τ up to the 50th percentile, and positive (up to +0.064) for moderately crowded questions only at approx. τ ≥ 70-80th percentile. If crowding were the mechanism, the most crowded questions, which also have the lowest base accuracy and therefore the most headroom, would be expected to benefit the most.
  • The paper overstates its claims. The title (“The Knowledge Is Already There”) and the first contribution (“We identify representation collapse as a mode by which language models can fail to retrieve factual knowledge”) imply a causal link, while the authors later state that causality cannot be established from their experiments.
  • Nearly all figures are very hard to read because of the extremely small font; Figure 13 in the Appendix is the only exception.
Questions for authors
  • Which TriviaQA split is used for the results in Table 1: the de-duplicated validation set, or the hidden test set via the official leaderboard? Lines 124–125 state that “training CRAFT with final-layer similarity gives higher accuracy than using all layers”. On which data was this accuracy measured? The same question applies to the choice of λ (§B.3), the negatives (§B.4), the paraphrase source (§B.5), and the all-layer similarity for Llama-2-7B (§B.2). Was a held-out development split used?
  • Why not include more recent base models (e.g., Qwen3 base)? Llama-2 and Qwen-2.5 are outdated.
  • §B.10 shows results on the post-trained SmolLM3-3B. Could you also report results on Qwen2.5-Instruct, where the answer-format confound is weaker?
  • Why don’t you use few-shot prompting? It is the standard way to evaluate base models on TriviaQA and would largely remove the answer-format issue.
  • Could you report a metric that is less sensitive to answer format (e.g., token-level F1, or an LLM judge), or EM under few-shot prompting?
  • Why are Qwen’s answers short while Llama’s are long? What stops generation for Qwen2.5 base models (EOS, newline, other)? Which answer normalization is used for EM?
3 comments
AuthorsOpenReview ↗

Response to Reviewer 9fS5 (1/2)

We thank the reviewer and respond to each point below.

1. Design Choices and Dataset Splits. Thanks for raising these important methodological questions.

  • All TriviaQA results are given on the de-duplicated validation set. This is because the test set is not publicly available.
  • Our choice of the final layer was motivated first by the base model, before any fine-tuning: in Table 6, the answer-free relationship between crowdedness and first-token entropy is strongest at the final layer. The post-fine-tuning comparison in Table 7 was run afterwards, and it agrees on all three benchmarks.
    • Nevertheless, we appreciate the concern. To this end, we note that the choice transfers from these Qwen2.5-7B models to other models, including to the Qwen3-1.7B-Base model (reported below).
    • Further, let us note that the differences between the variants we compared are small compared with CRAFT's gains over the baselines. As such, selection itself cannot account for the gains seen by the method.
  • Defaults: throughout, our defaults are the simplest options: equal weighting of the two loss terms (lambda = 1), random negatives, and the final layer. These defaults are best or within noise on all three benchmarks, including NQ-Open and HotpotQA, which were never used for any choice (Tables 7, 9, 10, 11). The results therefore do not depend on the TriviaQA validation set.
    • For Llama-2-7B, we found (without using answers) that final-layer training left its representations unchanged (Table 8), prompting a move to the all-layer representation.
    • The defaults also transfer to a model on which nothing was tuned. Qwen3-1.7B-Base, trained with exactly the same settings, improves from 15.3 to 33.9 EM and from 32.7 to 37.5 C-EM on TriviaQA.

2. Answer Format, and Instruct-Tuned Baselines

  • The gains are not just answer format Yes, EM is a harder metric than C-EM. Let us note that most of the literature only reports EM accuracy. We introduce C-EM as a way to control for answer format, and as expected, we see a reduction in gains. Nevertheless, we still see large improvements in C-EM as well.

    • Further, precisely to study the effect of answer formatting and length, we already included an analysis in the submission (line 414 and appx B.8).

    • The most important observation is this: about 70% of base model answers are already short (1-2 words). On these, CRAFT

      1. increases average answer length (1.41 -> 1.56).
      2. increases EM accuracy by 12.4%
      3. increases C-EM accuracy by 11.8%.

      Any increase in accuracy on these cannot be explained by making the answers shorter, or improving their format in any way.

    • Further, we show that both picking the shortest of 1010 sampled answers, and instructing for shorter answers does not get anywhere close to CRAFT performance.

  • Few-Shot Prompting, and Instruct models That said, we agree with the point made: instruct models, especially with few-shot prompting, should give a cleaner view. While we studied base models to keep in line with the existing pipeline in the literature (and especially with the methods we mainly compare to), we carry this out below.

    • We evaluate both the base and instruct versions of these models with 5 shot prompts. These just give five question-answer pairs as examples. The five examples are fixed general-knowledge questions, not taken from TriviaQA, and are the same for every prompt.

    • The resulting performance on TriviaQA is thus (entries are EM-acc/C-EM-acc)

      ModelBase, 0-shotBase, 5-shotInstruct, 0-shotInstruct, 5-shotCRAFT, 0-shot
      Qwen2.5-1.5B20.97 / 34.1831.33 / 35.5828.99 / 33.6429.89 / 32.9836.72 / 41.61
      Qwen2.5-3B26.15 / 39.6739.54 / 44.1135.92 / 39.9535.15 / 38.7147.45 / 52.29
      Qwen2.5-7B38.36 / 49.8147.76 / 53.7149.66 / 53.7048.46 / 52.2258.18 / 60.70

For the base models with 5-shot prompting, EM is computed on the first line of the output, since these models continue with a new made up question after answering; CRAFT and the instruction-tuned models are scored on the full output, which is stricter.

  • Observations:
    • Interestingly, C-EM accuracy for the instruct 5-shot is smaller than instruct 0-shot. It is unclear why this occurs.
    • Few-shot prompting improves the base models substantially, as the reviewer expected, and removes most of the format issue (average answer length 1.9-2.2 words for both base and instruct models).
    • However, zero-shot CRAFT still outperforms: on 7B, it is ahead of every one of these settings by more than 8% in EM and 7% in C-EM.
    • Thus, this experiment bears out our work in a very stark way: we see large improvements over even this more stringent baseline, while competing methods cannot match them.
AuthorsOpenReview ↗

Response to Reviewer 9fS5 (2/2)

3. Global Spreading and Crowding

  • Note that global spreading reduces crowding too.
    • Imagine, for instance, a surface with a varying density of points. Global spreading uniformly increases the distances between them. Local refinement takes small patches, and evens out the clumps within it. Either way, crowding gets reduced.
    • Therefore, it is not inherently inconsistent if the mechanism of CRAFT passes through global spreading.
      • It is worth noting that CRAFT certainly does reduce crowding. Lexically near-identical pairs with different answers move apart (mean similarity 0.889 -> 0.762 over 100 pairs), and each question's 5% most similar neighbours move away for every question (0.813 -> 0.561, appx B.6).
    • Within the method, the diversity term contributes to the global effects, while the contrastive term is more about local refinement. It is important to note that both effects are important. Table 9 is the relevant ablation, wherein we see a 5% and 3% boosts over the pure diversity loss with λ=1\lambda = 1.
  • Your argument about the headroom is very reasonable. However, there is an important confounder: it is possible that the most crowded questions are significantly harder, purely in terms of the available knowledge, in which case it could be that these questions are unanswerable for these models. Of course, it is also possible that the most crowded questions remain fairly crowded (perhaps at a slightly lower threshold τ\tau)
    • To test this, we prompted the base model on every question with ten different four-shot prompts, both greedily and with 16 samples per prompt (160 samples per question). Among the 100 most crowded questions, the base model never answers 81% (1.5B), 69% (3B) and 73% (7B) correctly in any of these attempts. Further, CRAFT could achieve no gain on these questions either (C-EM accuracy remained same across base and fine-tuned versions). Thus, there does appear to be such a systematic issue.
    • Re. Fig. 6 specifically, you are right about line 459, and we will fix this.
  • If we consider crowdedness as the explanation, and reduction in this as the mechanism behind CRAFT, then the increase in accuracy should, in principle, be associated with the extent to which CRAFT "de-crowds" questions. In this spirit, we present plots which measure the increase in accuracy versus the fraction of neighbours removed by CRAFT. The plots are available here, (figures: 1.5B, 3B, 7B).
    • Here, we do see a trend of increase in the accuracy gain relative to the fraction reduced, although the curves show a number of different behaviours (including maxima at the low-deletion end).
    • Overall, the trends are not extremely clear, but they are not inconsistent with de-crowding as the mechanism underlying CRAFT, and arguably aspects of this are present (for instance, in the high regions in Fig. 6 being focused towards the right, and in the uptick in accuracy gain at the fraction = 1 end of the above curves).

4. We make no causal claims, and the title

  • We want to be clear that the paper does not make causal claims. This is repeatedly discussed in the paper, and mentioned as a key point in the limitation. Any impression of this that comes through is wrong, and we will fix it.
  • The title refers to the fact that answers are recovered without any answer supervision, so these answers must have been in the model all along. Perhaps adding "Finding" to the title (so Finding Knowledge That is Already There) would be more amenable?

5. Figure readability.

  • Thank you. We will enlarge the text in all figures to match Fig. 13.

6. Other Questions

  • More up to date models: Initially, we had evaluated Qwen3-1.7B-Base on TriviaQA, and found that its accuracy was much lower than Qwen2.5-1.5B, and so just went with the 2.5 models, especially since this was the model studied by the competitors. For the sake of completeness, we have done fine-tuning on this model.
    • We trained with paraphrases and negatives prepared for Qwen2.5-1.5B, since these were already available. Last layer representations are used.
    • CRAFT improves TriviaQA from 15.3 to 33.9 EM and 32.7 to 37.5 C-EM. It also transfers in EM to HotpotQA (7.8 -> 14.8) and NQ-Open (5.6 -> 11.9). C-EM rises on HotpotQA (14.3 -> 15.9) and is unchanged within noise on NQ-Open (16.8 -> 16.1).
  • Answer length, stopping and normalization: generation stops at the EOS token (with a limit of 50 new tokens). Qwen2.5 base models usually emit EOS right after a short answer, while Llama-2-7B continues with unrelated text, which is why we score Llama on the first line. EM uses the standard TriviaQA normalization (lower-casing and removing punctuation, articles and extra whitespace).
AuthorsOpenReview ↗

Follow-up on our response

Dear Reviewer 9fS5,

Thank you again for your detailed review. As the discussion period ends soon, we wanted to check whether our response addresses your concerns. Following your suggestions, we:

  • clarified the evaluation split and design choices, and showed that the defaults transfer untuned to Qwen3-1.7B-Base (point 1);
  • added few-shot and instruction-tuned baselines, and addressed answer format and the format-robust metric (point 2);
  • analysed how gains relate to decrowding;
  • clarified the causal wording and the title (point 4);
  • will enlarge the figure text (point 5);
  • answered the remaining questions, including results on Qwen3-1.7B-Base, answer length, stopping and normalization (point 6).

We would be grateful for your feedback and are happy to clarify anything further.

Reviewer_zJCUOpenReview ↗
Rating
1
Confidence
4
Informativeness
1
Presentation
2
Soundness
1
Read review
Paper summary

This paper studies why LLMs fail on factual questions. The authors argue that part of the failures come from "representational collapse", when questions with different answers get very similar hidden states. They show that more crowded questions are answered worse and propose CRAFT, a contrastive fine-tuning method that pushes question representations apart without using answers. CRAFT shows large gains on TriviaQA and smaller gains on other QA datasets.

Critical strengths
  • I find the research question interesting, and the idea of improving factuality without answers is appealing.
  • The method is simple and cheap, and does not hurt general capabilities.
  • I appreciate that the authors include several controls for answer length.
Critical weaknesses

Method

The mechanism is not supported.

The paper claims that CRAFT works by separating confusable questions. However, the authors' own ablations show that the gains are not concentrated on crowded questions, and that the diversity term alone gives most of the improvement. To me this looks like a global spreading of representations rather than fixing collapse. The title and abstract claim more than the experiments show.

Crowdedness is not well defined.

The measure uses raw cosine similarity of final-layer states, which are known to be anisotropic. So crowdedness may reflect question length or template rather than meaning. The authors mention confounders such as entity frequency but never control for them. Some results also contradict the story, e.g. on HotpotQA accuracy grows with crowdedness, and the 32B model is more crowded but much more accurate.

Limited Novelty

training on paraphrases is a pretty standard technique and there are plenty of methods. One can also think of it as one form of best-of-n and methods that involve distillation of best-of-n techniques.

Experimental design

The gains are mostly answer format.

Base models in zero-shot setting answer verbosely, and strict EM is too strict on this matter. CRAFT mostly makes answers shorter, so EM gains are much larger than C-EM gains. I believe the real effect is much smaller than the headline numbers.

The setup favors the method.

I do not understand why the paper uses base models in a zero-shot setting. QA is usually evaluated with few-shot prompting or on instruct models, both of which fix the format for free. The instruct models are even used in the pipeline but never evaluated. On the only post-trained model, the gains almost vanish outside TriviaQA.

Gains do not transfer beyond TriviaQA.

The real improvements are on TriviaQA, which is also the training data. On other datasets the gains are within noise. Moreover, TriviaQA and NQ are known to have train-test overlap [1], are likely in pretraining data, and NQ contains many questions with outdated answers [2]. I would encourage authors to aim for stronger generalization and better datasets such as PopQA or SimpleQA.

Baselines are not tuned fairly.

CRAFT was tuned on the same set used for evaluation and uses an extra instruct model, while baselines use one fixed configuration. Some baselines look misconfigured. When CRAFT uses the same resources as the baselines, I doubt it will be better on NQ and HotpotQA.

No variance or confidence intervals.

All results are single runs, and the method is sensitive to hyperparameters. The crowdedness curves use small bins, so many of the claimed trends are within noise.

References

[1] Lewis, Patrick, Pontus Stenetorp, and Sebastian Riedel. "Question and answer test-train overlap in open-domain question answering datasets." Proceedings of the 16th Conference of the European Chapter of the Association for Computational Linguistics: Main Volume. 2021.

[2] Pletenev, Sergey, et al. "Will it still be true tomorrow? multilingual evergreen question classification to improve trustworthy QA." Proceedings of the 2025 Conference on Empirical Methods in Natural Language Processing. 2025.

Questions for authors
5 comments
AuthorsOpenReview ↗

Response to Reviewer zJCU (1/3)

We respond to each point below; several overlap with the other reviews, and we refer to those responses where relevant.

1. "The mechanism is not supported."

  • The paper does not claim a causal mechanism. The Limitations state explicitly that the relationship between how crowded a question's representation is and whether the model answers it correctly is associative, not causal.
  • This relationship (more crowded questions are answered correctly less often) does not depend on CRAFT: it appears in the Qwen2.5 base models at every size from 1.5B to 32B before any training (Fig. 1, Fig. 13), and now also in Qwen3-1.7B-Base (Figure: Qwen3-1.7B-Base accuracy vs. crowdedness).
  • How far this association extends to other architectures is not the question this study addresses. What the paper shows is a strong association between crowdedness and the accuracy of factual retrieval for certain models and architectures, including the Qwen2.5 and Qwen3 models studied here; it does not claim the association is universal.
  • Global spreading is acknowledged in the Limitations, but it also reduces crowding: crowding can be reduced locally, by reshaping small regions, or globally, by spreading all questions apart, and either way each question ends up with fewer close neighbours. The usual risk of a strong global term is that it hurts other tasks; this does not happen with CRAFT (line 420). CRAFT measurably reduces crowdedness (Sec. 5.2, appx A.3), and questions that lose a larger fraction of their neighbours tend to gain more accuracy (details and figures in our response to Reviewer 9fS5, point 3).
  • The diversity term alone does not explain the gain: the full objective improves on diversity-only by 4.7 EM on TriviaQA (58.18 vs. 53.49, Table 9) and by about 3 points in C-EM; on NQ-Open and HotpotQA it adds about 1 EM.

2. "Crowdedness is not well defined."

Crowdedness is defined explicitly in Section 3. The review raises three concerns about it: anisotropy, surface features, and the HotpotQA and 32B results.

  • We do not understand why anisotropy in the overall distribution of representations means that similarity stops being a pertinent metric. Nevertheless, at best this criticism seems to be saying that we can capture similarity in a better way. Even if that is true, this does not eliminate the effect we study.
  • Surface features (question length, template). Crowdedness can arise from many sources, including shared templates (line 40), and it lowers accuracy however it arises. Moreover, if crowdedness reflected only surface form, it would predict failure in early layers, where surface form dominates. It does not: the relationship is near zero in early and middle layers and appears only in the last layers, strongest at the final layer (Table 6); the same holds on Qwen3-1.7B-Base (Figure: per-layer correlation, Qwen3-1.7B-Base). Crowdedness is also not specific to one benchmark's question mix (see our response to Reviewer jjE7, point 1). Entity frequency is harder to control, since its frequency in pretraining data is not observable; we list it as a possible common cause in the Limitations.
  • HotpotQA and 32B. We include HotpotQA, a multi-hop benchmark, to measure whether CRAFT transfers beyond the single-hop recall it is trained on, not as a test of the crowdedness association, which concerns single-hop questions (appx A.4). CRAFT still improves on HotpotQA (point 5). For 32B, crowdedness is defined by percentiles within each model, so its level cannot be compared across models (appx B.7); within Qwen2.5-32B, accuracy again falls with crowdedness, from about 55-60% to 31-46% (Fig. 13).
AuthorsOpenReview ↗

Response to Reviewer zJCU (2/3)

3. "Limited novelty."

Our contributions are twofold: (a) identifying representation crowding as a possible cause of failure to retrieve answers, and showing its negative association with accuracy; (b) an answer-free method, CRAFT, that reduces crowding and yields large gains in factual QA. The characterization of CRAFT as paraphrase training or best-of-n does not match the method:

  • Paraphrase training fits the model to question-answer pairs. CRAFT never sees an answer: no gold answers, no generated answers, and no correctness signal of any kind.
  • Best-of-n, and methods that distil it, generate several answers and select among them, which requires a selection signal and, at inference, several generations. CRAFT generates no answers during training (only paraphrases of the questions), and at inference it answers once, greedily, from the original question, at the same cost as the base model. Best-of-n is also orthogonal to CRAFT: it is an inference procedure that could equally be applied on top of a CRAFT-trained model.
  • What CRAFT changes is the geometry of question representations, guided by a failure mode the paper identifies (crowding). We are not aware of prior methods that improve factual recall this way. If the reviewer knows of one, we would be glad to see the reference and discuss it.

4. "The gains are mostly answer format" and "the setup favors the method."

  • This experimental design is standard in factual QA: all of the methods we compare against use zero-shot QA on the base model as the baseline. We are not introducing a new setting; we follow the existing one.
    • We use instruct models only for paraphrase generation. We also ablate this by generating paraphrases with the base model alone, which recovers most of the EM gain and nearly all of the C-EM gain over the base model on TriviaQA (line 258 and appx B.5).
  • Nevertheless, we also evaluated the base models with 5-shot prompting and the instruction-tuned Qwen2.5 models (the same models used in our pipeline) with their chat template and a system instruction to output only the answer, with and without the same five examples, on the same 9,961 de-duplicated TriviaQA validation questions (EM / C-EM, %):
ModelBase, 0-shotBase, 5-shotInstruct, 0-shotInstruct, 5-shotCRAFT, 0-shot
Qwen2.5-1.5B20.97 / 34.1831.33 / 35.5828.99 / 33.6429.89 / 32.9836.72 / 41.61
Qwen2.5-3B26.15 / 39.6739.54 / 44.1135.92 / 39.9535.15 / 38.7147.45 / 52.29
Qwen2.5-7B38.36 / 49.8147.76 / 53.7149.66 / 53.7048.46 / 52.2258.18 / 60.70
  • These settings fix the answer format, and CRAFT remains ahead of all of them at every size, on both EM and C-EM.

  • For the base models with 5-shot prompting, EM is computed on the first line of the output, since these models continue with a new question after answering; CRAFT and the instruction-tuned models are scored on the full output, which is stricter.

  • EM is a harder metric than C-EM, and most of the literature reports EM only. We introduce C-EM precisely to control for answer format.

  • We analyze the effect of answer length on accuracy in detail (line 414 and appx B.8). About 70% of base-model answers are already short (1-2 words). On these, CRAFT:

    1. increases average answer length (1.41 -> 1.56),
    2. increases EM accuracy by 12.4%,
    3. increases C-EM accuracy by 11.8%.

    These gains cannot be explained by shorter answers or by better formatting.

  • We also show that neither picking the shortest of 10 sampled answers nor instructing the model to answer briefly comes close to CRAFT's performance.

AuthorsOpenReview ↗

Response to Reviewer zJCU (3/3)

5. "Gains do not transfer beyond TriviaQA."

  • Overlap: CRAFT uses only the questions of the TriviaQA training split, never their answers; all results are on the de-duplicated validation set (lines 180, 337). Train-test overlap [1] and outdated answers [2] would affect every method equally, including the base model. CRAFT also does not learn new answers: on questions the base model never answers correctly, even with ten different few-shot prompts and 160 samples per question, its C-EM is unchanged (details in our response to Reviewer 9fS5, point 3).
  • Significance: Hoeffding-based 95% margins for TriviaQA, NQ-Open and HotpotQA are 0.8, 1.4 and 1.0 points (appx B.11). The EM gains over the base model are significant in every case except Llama on HotpotQA. C-EM gains are significant on TriviaQA; on HotpotQA they are significant for the smaller models and within the margin for the rest; on NQ-Open all methods are within the margin.
    • In every group of Table 1, CRAFT never significantly underperforms, and often outperforms.
  • Further datasets:
    • WebQuestions (Table 15): EM +11.9. Qwen3-1.7B-Base: EM +6.3 on NQ-Open and +7.0 on HotpotQA.
    • PopQA and SimpleQA: EM rises on both (PopQA 3.6 -> 19.1, SimpleQA 1.3 -> 3.3), while C-EM is unchanged within noise (PopQA 20.4 -> 20.1, SimpleQA 4.0 -> 3.9). On SimpleQA we also used its official grading prompt with DeepSeek-V4.1-Flash as the judge: CRAFT is slightly above the base model (4.7% vs. 4.0% correct).

6. "Baselines are not tuned fairly."

  • The baselines follow their original implementations (appx C), with only one noted change: for FactTune-MC we skip the step that uses a larger external model, so that the baseline is trained only from the model's own outputs.
  • Every baseline was trained on TriviaQA, just like CRAFT.
  • No baseline is misconfigured. We made every effort to produce a fair comparison, and we do not appreciate this accusation without justification.
  • Resource-wise, the baselines are afforded all the resources they demand. In fact, FactTune and Self-Eval are trained for 3 epochs, while CRAFT is trained for only one (line 342).
  • The only possible imbalance is that CRAFT uses an instruct model for paraphrase generation. CRAFT fine-tunes the base model; the instruct model is used only to generate paraphrases. With paraphrases from the base model alone, CRAFT reaches 52.08 TriviaQA EM, above every baseline in Table 1 (best: FactTune-MC, 45.63) (Table 11, appx B.5). Neither method then uses a larger model that could implicitly leak answers.

7. "No variance or confidence intervals."

  • Significance of the main results is discussed in point 5.
  • Crowdedness curves: each bin holds 2% of the data, about 200 questions, so the 95% confidence interval is about ±7 points (using the variance bound 0.25 for a Bernoulli). The drop from the least to the most crowded end of the curves in Fig. 1 is 20-40 points, far larger.
  • We reran the full CRAFT pipeline for Qwen2.5-1.5B three more times, each with newly generated training data and a new training run. Across the four runs (including Table 1), TriviaQA results are 36.57 ± 1.05 EM and 41.36 ± 0.41 C-EM (mean ± standard deviation; lowest run 35.05 / 40.74). Even the lowest run is well above the base model (20.97 / 34.18) and every 1.5B baseline (best EM 28.02, best C-EM 36.85).
    • The main source of variation in CRAFT is paraphrase generation, which these reruns vary; its effect is small.
    • The only other hyperparameters concern the learning rate and momentum terms during fine-tuning; the initialisation is not random.

Comments on the paper

1 comment
AuthorsOpenReview ↗

General response to all reviewers

We thank all reviewers for their time and feedback. Below we summarize the further analyses we ran in response to the reviews.

Further analyses

  1. Few-shot and instruction-tuned baselines. 9fS5 and zJCU asked whether few-shot prompting or instruct models would remove the gains. We evaluated the Qwen2.5 1.5B/3B/7B base models with 5-shot prompting, and the instruct models with and without examples. Zero-shot CRAFT stays ahead at every size on both EM and C-EM, by more than 8 EM and 7 C-EM points at 7B (9fS5, point 2; zJCU, point 4).
  2. Newer base model. 9fS5 asked for a more recent model. On Qwen3-1.7B-Base, more crowded questions are again answered correctly less often, and CRAFT, trained with the same settings and no tuning, improves TriviaQA from 15.3 to 33.9 EM and from 32.7 to 37.5 C-EM, with EM gains on NQ-Open and HotpotQA (9fS5, points 1 and 6).
  3. Variance. zJCU asked about variance. We reran the full pipeline at 1.5B three more times, each with new paraphrases and a new training run. Across the four runs, CRAFT reaches 36.57 ± 1.05 EM and 41.36 ± 0.41 C-EM, and even the lowest run is well above every baseline (zJCU, point 7).
  4. Reference set. jjE7 asked whether crowdedness depends on the reference set. We computed each question's crowdedness twice: against its own dataset only, and against TriviaQA, NQ-Open and HotpotQA combined. Both rank the questions from least to most crowded in nearly the same order (Spearman 0.89-0.99), and the link with accuracy stays the same (jjE7, point 1).
  5. Hard negatives. jjE7 noted that random negatives do not target confusable questions. Negatives generated by a larger LLM to be close in form but require a different answer give results within noise of random negatives (jjE7, point 2).
  6. The most crowded questions. 9fS5 noted (Fig. 6) that the most crowded questions gain less. Among the 100 most crowded, the base model never answers 81% (1.5B), 69% (3B) and 73% (7B) correctly, even with 160 samples under ten few-shot prompts, and CRAFT does not change this (9fS5, point 3).
  7. Decrowding and gains. 9fS5 and zJCU asked whether the gains relate to decrowding. We plot each question's accuracy gain against the fraction of neighbours it loses; questions that lose more of their neighbours tend to gain more (9fS5, point 3).

We will add all of these analyses to the paper, correct the text describing Fig. 6, align the introduction's wording with the Limitations, and enlarge the text in all figures.

From OpenReview, CC BY 4.0. Authors as signed there; shown unedited.

Related papers

Loading the map…

Discussion (0)

Sign in to comment.