acceptodds
Under review as a conference paper at ICLR 2027

FactGRASP: Graph-Aware Relational Activation Shaping for Factual Knowledge Discovery, Recall, and Transfer in LLMs

Abstract

Large Language Models (LLMs) implicitly encode factual knowledge in their hidden representations, which are entangled across layers and other learned information, making them difficult to discover and reinforce. Existing solutions, such as retrieval augmentation, external probing, and knowledge editing, either leave the hidden states unchanged, provide a mechanism for diagnosis without reinforcing knowledge recall or modify parameters in a way that may cause interference. We propose Graph-Aware Activation ShaPing (FactGRASP) as a method to linearly transform activations of LLM's hidden states with explicit Knowledge Graph (KG) triples. FactGRASP learns a linear map of subject-relation to object representations using a KG-contrastive objective with and without relation-aware gating. The two major objectives are (i) to provide a diagnostic tool to identify the types of factual associations that are linearly detectable, and (ii) to reshape activations to improve model representations in knowledge-rich settings. We evaluate the approach on KG completion across 19 LLMs, where the object representation obtained by simply composing the subject and relation representations leads to 0.064 MRR. With the learned linear map this increases to 0.405 and with relation-aware gating to 0.417 MRR. We further show that a trained linear map can be reused across models with an MRR comparability of up to 95.9%. Additional experiments examine when relational predictions become accessible during training and whether factual associations are consistently difficult across models. The results demonstrate that graph-aware activation shaping is a lightweight mechanism to make factual associations more accessible and available in language models.

Primary area:
neurosymbolic & hybrid AI systems (physics-informed, logic & formal reasoning, etc.)
Keywords:
knowledge representations, mechanistic interpretability, knowledge graphs, linear transformations, large language models, hidden states, factual associations

Reviews

Among:
Shape:
1234this paper, Bottom 27%
Figure 1. Mean review rating of the reviewed papers at ICLR 2027. This paper averages 2.00 over 3 reviews, above 27% of the others.
Reviewer_TVMqOpenReview ↗
Rating
2(+1)
Confidence
3
Informativeness
1
Presentation
2
Soundness
2(+1)
Read review
Paper summary

This paper proposes FactGRASP, which learns a linear transformation over frozen LLM representations to predict the object of a knowledge graph triple from the sum of subject and relation representations. The authors additionally introduce a relation-gated mixture of transformations (MoT), and study whether a transformation learned on one model can be reused on another model after fitting a linear alignment between their representation spaces. Across 19 models, the authors report that the learned map raises KG completion MRR from 0.064 (untransformed composition) to 0.405 (0.417 with MoT), and that transferred maps retain a median of 95.9% of the reference model’s MRR.

Critical strengths

S1. The paper evaluates the proposed method on 19 language models from nine model families and provides a variety of analyses of the results, including layer selection and fact-level agreement across models.

Critical weaknesses

W1. Insufficient Controls for Assessing Intrinsic LLM Knowledge.

Comparing a parameter-free additive baseline (hs+hrh_s + h_r) against a parameterized transformation trained directly on KG triples does not establish that the MRR improvement (from 0.064 to 0.405) reflects latent knowledge in the LLM's representations. Because the linear map and MoT are explicitly optimized using a contrastive objective over tens of thousands of training triples, the performance gain may largely reflect task-specific supervision and dataset regularities rather than pretrained factual knowledge. Without rigorous control baselines that isolate the contribution of the supervised mapping and direct comparisons with established relational probing methods such as Hernandez et al. (2024), the results do not sufficiently support the claim that FactGRASP diagnoses or unlocks intrinsic LLM knowledge.

W2. Overstating Cross-Model Knowledge Transfer.

The experimental setup does not substantiate the claim that learned mappings can be reused across models “without retraining.” Applying model A’s mapping WAW_A to model B requires fitting a pair-specific alignment matrix SB→AS_{B \rightarrow A}, with the same parameter count as a native probe, using the same training subject–relation pairs. Thus, although WAW_A remains fixed, adaptation still requires a newly fitted transformation and does not reduce the stated training-data or parameter requirements relative to training a probe directly on model B. Moreover, the composition WASB→AW_A S_{B \rightarrow A} is itself a single linear map from B’s representation space. A high transfer ratio is therefore consistent with successful supervised alignment between representation spaces and does not, by itself, establish the transfer of relational knowledge. The paper should qualify the “without retraining” claim and distinguish representation alignment from knowledge transfer, ideally through a matched comparison with a native probe trained on model B.

Questions for authors

Q1. To conclusively determine whether the high MRR (0.405) stems from the LLM’s intrinsic knowledge rather than the fitting capacity of the supervised linear map, could the authors provide a comparison between pretrained LLMs and randomly initialized LLMs of the same architectures across the evaluated models? Specifically, how much of the performance gap disappears when the underlying LLM representations contain no pretrained factual knowledge?

4 comments
AuthorsOpenReview ↗

New experiments based on very useful pointers to strengthen our paper and position - thank you

Thank you very much for your detailed review, and for noting the breadth of the evaluation across 19 models from nine families. You raised two questions: whether the score reflects knowledge inside the model, and whether the transfer claim is overstated. These are exactly the right questions. We ran new experiments to answer both, and we changed the claims where you were right. All scores below are MRR (mean reciprocal rank, from 0 to 1, higher is better), and all percentages are shares of an MRR. We uploaded a revised version of the paper with changes highlighted in blue.

Q1 and W1. Trained models against randomly initialized models of the same architecture. We ran exactly the comparison you suggested on all 19 architectures. We kept everything else the same: layer, map, training, three seeds and evaluation (new Section 4.2 with Figure 3, Appendix F and Table 13).

  • Overall: with random weights the map reaches an MRR of 0.186, against 0.405 with trained weights. About 54% of the MRR disappears without pretraining.
  • Every model: the trained model is better for all 19 models, and the difference is statistically significant for each one (paired bootstrap test over the test facts, 95% interval). The gap ranges from 0.026 MRR (ModernBERT-base, with its which uses its third layer) to 0.313 MRR (Llama-3.1-8B).
  • What the map learns by itself: the random-weight MRR is close to simply predicting the most common answer of each relation (0.215). This is roughly what supervision alone can learn from the dataset.
  • Unseen entities: on test facts whose subject and answer never appear in training, memorising answers cannot help. Trained models still reach an MRR of 0.318, while random-weight models fall to 0.045. Here about 86% of the MRR disappears without pretraining.

More controls point the same way (MRR):

  • training on shuffled answers gives 0.051;
  • copying the answers of the most similar training questions gives 0.295;
  • with 4,096 training facts, trained models reach 0.369, while random-weight models reach 0.185 even with all of them;
  • we asked 7 models to complete 2,000 unfinished sentences by themselves; on facts where the model writes the correct answer itself, the map reaches 0.671, and on facts where it writes a wrong answer, 0.384; without the map, both groups reach only 0.065 and 0.066 ; overall, the map ranks the correct answer first for 31% of these facts, against 10% that the models answer by themselves and 11% with random weights.

Direct comparison with Hernandez et al. (2024). Hernandez et al. use one linear map per relation, hoWrhs+br. We added this as a baseline. To give it every advantage, we fit each Wr directly to our answer vectors instead of estimating it from the model's gradients, as they do. MRR:

  • fitted on all training facts of each relation: 0.368;
  • fitted on 8 examples per relation, the number Hernandez et al. use: 0.193;
  • our single shared map: 0.405, better on all 19 models.

The goals also differ. Hernandez et al. approximate what the model computes from a prompt. We retrieve and rely on vector representations from the model's representations of the subject and the relation.

W2. Overstating cross-model transfer. We agree that "without retraining" was too strong. The paper now says "without retraining the map, after fitting an alignment that uses no answers", and we no longer call it knowledge transfer but representation alignment. Thank you very much for suggesting this rewording. We also ran the matched comparison you proposed (Figure 3c, Appendix H with Figure 14 and Table 16).

What is trained, and on what data. Reusing map A on model B has two pieces:

  • Map A (WA) is our linear map for model A, trained exactly as in the main experiments: on all 33,937 training facts, each with its correct answer. It is trained once and then kept fixed.
  • The alignment SBA is not trained by gradient descent. It is one 768 by 768 matrix, fitted in a single step by ridge regression (Equation 7). It maps model B's vector for an input to model A's vector for the same input. Each input is only a subject and a relation, for example "Paris, capital of". The answer ("France") is never used. In the paper, we fit it on all 33,937 training inputs.

So the reused map does use answered facts, but only from model A. It uses no answer from model B.

  • Why 256 examples. As you point out, SBA has as many parameters as a new map for model B, so the fair comparison is at an equal, small amount of model B data. We therefore repeated the experiment with only 256 examples on the model B side:
    • the alignment SBA is fitted on 256 inputs, without answers;
    • for comparison, a new map is trained directly on model B with 256 inputs and their correct answers.
AuthorsOpenReview ↗

Second part that did not fit into the first response

A small budget is where it matters whether answers are needed. With the full data, both approaches reach almost the same MRR, as the paper already reported (96.9% to 98.7% of the MRR).

  • Results with 256 examples. Over all 342 model pairs, the reused map keeps a median of 68% of map A's MRR, while the new 256-fact map on model B keeps only 30% of model B's full MRR. The reused map scores higher in 337 of the 342 pairs. With 1,024 examples, the numbers are 76% against 55%, and the reused map is ahead in 300 pairs. On the six pairs of Appendix H, a new 256-fact map trained on model B but ranking model A's candidates, exactly as the reused map does, reaches only 18% of map A's MRR, against 73% for the reused map. So reusing a map really does need less data: no answers, and far fewer examples.

  • "WASBA is itself one linear map from B." This is true. Applying the translation SBA and then map A's matrix WA is the same as applying one single linear map to model B's vectors. What differs is how we obtain that map:

    • A new map for model B must be trained on model B's vectors together with the correct answers, for example the input "Paris, capital of" paired with the answer "France".
    • The reused map never sees a correct answer for model B. WA is trained on model A alone. SBA is fitted only by passing the same input text through both models and matching their two vectors, which needs no answers at all.

    So we end up with a working map for model B without giving it a single answer from model B. A natural question is whether the translation SBA secretly learns the facts by itself. If it did, the reused map would still work with a useless map A, or with a model B that knows nothing. It does not:

    • with a map A trained on shuffled answers, the MRR drops to 10% of the original;
    • with model B replaced by its random-weight version, the MRR drops to the random-weight level (0.197);
    • an alignment fitted on data with none of the tested facts still works: it keeps 60% of the MRR when fitted on ConceptNet, and 60% when fitted on subject names only. With no map at all, the aligned vectors keep only 14%.

    We also report a weaker result. An alignment fitted without the relations being tested keeps 65% of the MRR, so it does help the alignment to see those relations. In short, the alignment translates between the two models' vector spaces. The facts come from map A and from what model B already knows.

We are grateful for this review. It led to the most important new experiments in the revision. We hope the random-weight results on all 19 architectures and the comparison at equal data answer your concerns about soundness. If they do, we would be grateful if you would consider updating your score.

Thank you for the detailed response. The new experiments address several of my concerns, and I have raised my score accordingly. However, the extent of the revision has led to an inconsistency in the paper’s stated contributions: the title and abstract still emphasize activation shaping, while the revised results and limitations (Appendix J) support reading out knowledge from frozen representations. The revision also appears somewhat rushed, with inconsistent numbers and mismatched figure captions across sections (e.g., the MoT MRR reported as both 0.417 and 0.416). While the paper has improved, I believe it would benefit from another round of careful revision before it is ready for acceptance.

AuthorsOpenReview ↗

Thank you and fixes in the revised pdf

Thank you for raising your score and for the careful reading. We agree with your points and will upload a revised PDF after fixing the issues raised.

  • We will edit the abstract and title accordingly.
  • The MoT is now reported as 0.417 throughout. The control table previously came from a second training run with the same settings, whose mean rounded to 0.416.
  • We will correct the caption of Figure 5 (panel b shows a rank correlation, not Hits@1).
  • We will correct the description of the early pretraining checkpoints.
  • We now state 95.9% consistently as the median.

Thank you again for the very useful suggestions.

Reviewer_LT7kOpenReview ↗
Rating
2
Confidence
3
Informativeness
2
Presentation
2
Soundness
3
Read review
Paper summary

The paper trains a linear predictor on frozen LLM representations to rank the object of a subject–relation pair. It also studies whether a predictor trained on one model can be reused with another model’s representations after learning an alignment between their embedding spaces. The experiments examine factual prediction across models, layers, and training checkpoints.

Critical strengths

The approach is simple and does not require updating the underlying LLM. The evaluation across 19 models is substantial, and the random-encoder, majority-object, and unseen-entity controls help establish that pretrained representations contribute useful information beyond patterns learned from the supervised triples alone. The cross-model results are also interesting. Reusing a fixed predictor after alignment suggests some compatibility in how different models represent relational information. This is a useful empirical finding, even without demonstrating improvements to the models’ generated answers.

Critical weaknesses

The title and framing suggest a broader contribution than the main experiments establish. The predictor is trained on labeled knowledge-graph triples, so improved ranking does not necessarily mean it has uncovered factual knowledge already present in the LLM. Likewise, the transfer experiment demonstrates reuse of an external predictor, rather than knowledge being transferred into another model. The transfer also requires a separately fitted alignment for each model pair and evaluates predictions against the reference model’s object embeddings. Its practical scope is therefore narrower than the phrase “knowledge transfer” suggests. Finally, most of the improvement comes from the supervised linear map over raw vector addition; the relation-gated extension adds relatively little, and stronger prompting baselines narrow or sometimes reverse the advantage. The paper is more convincing as a study of relational prediction and representation alignment than as a method for improving factual recall in LLMs.

Questions for authors

Not for now

5 comments
AuthorsOpenReview ↗

Addressing Weaknesses and Questions

Thank you very much for your careful and balanced review, and for your kind words about the number of models, the controls and the cross-model results. We agree with your main point about framing and have changed the paper accordingly, where we indicated the applied changes to address all review comments in blue. We ran new experiments for each concern. All scores below are Mean Reciprocal Rank (MRR) and all percentages are shares of an MRR.

Title and framing. We agree with having to revise the positioning of our approach. The paper now describes the method as identifying facts in the hidden representations of a frozen model with a trained map and the cross-model result as reusing that map in another model, which we now call representation alignment, as kindly suggested by Reviewer TVMq. We rewrote the abstract, introduction and conclusion in this way. We also propose a new title, which is *"FactGRASP: Graph-Aware Relational Activation Shaping for Factual Knowledge Discovery and Representation Alignment in LLMs” should we get the chance to revise after discussion.

Uncover factual knowledge. This is the central question you raised. We added several tests to address it (new Section 4.2 with Figure 3, and Appendix F):

  1. Random weights. We repeated the full setup on all 19 architectures with random weights. The map reaches an MRR of 0.186 there, against 0.405 on the trained models. The difference is statistically significant for every model (paired bootstrap test over the test facts, 95% interval). The random-weight MRR is close to simply predicting the most common answer of each relation (0.215). So the map alone learns little more than these frequencies, and the rest comes from pretraining.
  2. Unseen entities. On test facts whose subject and answer never appear in training, trained models reach an MRR of 0.318 and random-weight models 0.045 (8 models).
  3. Shuffled answers. Training the map on shuffled answers gives an MRR of 0.051.
  4. Amount of training data. With 4,096 training facts, trained models reach an MRR of 0.369. Random-weight models reach only 0.185, even with all 33,937.
  5. Predictions of pre-trained models alone We asked 7 models to complete 2,000 unfinished sentences, such as "Rudboneh Rural District is located in the country of ...". ur map reaches an MRR of 0.671 on facts correctly predicted by some models, where the models on average reach 0.065.The MRR on correct predictions is not 1 because the map reads only the separately encoded subject and relation vectors from one layer, but overall it ranks the correct answer first for 31% of these facts, against 10% that the models answer correctly by themselves and 11% with random weights.
  6. Model size and pretraining. The MRR grows with model size within each family. It also grows during OLMo-2 pretraining, while the map's training stays the same.

Together, these results show that the map reads information that pretraining put into the model. It does not create it from the training facts.

Knowledge transfer. We agree with your description and changed the wording to representation alignment.

To make clear what this means, we added three results:

  • Model B must already hold the facts. If model B is replaced by the same architecture with random weights, the reused map drops to the random-weight level (MRR 0.197 with the Llama-3.1-8B map). If model A's map is trained on shuffled answers, the MRR drops to 10% of the original.
  • The alignment needs no answers and few examples. Map A is trained on model A's 33,937 training facts with their answers. The alignment is a single matrix, fitted in one step by ridge regression, that only matches the two models' vectors for the same inputs (a subject and a relation, never the answer). In the paper it uses all 33,937 inputs. With only 256 such inputs, the reused map keeps a median of 68% of its original MRR over all 342 model pairs. A new map trained on model B with 256 answered examples keeps only 30% of its full MRR. The reused map is ahead in 337 of 342 pairs. Even an alignment fitted only on ConceptNet data, which contains none of the tested facts, keeps 60% of the MRR.
  • Ranking in model B's own space. You are right that the candidates come from model A. Our Appendix D already tests moving the prediction back into model B's space, which keeps 64% to 80% of model B's own MRR. We now state this limitation in Appendix J.
AuthorsOpenReview ↗

Second part

The relation gate adds relatively little. Thank you for this comment. It led us to study the gate in much more depth, and the revised paper now reports the full picture (Appendix J). In summary:

  • A consistent gain on every model. The relation-gated mixture (MoT) improves on the single linear map for all 19 models. The average MRR rises from 0.405 to 0.416 (+0.012 MRR, 2.9% higher). The share of facts whose correct answer is ranked first (Hits@1) rises from 0.302 to 0.316 (4.8% higher), again on all 19 models.
  • Statistically significant. Because the gain is positive on every model, it is significant across models (Wilcoxon signed-rank test, p < 0.0001 for both MRR and Hits@1). Within single models, a paired bootstrap test over the test facts (95% interval) confirms it for 14 of the 19 models.
  • Large gains where the single map is weakest. MRR +12% for ModernBERT-base (0.268 to 0.301), +9% for SmolLM3-3B (0.344 to 0.376) and +7% for Granite-4.2-8B (0.296 to 0.316). With the cosine loss, the gate raises the MRR from 0.290 to 0.361, a 24% gain. On ConceptNet it adds 0.019 MRR (8% higher).
  • Better than adding nonlinearity. On the six models of our design comparison, the MoT also beats nonlinear maps (MLPs with one and two hidden layers) on 5 of 6 models (MRR 0.445 against 0.442 and 0.437).
  • It draws on pretraining. On random-weight models the gate gives no gain (-0.001 MRR). So the gate uses what the pretrained model knows about each relation, not just extra fitting capacity.

In short, the gate does not give a huge boost on average, but its gains are consistent, statistically significant, and largest for the weaker models, which we find promising. On entities and relations never seen in training, the single linear map stays slightly ahead, so we keep the single map as the main result and present the MoT as a reliable improvement on top of it.

Stronger prompting baselines narrow or reverse the advantage. This is correct, and we report it as it is. The linear map is ahead for Qwen3.5-4B and Gemma-4-E4B. For Llama-3.1-8B, a trained soft prompt is ahead. The soft prompt also learns from the same training facts, but it is trained through the whole model, while our map never touches the model. We now present the two as different measurements rather than competitors.

"More convincing as a study of relational prediction and representation alignment." We think this is a fair summary, and the revised framing follows it. Thank you again for a very helpful review. We hope the new framing and the new tests answer your concerns. If they do, we would be grateful if you would consider updating your score.

AuthorsOpenReview ↗

A gentle follow up on our submitted rebuttal

Thank you again for your review. We hope our reply and the revised paper address your concerns, especially the new framing as knowledge discovery and representation alignment. We will upload a further revised PDF today. We would be happy to answer any remaining questions before the discussion ends.

Reviewer_5sSnOpenReview ↗
Rating
2
Confidence
3
Informativeness
2
Presentation
2
Soundness
2
Read review
Paper summary

LLMs have been shown to encode factual knowledge implicitly within their hidden representation. Recent works have explored different approaches to discover and exploit such encoded knowledge. They usually keep the hidden representation of knowledge items within LLMs unchanged, but focus on providing a mechanism for diagnosis without reinforcing knowledge recall or modify parameters in a way that may cause interference. This paper proposes factGrasp, a method that learns a linear map (and also mixture of multiple maps on extended version) on the hidden state of LLMs for knowledge items, i.e., subject, predicate, and object. In particular, the linear map is applied on the sum of the subject and predicate to fit to the object. KG-contrastive objective is employed to tune the map. The results show substantial improvement over the baseline (h+p, t). In addition, the paper also shows that the learned linear map for one model can be transferred to another model by tuning a transformation matrix.

Critical strengths

– The problem that this paper tackles, i.e., identifying factual knowledge in LLM hidden state, is important and requires further research as most works only look at the output of LLMs, not the internal representations.

– The gain over the baseline is substantial.

– the cross-model transfer finding is insightful: only tuning a linear function is enough for transfer.

Critical weaknesses

— only one very simple and trivial baseline (h+p,t) is used. However, why only translation, but not other operations between h and p were used?

– Using linear map might be too restricted choice limiting the generalization over a broad range of relational knowledge. Some further studies is required to check if the hidden state space is actually globally flat or curved. In case of curved space, the geometric-aware transformations are required.

– The paper could benefit from further analysis on various compositions of facts, such as symmetry, (h,p,o) and (o,p,h) for some p need to be true at the same time, so this requires to be algebraically consistent.

– page 4, notation inconsistency, bold used for embedding, but also italic and unbold

– despite the substantial improvement over the baseline, the overall performance is low. An in-depth analysis is required to show such low performance related to linear map limitation, or lack of knowledge in LLM’s hidden state, or complexity of relations involved in facts.

Questions for authors

– which relations types or concepts are identifiable by the proposed approach?

4 comments

Thank you very much for your thoughtful review. Thank you also for noting that identifying facts from a model's internal representations is an important problem, and that the cross-model transfer is insightful. Your questions led to several new analyses that make our paper stronger. We answer each point address below and also provide a revised paper with all changes marked in blue. The reported new results use the setup initially reported in terms of selected layer, 2,766 candidate answers, three random seeds, and MRR as evaluation score.

Operations to compose vectors. This is a fair and interesting point. We now compare several alternatives of how to compose vectors instead of only addition (Appendix G, Figure 13, six models). For the proposed addition, we receive an MRR of 0.435, subtraction leads to an interesting 0.431, concatenation to 0.430 leading to twice the parameters needed, element-wise product to 0.234, only using the subject without operation to 0.147 MRR, and only using the object to 0.126. The sum works as well as any other choice. Both the subject and the relation are needed. Even with no trained map at all, adding one fixed offset per relation to the subject vector (as in TransE) already reaches an MRR of 0.275.

Second, we added baselines that do not combine vectors at all (MRR):

  • predict the most common answer of the relation: 0.215;
  • copy the answers of the most similar training questions: 0.295;
  • TransE trained on the same facts: 0.136;
  • one separate map per relation, in the style of Hernandez et al. (2024): 0.368, below our map on all 19 models;
  • our map on models with random weights: 0.186.

The proposed linear map method (MRR 0.405 on average over the 19 models) outperforms these baselines. To compare to fine-tuning, fine-tuned BERT in Table 8 (MRR 0.381) was in the initial submission.

Geometry of the hidden state space Thank you for this very valid and interesting pointer. To try to address it, we trained nonlinear maps with the same loss (Appendix G). This experiment results in an MRR of 0.442 for an MLP with one hidden layer, and 0.437 for two hidden layers. THe linear map achieved 0.435 on one selected layer, being improved to 0.445 for the MoT method. The proposed MoT, which switches between linear maps depending on the relation, does better than both MLPs on 5 of 6 models. This does not prove that the space is flat everywhere and we fully acknowledge that maps for curved spaces are a good direction for future work, which we currently include in our limitations (Appendix J).

Symmetric and inverse facts. Thank you for this suggestion, which led to an interesting new analysis (Appendix I, Figure 16). We tested the reversed order of these relation types on the test set. Symmetric relations resulted in the same score of 0.112 for both orders. Inverse pairs resulted in 0.145 MRR on inverse triples as opposed to 0.227 for the original order.

So the map partly is able to handle these relation properties without being trained on them. The MRR values are low in absolute terms because these relations are among the hardest, and we say this in the paper.

Notation on page 4. Thank you for pointing this out. All vectors are now plain italic and all matrices are capital letters. We also removed a stray from Equation 6.

This is an important question. We tested each of the three explanations. Since the linear map still improves the results and the nonlinear map does not provide a substantial gain, we believe the overall low performance comes from mostly missing knowledge and complexity of relations.

As regards the missing knowledge, we ran a simple test in two steps: 1) turn the object prediction into a sentence completion task and evaluate how many facts were (un)known, and 2) compare the performance of our proposed method on the exact same facts. As regards 1), we used 2,000 test triples as a sentence, e.g. "Rudboneh Rural District is located in the country of _____" and the correct prediction, e.g. “Iran”, was considered a known fact. Llama-3.1-8B predicted 277 of 2,000 objects correctly and Gemma-4-E4B-it only 49. Even the best of these models knows only 14% of the 2,000 facts. In Step 2), we compare the proposed linear map with pre-trained model representations alone as well as randomly initialized weights for the same models as depicted in the following table.

MRR of the rankingCorrect predictionsIncorrect predictions
Proposed linear map, pretrained models0.6710.384
Only pretrained models0.0650.066
Proposed linearmap, same models with random weights0.2800.182
Same models with random weights0.0440.021

The proposed linear map outperforms the other scenarios, which makes us believe that the low overall scores can be attributed to missing knowledge. As regards the complexity of relations, the MRR is higher for triples with one of a few common objects and lower for rare ones.

To answer your main question, please consider this reply. For your other points, please consider the second reply. Q. Identifiable relations and concepts

  • Easy: relations whose answer is a type or category, such as taxon rank, manner of death, native language, religion, country, country of citizenship, sport, time zone and instance of (MRR 0.68 to 0.94).
  • Hard: relations whose answer is one specific name out of hundreds or thousands, such as cast member, director, place of birth, member of sports team and twinned administrative body.

Of the 62 relations with at least ten test facts, 32% have an MRR above 0.5. Appendix I (Figure 15, Table 17) lists the easiest and hardest relations. It also gives the MRR on four other knowledge graphs: T-REx 0.570, FB15k-237 0.325, PopQA 0.308 and ConceptNet 0.246.

Thank you again for these suggestions. We hope the new results answer your concerns. If they do, we would be grateful if you would consider updating your score.

Reviewer_5sSnOpenReview ↗

Comments on the authors reply

Thanks to the authors for their reply. The analysis related to missing knowledge is appreciated. However, it is too narrow as a central explanatory finding. It documents that performance correlates with the model's own recall rate. But, it does not explain what determines knowledge encodability or how the method can be improved beyond this constraint.

In addition, the geometry concern also remains underaddressed: marginal MLP gains do not mean there is a systematic failures on curved relational subspaces.

I would keep my original score.

AuthorsOpenReview ↗

reply to reviewer's second set of comments

Thank you for taking the time to consider our reply and for your response. The experiments in our reply not only analyzed missing knowledge, but whether the overall low performance was related to the linear map, missing knowledge in the LLM, or the complexity of relations involved in facts, as suggested by your review. However, following your review, we conducted further tests to clarify and ran further experiments on all 19 models.

1. What determines whether a fact is recovered

The model's own recall explains only a small part of the score.

  • Please note that the results do not relate to the model’s recall rate, as we do not rely on the model as such, but only on the hidden representations of a systematically selected layer. Thus, we find that the low performance relates to both, knowledge not encoded in the hidden representations and the complexity of relations. Moreover, the objective of the proposed approach is not to identify what determines knowledge encodability, but to improve knowledge discovery and representation alignment across models.
  • Whether a score is high or low depends mostly on which fact is asked (66% of the differences between scores) and hardly on which of the 19 pretrained models is used (3%). The same facts are hard for all of them. Pretraining itself still matters a lot: on models with random weights, the same map reaches only 0.186 compared to 0.405 on pretrained models.
  • We observe that of the facts on which the model is wrong, our map is able to rank the correct answer first for 28% of them, against 11% for the same map on models with random weights. This suggests that the map could help retrieve the hidden knowledge from the model.

2. On the complexity of relations We checked which properties make a fact hard. A fact is hard mainly when its answer is rarely seen, for example a specific person's name for a cast member or a small town for a place of birth. Specifically, when test facts are divided into quartiles based on the frequency of their answers in the training facts, the linear map achieves an average MRR of 0.25 for the rarest quartile, compared with 0.70 for the most frequent quartile, averaged across all 19 models. A similar trend emerges with relation cardinality: relations with many possible answers achieve an MRR of 0.15, compared with 0.56 for relations with few possible answers. Thus, fact complexity in our evaluation is driven largely by answer rarity and relation cardinality. Moreover, this performance is not merely attributable to memorizing frequent answers: even for answers absent from the training facts, pretrained models achieve an MRR of 0.250, substantially exceeding the 0.041 obtained with randomly initialized weights.

3. On improving knowledge discovery and representation alignment across models

We would like to point out that the objective of our approach is not to identify what determines knowledge encodability, but more practically to improve knowledge discovery and representation alignment across models. In this direction, we have now tested whether other models can fill in what one model is missing. Normally, the map takes the vector of one model for a question (subject plus relation) and predicts the answer. In this new experiment, we placed the vectors of the same question from three models side by side and trained one map on the concatenated input of vectors; it still predicted the answer of the first model. The mean MRR rises from 0.405 to 0.436, and 17 of 19 models improve. The weakest models gain the most. For example, Gemma-4-E4B-it shows an improvement in MRR from 0.272 to 0.355. When the two added models have random weights instead, the MRR drops to 0.384 from 0.405, so the gain comes from the knowledge in the other models, not from a larger input. This suggests that combining models is a promising direction for knowledge discovery across models.

4. On the geometry of the hidden state space As suggested in your original review, besides the MLP, we now trained a hyperbolic map, which is built for curved space and learns how curved the space is. It scores lower than the linear map on all 19 models (0.380 vs 0.405). On the 15 hardest relations, all maps stay between 0.14 and 0.17 MRR. On the other hand, the MLP with one hidden layer is slightly better than the linear map on all 19 models (0.414 vs 0.405). It is better on 14 relations and worse on none, but its largest gain on any relation is 0.08. Figure 17 in the revised PDF will show this comparison for each relation. So we find no group of relations where the linear map fails and a curved or nonlinear map succeeds. We did our best to address your comment within the limited time frame by providing non-linear maps as countertests to the assumption of a flat space. Nevertheless, in our opinion, a full investigation into the geometry of the hidden state spaces to go beyond a practical experiment requires a standalone paper on its own and is beyond the scope of this work.

Comments on the paper

1 comment
AuthorsOpenReview ↗

Thank you for the reviews and interaction!

We would like to thank all reviewers for their valuable reviews and for interacting with us during the rebuttal period. On a side note, we would like to explain the mismatch between the OpenReview and revised paper title with the ICLR 2027 author guidelines permitting no changes to the title on OpenReview until the end of the discussion period.

From OpenReview, CC BY 4.0. Authors as signed there; shown unedited.

Related papers

Loading the map…

Discussion (0)

Sign in to comment.