- Rating
- 2(+1)
- Confidence
- 3
- Informativeness
- 1
- Presentation
- 2
- Soundness
- 2(+1)
Read review
Paper summary
This paper proposes FactGRASP, which learns a linear transformation over frozen LLM representations to predict the object of a knowledge graph triple from the sum of subject and relation representations. The authors additionally introduce a relation-gated mixture of transformations (MoT), and study whether a transformation learned on one model can be reused on another model after fitting a linear alignment between their representation spaces. Across 19 models, the authors report that the learned map raises KG completion MRR from 0.064 (untransformed composition) to 0.405 (0.417 with MoT), and that transferred maps retain a median of 95.9% of the reference model’s MRR.
Critical strengths
S1. The paper evaluates the proposed method on 19 language models from nine model families and provides a variety of analyses of the results, including layer selection and fact-level agreement across models.
Critical weaknesses
W1. Insufficient Controls for Assessing Intrinsic LLM Knowledge.
Comparing a parameter-free additive baseline () against a parameterized transformation trained directly on KG triples does not establish that the MRR improvement (from 0.064 to 0.405) reflects latent knowledge in the LLM's representations. Because the linear map and MoT are explicitly optimized using a contrastive objective over tens of thousands of training triples, the performance gain may largely reflect task-specific supervision and dataset regularities rather than pretrained factual knowledge. Without rigorous control baselines that isolate the contribution of the supervised mapping and direct comparisons with established relational probing methods such as Hernandez et al. (2024), the results do not sufficiently support the claim that FactGRASP diagnoses or unlocks intrinsic LLM knowledge.
W2. Overstating Cross-Model Knowledge Transfer.
The experimental setup does not substantiate the claim that learned mappings can be reused across models “without retraining.” Applying model A’s mapping to model B requires fitting a pair-specific alignment matrix , with the same parameter count as a native probe, using the same training subject–relation pairs. Thus, although remains fixed, adaptation still requires a newly fitted transformation and does not reduce the stated training-data or parameter requirements relative to training a probe directly on model B. Moreover, the composition is itself a single linear map from B’s representation space. A high transfer ratio is therefore consistent with successful supervised alignment between representation spaces and does not, by itself, establish the transfer of relational knowledge. The paper should qualify the “without retraining” claim and distinguish representation alignment from knowledge transfer, ideally through a matched comparison with a native probe trained on model B.
Questions for authors
Q1. To conclusively determine whether the high MRR (0.405) stems from the LLM’s intrinsic knowledge rather than the fitting capacity of the supervised linear map, could the authors provide a comparison between pretrained LLMs and randomly initialized LLMs of the same architectures across the evaluated models? Specifically, how much of the performance gap disappears when the underlying LLM representations contain no pretrained factual knowledge?
4 comments
Second part that did not fit into the first response
A small budget is where it matters whether answers are needed. With the full data, both approaches reach almost the same MRR, as the paper already reported (96.9% to 98.7% of the MRR).
-
Results with 256 examples. Over all 342 model pairs, the reused map keeps a median of 68% of map A's MRR, while the new 256-fact map on model B keeps only 30% of model B's full MRR. The reused map scores higher in 337 of the 342 pairs. With 1,024 examples, the numbers are 76% against 55%, and the reused map is ahead in 300 pairs. On the six pairs of Appendix H, a new 256-fact map trained on model B but ranking model A's candidates, exactly as the reused map does, reaches only 18% of map A's MRR, against 73% for the reused map. So reusing a map really does need less data: no answers, and far fewer examples.
-
"WASBA is itself one linear map from B." This is true. Applying the translation SBA and then map A's matrix WA is the same as applying one single linear map to model B's vectors. What differs is how we obtain that map:
- A new map for model B must be trained on model B's vectors together with the correct answers, for example the input "Paris, capital of" paired with the answer "France".
- The reused map never sees a correct answer for model B. WA is trained on model A alone. SBA is fitted only by passing the same input text through both models and matching their two vectors, which needs no answers at all.
So we end up with a working map for model B without giving it a single answer from model B. A natural question is whether the translation SBA secretly learns the facts by itself. If it did, the reused map would still work with a useless map A, or with a model B that knows nothing. It does not:
- with a map A trained on shuffled answers, the MRR drops to 10% of the original;
- with model B replaced by its random-weight version, the MRR drops to the random-weight level (0.197);
- an alignment fitted on data with none of the tested facts still works: it keeps 60% of the MRR when fitted on ConceptNet, and 60% when fitted on subject names only. With no map at all, the aligned vectors keep only 14%.
We also report a weaker result. An alignment fitted without the relations being tested keeps 65% of the MRR, so it does help the alignment to see those relations. In short, the alignment translates between the two models' vector spaces. The facts come from map A and from what model B already knows.
We are grateful for this review. It led to the most important new experiments in the revision. We hope the random-weight results on all 19 architectures and the comparison at equal data answer your concerns about soundness. If they do, we would be grateful if you would consider updating your score.
Thank you for the detailed response. The new experiments address several of my concerns, and I have raised my score accordingly. However, the extent of the revision has led to an inconsistency in the paper’s stated contributions: the title and abstract still emphasize activation shaping, while the revised results and limitations (Appendix J) support reading out knowledge from frozen representations. The revision also appears somewhat rushed, with inconsistent numbers and mismatched figure captions across sections (e.g., the MoT MRR reported as both 0.417 and 0.416). While the paper has improved, I believe it would benefit from another round of careful revision before it is ready for acceptance.
Thank you and fixes in the revised pdf
Thank you for raising your score and for the careful reading. We agree with your points and will upload a revised PDF after fixing the issues raised.
- We will edit the abstract and title accordingly.
- The MoT is now reported as 0.417 throughout. The control table previously came from a second training run with the same settings, whose mean rounded to 0.416.
- We will correct the caption of Figure 5 (panel b shows a rank correlation, not Hits@1).
- We will correct the description of the early pretraining checkpoints.
- We now state 95.9% consistently as the median.
Thank you again for the very useful suggestions.
New experiments based on very useful pointers to strengthen our paper and position - thank you
Thank you very much for your detailed review, and for noting the breadth of the evaluation across 19 models from nine families. You raised two questions: whether the score reflects knowledge inside the model, and whether the transfer claim is overstated. These are exactly the right questions. We ran new experiments to answer both, and we changed the claims where you were right. All scores below are MRR (mean reciprocal rank, from 0 to 1, higher is better), and all percentages are shares of an MRR. We uploaded a revised version of the paper with changes highlighted in blue.
Q1 and W1. Trained models against randomly initialized models of the same architecture. We ran exactly the comparison you suggested on all 19 architectures. We kept everything else the same: layer, map, training, three seeds and evaluation (new Section 4.2 with Figure 3, Appendix F and Table 13).
More controls point the same way (MRR):
Direct comparison with Hernandez et al. (2024). Hernandez et al. use one linear map per relation, hoWrhs+br. We added this as a baseline. To give it every advantage, we fit each Wr directly to our answer vectors instead of estimating it from the model's gradients, as they do. MRR:
The goals also differ. Hernandez et al. approximate what the model computes from a prompt. We retrieve and rely on vector representations from the model's representations of the subject and the relation.
W2. Overstating cross-model transfer. We agree that "without retraining" was too strong. The paper now says "without retraining the map, after fitting an alignment that uses no answers", and we no longer call it knowledge transfer but representation alignment. Thank you very much for suggesting this rewording. We also ran the matched comparison you proposed (Figure 3c, Appendix H with Figure 14 and Table 16).
What is trained, and on what data. Reusing map A on model B has two pieces:
So the reused map does use answered facts, but only from model A. It uses no answer from model B.