FactGRASP: Graph-Aware Relational Activation Shaping for Factual Knowledge Discovery, Recall, and Transfer in LLMs

Large Language Models (LLMs) implicitly encode factual knowledge in their hidden representations, which are entangled across layers and other learned information, making them difficult to discover and reinforce. Existing solutions, such as retrieval augmentation, external probing, and knowledge editing, either leave the hidden states unchanged, provide a mechanism for diagnosis without reinforcing knowledge recall or modify parameters in a way that may cause interference. We propose Graph-Aware Activation ShaPing (FactGRASP) as a method to linearly transform activations of LLM's hidden states with explicit Knowledge Graph (KG) triples. FactGRASP learns a linear map of subject-relation to object representations using a KG-contrastive objective with and without relation-aware gating. The two major objectives are (i) to provide a diagnostic tool to identify the types of factual associations that are linearly detectable, and (ii) to reshape activations to improve model representations in knowledge-rich settings. We evaluate the approach on KG completion across 19 LLMs, where the object representation obtained by simply composing the subject and relation representations leads to 0.064 MRR. With the learned linear map this increases to 0.405 and with relation-aware gating to 0.417 MRR. We further show that a trained linear map can be reused across models with an MRR comparability of up to 95.9%. Additional experiments examine when relational predictions become accessible during training and whether factual associations are consistently difficult across models. The results demonstrate that graph-aware activation shaping is a lightweight mechanism to make factual associations more accessible and available in language models.