acceptodds
Under review as a conference paper at ICLR 2027

Anchoring the Bridge Across Hops: Unlocking Latent Multi-Hop Reasoning under Knowledge Injection

Abstract

Test-time training (TTT) keeps parametric knowledge current by writing updated facts into the parameters, but those facts are queried in multi-hop form at inference and their combinations cannot be enumerated during training. Whether a model trained only on the atomic form composes arbitrary chains within a single forward pass remains a critical question. We evaluate this on a two-hop benchmark of updated facts in which no training sample contains both hops of a chain. We show that latent composition remains hard for existing rewriting methods for knowledge injection (paraphrase, keyword diversification, and added context), which help the model acquire the updated facts but leave composed accuracy low (below at B). We attribute this to the two hops never being aligned on the bridge entity during training. We propose Bridge Anchoring by Common Enrichment (BRACE), which reuses the same descriptor pool on the bridge in both hops, so that the two hops present it in a common form. Across four pretrained LLMs spanning two families and B–B and five domains, BRACE improves composed accuracy over the strongest baseline in all model and domain cells, by to in relative terms, and raises the composition rate, composed direct-QA accuracy restricted to the chains that answer at least half the paraphrases of both hops, up to . A count-matched control isolates that alignment: at matched atomic acquisition, drawing the bridge descriptor from the same pool in both hops raises composed accuracy, and a type-incorrect descriptor costs acquisition rather than composition. A logit lens reads BRACE resolving the bridge to layers earlier. We further assess two other routes to elicit composition, chain-of-thought prompting and composition demonstrations on held-out entities, and find that the first lowers composed accuracy by recalling the pretrained bridge, while the second raises composition only when it precedes the target atoms and only on BRACE's training text. This paper presents a basis for compositional generalization under continual knowledge injection, where the chains queried at inference are by construction absent from the training data.

Primary area:
transfer learning, meta learning, and lifelong learning
Keywords:
Latent multi-hop reasoning, continual knowledge injection, data rewriting

Reviews

Among:
Shape:
1234this paper, Top 32%
Figure 1. Mean review rating of the reviewed papers at ICLR 2027. This paper averages 2.50 over 2 reviews, above 68% of the others.
Reviewer_ByigOpenReview ↗
Rating
3
Confidence
3
Informativeness
2
Presentation
3
Soundness
2
Read review
Paper summary

The paper studies whether language models can compose newly injected facts when training presents each fact separately. It constructs 300 counterfactual two-hop chains across five domains and distinguishes atomic-fact acquisition from successful composition. The proposed method, BRACE, attaches descriptors drawn from a shared pool to the bridge entity in both hops, creating a common representation across otherwise separate training examples. Experiments cover four models from two families, ranging from 1.7B to 14B parameters. BRACE achieves the highest composed-accuracy point estimate in every model–domain combination. Further analyses examine descriptor sharing, descriptor correctness, intermediate bridge predictions, chain-of-thought prompting, and the ordering of composition demonstrations.

Critical strengths
  1. The benchmark separates atomic acquisition from composition and controls for answers derived from real-world knowledge.
  2. BRACE achieves the highest composed-accuracy point estimate in all 20 model–domain combinations. The descriptor controls and training-budget comparisons help interpret these gains.
Critical weaknesses
  1. Composition rate uses a different acquired-chain subset for each method. Differences in subset difficulty can therefore influence the comparison. The smaller denominators reported in Appendix H leave this issue unresolved.
  2. The benchmark uses entity-disjoint two-hop chains trained together. Shared entities and sequential revisions would provide a stronger test of the proposed continual-updating application.
Questions for authors
  1. How does the composition-rate advantage change on chains acquired by both methods and across acquisition thresholds?
  2. Does the descriptor-sharing gain persist across randomized descriptor partitions and sequential updates with reused entities?
Reviewer_FtJqOpenReview ↗
Rating
2
Confidence
2
Informativeness
2
Presentation
2
Soundness
2
Read review
Paper summary

This paper studies whether language models can combine two updated facts after learning them in separate training samples. The authors propose BRACE, a data rewriting method that uses the same pool of descriptions for the bridge entity in both facts. They evaluate it on 300 counterfactual two-hop chains across five domains, using four models and three rewriting baselines. BRACE has the highest mean composed accuracy in all 20 model-domain settings. The paper also tests shared versus separate descriptor pools, examines intermediate-layer predictions. The paper also studies CoT prompting and additional training on two-hop examples whose entities do not overlap with those in the test chains

Critical strengths

The method is simple and improves results across several settings. BRACE changes the training text and uses the same fine-tuning setup as the baselines. Its gains appear across two model families and five domains. On Qwen3-1.7B, its average single-hop score is close to DeepKI's, while its average composed score is higher. This is useful evidence that how facts are written can affect how well a model combines them.

Critical weaknesses

The explanation of why BRACE works needs more support. Figure 4 shows earlier bridge readout under BRACE, but the paper provides little analysis of why earlier bridge readout helps BRACE perform better on composed questions. With only GEO and one random seed, the analysis offers limited evidence for how shared descriptors help composition.

The experiments do not show whether BRACE helps with true new facts or actual knowledge updates. The main benchmark uses real entities but deliberately false relations, arranged into separate two-hop chains with no shared entities across chains. This is useful for testing whether a model uses injected knowledge. The paper does not test learning true facts that the model did not know, updates that actually occurred, or facts drawn from natural documents. It is also unclear how descriptor pools would be built for new entities with little known information, or used across many related facts without knowing the future test chains. An experiment on true new knowledge or real updates, with a description of how the rewriting process works in that setting, would better support the method's practical value.

Questions for authors

CR is calculated on separately selected chains for each method, and these sets may differ in difficulty. Does BRACE still have a higher CR when evaluated on the same chains that pass the single-hop threshold for both methods?

Why use counterfactual facts about real entities rather than facts built from abstract symbols? Would an additional experiment with abstract symbols help separate gains in composition from gains in overcoming conflicting pretrained knowledge?

From OpenReview, CC BY 4.0. Authors as signed there; shown unedited.

Related papers

Loading the map…

Discussion (0)

Sign in to comment.