acceptodds
Under review as a conference paper at ICLR 2027

Learning the Map, Not the Route: Curriculum, Capacity, and Transfer in Tool-Navigating Language Models

Abstract

We compare what language models learn from knowledge-graph navigation traces, which query a graph with tools and end in a machine-verifiable evidence path, and from free-form chain-of-thought distilled from a teacher (free-CoT). In a pre-registered study of 0.6B to 14B models on a synthetic, SHACL-governed kinship benchmark with strict format isolation, facts are withheld from the prompt and every edge in an emitted evidence path must have been retrieved during the episode, so a recited path scores zero. Each finding is tied to a registered criterion or labelled post hoc. (1) On the original corpus, navigation traces lose to free-CoT by 21.0 points at 12B. The decisive split inverts an ordering direction the trace compiler had held constant, and the navigation model's failures recite that constant: it learned the route, not the map. (2) When both corpora are regenerated with that direction varied, navigation leads by 23.7 points (Gemma-12B; seed-level interval +5.2 to +42.3), and the result replicates at Qwen3-14B under a rule registered before training (+13.9; +5.1 to +22.8). Varying it improves navigation by 25 to 29 points in every family tested; free-CoT drops in Gemma but not in Qwen3-14B. (3) Training on prose-only reasoning traces suppresses tool use (tool-call rates of 1.3% to 58.6% at 0.6B to 4B). Adding the tool calls back to the same prose restores tool use to 100%, and the gain from varying the direction holds for prose traces too (+23.8). At 4B the compiled notation adds 6.1 points on the original curriculum and 9.1 on the repaired one, a smaller effect that comes mostly from how the evidence path is written and does not carry over to transfer. (4) On a bijectively relabeled second domain, every model evaluated without in-context examples scores close to zero on the verified metric, for opposite reasons at the two sizes: the 4B model keeps the notation but loses the task, and the 12B model does the opposite. Two in-context examples fully restore the notation, and the navigation-trained 12B model then produces verified evidence on 65.1 5.9% of the new domain, 13.8 points above the untrained base model given the same examples (below the registered 15-point margin; the seed-level interval includes zero) and 5.8 points below the free-CoT model. The ability transfers, but no better than after free-CoT training. We report the null result with its repair, seed-level uncertainty next to every item-level interval, and three comparisons that would have favored our hypothesis but that our registered gates refused.

Primary area:
foundation or frontier models, including LLMs
Keywords:
knowledge graphs, tool use, language model agents, supervised fine-tuning, chain-of-thought distillation, curriculum design, cross-domain transfer, pre-registration

Reviews

Among:
Shape:
1234this paper, Top 30%
Figure 1. Mean review rating of the reviewed papers at ICLR 2027. This paper averages 2.67 over 3 reviews, above 70% of the others.
Reviewer_EgY8OpenReview ↗
Rating
2
Confidence
3
Informativeness
2
Presentation
2
Soundness
2
Read review
Paper summary

This paper studies how curriculum, model capacity, and training-trace format affect tool-based graph navigation. A change to ordering-direction coverage reverses the relative performance of compiled navigation and free-form reasoning. Experiments also separate tool participation from task success and test transfer to a relabeled organizational domain. The synthetic graph allows the training curriculum to be changed while keeping the underlying navigation task fixed.

Critical strengths
  1. The curriculum intervention produces a effective result: navigation leads free-form reasoning by 23.7 pp on Gemma-12B and 13.9 pp on Qwen3-14B after the change. This is useful evidence that training coverage can determine the apparent benefit of a reasoning format.

  2. The paper reports negative transfer results and abstention trade-offs rather than relying only on a favorable aggregate metric. These analyses make the central curriculum finding more informative. Separating tool participation from grounded answer correctness helps reveal why aggregate success changes.

Critical weaknesses
  1. The format effect is not examined cleanly. Under token-based matching, the tool-matched prose control B′ covers only 49% of the training examples in D. So the evidence for compiled notation itself is weaker than that for curriculum coverage. Comparing both example-matched and token-matched variants would help quantify this confound instead of attributing the entire difference to representation.

  2. Transfer is tested through a narrow, largely structure-preserving domain relabeling. Together with the failed transfer advantage, this limits conclusions about learning reusable navigation procedures beyond the synthetic task. Besides, whether the learned procedure would survive changes in graph structure, relation composition, or the number of required tool steps.

Questions for authors

Does the D–B′ difference persist when unique training-item coverage is matched, with token budget reported separately?

Reviewer_CCMiOpenReview ↗
Rating
3
Confidence
2
Informativeness
3
Presentation
2
Soundness
3
Read review
Paper summary

This paper compares two kinds of data for fine-tuning 0.6B to 14B language models on a synthetic kinship knowledge graph: navigation traces compiled from the ontology, which query the graph with tools, and free-form chain-of-thought (CoT) distilled from a teacher model. The metric is strict. A model gets credit only if every edge in its evidence path was retrieved during the same episode. Navigation training first trails free-CoT by 21 points, which the authors trace to the trace compiler always using the same ordering direction. When the training data includes both directions, navigation training leads by 23.7 points on Gemma-12B and 13.9 points on Qwen3-14B. Further controls show that navigation traces mainly keep the model calling tools. With only prose reasoning traces, tool use drops sharply in small models (to 1.3% to 58.6% at 0.6B to 4B), and adding the tool calls back restores it. However, the in-domain lead does not carry over to a new domain. On a second domain where only the relation names change, navigation training transfers no better than free-CoT once two in-context examples are given. The authors describe the study as pre-registered and release code, data and per-item records.

Critical strengths
  1. The evaluation is strict in ways that matter for agent training, model cannot score by reciting a memorized path. Every arm must also clear a 90% tool-call floor before it can enter a comparison, and the analysis code refuses the contrast otherwise. The authors publish their bug ledger and retracted claims, and the released per-item records reproduce the reported numbers. I verified the Gemma-12B main table with AI assistance.

  2. The diagnosis of the initial null result is convincing. The authors trace the 21-point gap to the trace compiler, which always ordered candidates by “eldest”. The navigation model’s errors confirm this: in 60% to 86% of its wrong answers, it picks the eldest candidate. Varying this single decision lifts navigation by 25 to 29 points in every model family tested.

Critical weaknesses

The main result rests on a weakened baseline. The free-CoT arm (B) is trained with a Facts: block but evaluated without facts, and it can learn tool use only from two exemplars. Much of the +23.7 gap on Gemma comes from B itself dropping 19.8 points, a drop that does not appear on Qwen3-14B. Against Gemma's strongest B arm, navigation training leads by only 3.9 points, and the seed-level interval (-10.5 to 18.4) includes zero.

Questions for authors

At 4B, the B′ control narrows the in-domain margin to +9.1, compared with +23.7 and +13.9 for the B-based contrasts at 12B and 14B. How large is the margin against a B′ control at 12B and 14B?

Reviewer_mw7wOpenReview ↗
Rating
3
Confidence
3
Informativeness
3
Presentation
3(+1)
Soundness
3
Read review
Paper summary

This paper introduces empirical study investigations on: what LLMs learn when fine-tuned on tool-based knowledge-graph navigation traces against teacher-distilled free-CoT. The key findings include (a) static data generator rules force models to memorize static routes rather than learning the map. This can be resolved by varying decision rules during training. (b) Prose-only SFT unintentionally suppresses tool invocation in smaller models, this can be resolved by re-inserting explicit tool syntax. (c) under vocabulary shifts, 12B models retain task reasoning but lose vocabulary, this can be resolved by two in-context exemplars. These findings are evaluated on synthetic kinship and corporate structure benchmarks, proven valid.

Critical strengths
  1. sticking strictly to pre-registration by committing hypotheses, decision rules, thresholds etc. prior to running experiments. Also reporting initial hypothesis failures transparently.
  2. during evaluation, isolation of tool suppression (such as syntax omission by using identical prose plus interleaved tool syntax) is well design.
Critical weaknesses
  1. the paper presents and empirical audit rather than a unified agent framework, the lack of a clear visual diagram illustrating the experimental pipeline, conditions and evaluation gates reduces readability.
  2. the proposed resolution relies on manually varying a single explicit decision rule (flipping from eldest to youngest) on a clean synthetic graph. In real-world datasets, decision variables can be complex, it is unclear how applicable this fix is outside of controlled synthetic benchmarks.
  3. it remains unproven whether re-inserting tool calls into prose scales smoothly when agent context dynamically evolves during multi-turn open-ended reasoning.
Questions for authors
  1. In condition B', tool calls were statically spliced into teacher prose. How robust is this resolution when intermediate reasoning dynamically drifts or evolves during open-ended multi-turn tool interaction?
  2. What mechanism explains that varying the decision exposure caused free-CoT performance to decrease on Gemma-12B while increase on Qwen3-14B?
2 comments

Thank you for the thoughtful rebuttal, and for running the follow-up B' experiment on Gemma-12B.

W1: Adding the end-to-end overview figure significantly improves readability. W2/W3/Q1: I appreciate the explicit clarifications regarding the diagnostic role of B' and the scope boundaries around closed-form synthetic graph queries versus open-ended, real-world agent environments. Q2: Thank you for testing your train/evaluation mismatch hypothesis directly and transparently reporting both the partial recovery and the remaining unexplained residual.

All of my concerns have either been constructively addressed or properly acknowledged. I will update my Presentation score and maintain overall recommendation in support.

Thank you for engaging with the rebuttal and for updating your assessment. We appreciate the suggestions; the overview figure in particular made the paper easier to follow.

From OpenReview, CC BY 4.0. Authors as signed there; shown unedited.

Related papers

Loading the map…

Discussion (0)

Sign in to comment.