- Rating
- 2
- Confidence
- 3
- Informativeness
- 2
- Presentation
- 2
- Soundness
- 3
Read review
Paper summary
The authors study compositional generalization in the setting of an artificial task, and analyze how different compositional structures seen during training can contribute to the model’s generalization performance.
Critical strengths
I think the schema model is interesting, and a nice way to study compositional generalization.
Figures 5 and 9 are particularly interesting, and shed light on the out-of-distribution generalization abilities of transformers.
Critical weaknesses
While the schema model is interesting, understanding the notation takes more work than it should, particularly because of the mixed use of the diagrams and the G[*] notation. If possible, I would suggest reworking the notation to just use the diagrams.
I think the most interesting results in the paper are a bit buried and take too long to get to (Figures 5, 9). Most of the earlier figures serve just to validate the toy task, and could be de-emphasized.
It is still not clear to me when compositional generalization is possible, and whether it can be predicted just from schema combination. Are there graph operations on schema that would explain when generalization is possible and when it fails?
Questions for authors
I think the authors would do well to interpret more of their results in the context of the schema model – what do these simple rules reveal about the structure of compositional generalization?
How does the schema model relate to prior models of compositional generalization?
Could the authors expand the discussion of the results presented?
2 comments
Response to Reviewer fPYL (Part 2/2)
Q2: Discussion with Prior Models of Compositional Generalization
Our two-stage setup fixes Stage-1 single-primitive training and varies only the source graph set in RL, letting us test structural transfer directly against earlier compositional setups (references below):
| Line of Work | Structure | Training Setup | Relation to Our Schema Model |
|---|---|---|---|
| Grammar & Diverse SFT | Parse trees [1] & CFQ [2] | Supervised SFT [3] | Maps graph recombination to systematicity and scaling to productivity [4]; CoT SFT scores 18.4% vs. GRPO 45.2%. |
| 1D Sequential Chains | 1D chains | 2-step [5] or chain RL [6] | Extends 1D chains to 2D and merge DAGs; our 7-chain control (3.1%) does not separate branches. |
| Expression-Tree RL | Countdown trees | Tree-subset RL [7] | Separates chaining, branching, and merge schemas across 29 grid and spine sets plus . |
Unlike semantic parsing in SCAN [1] and CFQ [2] or supervised SFT in Oren et al. [3], we train single primitives in Stage 1 first and vary under on-policy RL. In the terms of Hupkes et al. [4], combining chaining, branching, and merge schemas ( and ) tests systematicity, and scaling from to calls tests productivity. Varying primitive weights also changes primitive difficulty independently of graph structure. While Yuan et al. [5], Xu et al. [6], and Park et al. [7] study 1D chains or Countdown trees without separating depth, branching, and merging, our schema split examines all three axes.
Q3: Regime Synthesis on Controlled Schemas and Real-World Tool Planning
Our controlled and real-world results follow a clear split. Primitive difficulty drives accuracy inside the direct-training frontier, while training graph structure drives zero-shot transfer beyond it.
Across the ten inside-the-frontier schemas spanning –, sampling Unimodal rather than Barbell primitives improves accuracy by +10.2 points on average and up to +14.2 points per schema. Beyond the frontier, varying primitive weights across all seven policies shifts accuracy relative to Uniform by at most 1.2 points and brings no extra schemas above 4%.
Separating primitive slips from routing errors explains 's advantage over SFT and target-grid RL (Global Response Q1). Joint training on cuts routing errors to 13.6% (vs. 37.6%–60.1%) and reaches 86.3% of our 52.4% oracle ceiling (92.9% with warm-up), while adding lifts unseen merge DAGs to 43.8% (Global Response Q2). The same pattern holds when prompts give only a natural-language request and a flat tool list. Across eight controlled tool catalogues, gains +2.4 to +18.2 points on the nine unseen target grids. On the frozen suites in Global Response Q3, our checkpoints beat direct training on every in-domain official suite (+8.3 on FuncBG, +8.1 on NESTFUL, +6.1 on TaskB). They also raise the 8-suite mean to 32.6%–33.7%, vs. 31.5%–31.9% for direct training.
[1] B. M. Lake and M. Baroni. Generalization without Systematicity: On the Compositional Skills of Sequence-to-Sequence Recurrent Networks. ICML'18.
[2] D. Keysers et al. Measuring Compositional Generalization: A Comprehensive Method on Realistic Data. ICLR'20.
[3] I. Oren et al. Finding Needles in a Haystack: Sampling Structurally-Diverse Training Sets from Synthetic Data for Compositional Generalization. EMNLP'21.
[4] D. Hupkes et al. Compositionality Decomposed: How Do Neural Networks Generalise? JAIR'20.
[5] L. Yuan et al. From and to : LLMs Learn New Skills in RL by Composing Old Ones. ICLR'26.
[6] X. Xu et al. Composition-RL: Compose Your Verifiable Prompts for Reinforcement Learning of Large Language Models. arXiv'26.
[7] S. Park et al. How Does RL Post-Training Induce Skill Composition? A Case Study on Countdown. arXiv'25.
Response to Reviewer fPYL (Part 1/2)
We sincerely thank Reviewer fPYL for the helpful review and positive comments on our schema setup and out-of-distribution generalization results.
W1: Visual Schema Notation
We agree with this suggestion. In the revised paper, we introduce the algebraic G[D1,…,DB] notation only in the Section 2 definition, and use inline diagrams across all subsequent text and tables in the main paper so each structure is clear at a glance.
W2: Pacing and Presentation of Main Generalization Results
Thank you for this suggestion on pacing. To reach the main generalization findings sooner, we shortened the training and evaluation description in Section 3.2 and moved the full objective equations to Appendix B.2 with the detailed evaluation protocol. We also tightened Section 4, moved schema combination into Section 5, and summarized our cross-axis, merge-DAG, error-breakdown, and tool-planning results up front in Section 1.
W3: Graph Operations on Schemas and Coverage Across Combinations
Thank you for suggesting that we frame schema combinations around explicit graph operations. From the single-step graph G[1], our source sets cover three graph operations. These are sequential chaining along a branch (a1={G[1],…,G[4]}), parallel branching from shared input x with ordered assembly (a2={G[1],…,G[1,1,1,1]}), and binary merging from J[1,1;1] for diamond D, funnel F, and cross-linked X DAGs. Transfer from S improves when S covers the target's graph operations and includes intermediate schemas that keep mixed-reward activity Act(g)≥2. Below is the structural summary across operations (full baselines and ceilings appear in Global Response Q1–Q2, where
--marks a merge-only run):Both parts matter in our ablations. Omitting branching in 7-chain c1 gives only 3.1% on 2D grids, and replacing intermediate G[2,2] with G[1] in s2→s18 drops 20-cell accuracy from 35.3% to 8.3%. Across all 29 source combinations, Active Level Coverage C∈[0,1] tracks 9-cell interior grid accuracy at Spearman ρs=0.94 and Pearson r=0.91.
Q1: Interpreting Our Results in the Context of the Schema Model
In the schema model, the depth and branching axes correspond directly to step-to-step propagation and branch-root resetting on unseen grids like G[4,4,4,4]. Full 9-grid accuracy, oracle-ceiling ratios, and route-free rates for all SFT and target-grid RL baselines appear in the Global Response Q1 table.
On G[4,4,4,4], single-axis policies fail along opposite dimensions. Depth-only a1 drops across branches (b=1→4 falls 88.2, d=1→4 stays 34.6, step mean 29.0%) by carrying the previous branch forward. Branch-only a2 drops across depth (d=1→4 falls 91.2, b=1→4 stays 30.1, step mean 29.1%) by resetting to
x. On a3, CoT SFT still drops across both axes (b=1→4 falls 74.5, d=1→4 falls 88.4, step mean 62.8%). In contrast, GRPO on a3 keeps both skills (b=1→4 holds 86.4, d=1→4 holds 93.6, step mean 83.9%) because joint optimization rules out both failure modes.