acceptodds
Under review as a conference paper at ICLR 2027

Familiar Operations, Unseen Structures: How Task Compositions Affect Task Generalization?

Abstract

Large language models are trained on data from various tasks and can generalize to unseen test tasks by reusing knowledge from training. Different training tasks demonstrate different ways to compose the knowledge pieces, and thus are central in shaping both the knowledge pieces a model acquires and the ways it learns to reuse them. Yet how different compositions in training affect generalization to unseen tasks remains elusive. We thus introduce schema-based data composition, a methodology that constructs structurally diverse tasks from a small library of reusable operations (i.e., primitives) according to predefined composition structures (i.e., schemas). A schema specifies dependencies and output assembly independently of the particular primitives and inputs; executing an instantiated schema provides its reference output. This representation allows us to study how the structural content of training data shapes generalization. Our empirical study yields three key insights. (1) Topology shapes learnability. Schemas of the same size can differ substantially in difficulty, and the tested primitive-sampling policies do not extend the observed direct-training frontier. (2) Structural complementarity enables transfer. Training on complementary forms of depth and branching enables transfer to unseen structures. It can even outperform direct training on those structures with the same prompt budget per run. (3) Source learnability matters. We observe stronger transfer when complementary training schemas continue to provide learning signals. Including intermediate schemas can also help the model begin learning from harder schemas. These findings identify computational structure as a distinct dimension of training-data design for generalization to new tasks.

Primary area:
transfer learning, meta learning, and lifelong learning
Keywords:
compositional generalization, task generalization, large language models

Reviews

Among:
Shape:
1234this paper, Top 49%
Figure 1. Mean review rating of the reviewed papers at ICLR 2027. This paper averages 2.33 over 3 reviews, above 51% of the others.
Reviewer_fPYLOpenReview ↗
Rating
2
Confidence
3
Informativeness
2
Presentation
2
Soundness
3
Read review
Paper summary

The authors study compositional generalization in the setting of an artificial task, and analyze how different compositional structures seen during training can contribute to the model’s generalization performance.

Critical strengths

I think the schema model is interesting, and a nice way to study compositional generalization.

Figures 5 and 9 are particularly interesting, and shed light on the out-of-distribution generalization abilities of transformers.

Critical weaknesses

While the schema model is interesting, understanding the notation takes more work than it should, particularly because of the mixed use of the diagrams and the G[*] notation. If possible, I would suggest reworking the notation to just use the diagrams.

I think the most interesting results in the paper are a bit buried and take too long to get to (Figures 5, 9). Most of the earlier figures serve just to validate the toy task, and could be de-emphasized.

It is still not clear to me when compositional generalization is possible, and whether it can be predicted just from schema combination. Are there graph operations on schema that would explain when generalization is possible and when it fails?

Questions for authors

I think the authors would do well to interpret more of their results in the context of the schema model – what do these simple rules reveal about the structure of compositional generalization?

How does the schema model relate to prior models of compositional generalization?

Could the authors expand the discussion of the results presented?

2 comments
AuthorsOpenReview ↗

Response to Reviewer fPYL (Part 1/2)

We sincerely thank Reviewer fPYL for the helpful review and positive comments on our schema setup and out-of-distribution generalization results.

W1: Visual Schema Notation

We agree with this suggestion. In the revised paper, we introduce the algebraic G[D1,…,DB]G[D_1, \dots, D_B] notation only in the Section 2 definition, and use inline diagrams across all subsequent text and tables in the main paper so each structure is clear at a glance.

W2: Pacing and Presentation of Main Generalization Results

Thank you for this suggestion on pacing. To reach the main generalization findings sooner, we shortened the training and evaluation description in Section 3.2 and moved the full objective equations to Appendix B.2 with the detailed evaluation protocol. We also tightened Section 4, moved schema combination into Section 5, and summarized our cross-axis, merge-DAG, error-breakdown, and tool-planning results up front in Section 1.

W3: Graph Operations on Schemas and Coverage Across Combinations

Thank you for suggesting that we frame schema combinations around explicit graph operations. From the single-step graph G[1]G[1], our source sets cover three graph operations. These are sequential chaining along a branch (a1={G[1],…,G[4]}a1 = \lbrace G[1], \dots, G[4] \rbrace), parallel branching from shared input xx with ordered assembly (a2={G[1],…,G[1,1,1,1]}a2 = \lbrace G[1], \dots, G[1,1,1,1] \rbrace), and binary merging from J[1,1;1]J[1,1;1] for diamond DD, funnel FF, and cross-linked XX DAGs. Transfer from SS improves when SS covers the target's graph operations and includes intermediate schemas that keep mixed-reward activity Act(g)≥2\text{Act}(g) \ge 2\\%. Below is the structural summary across operations (full baselines and ceilings appear in Global Response Q1–Q2, where -- marks a merge-only run):

Training Source Set SSGraph Operations Covered9-Grid Acc. (%)Asym. Grid (%)Merge DAGs (%)
Direct Target GRPOEach target, separate9.510.48.9
a1a1: G[1]…G[4]G[1] \dots G[4]Chaining only0.71.21.2
a2a2: G[1]…G[1,1,1,1]G[1] \dots G[1,1,1,1]Branching only8.610.84.8
a3=a1∪a2a3 = a1 \cup a2Chaining + Branching45.247.619.6
a3∪{J[1,1;1]}a3 \cup \lbrace J[1,1;1] \rbraceChaining + Branching + Merge----43.8

Both parts matter in our ablations. Omitting branching in 7-chain c1c1 gives only 3.1% on 2D grids, and replacing intermediate G[2,2]G[2,2] with G[1]G[1] in s2→s18s2 \to s18 drops 20-cell accuracy from 35.3% to 8.3%. Across all 29 source combinations, Active Level Coverage C∈[0,1]C \in [0, 1] tracks 9-cell interior grid accuracy at Spearman ρs=0.94\rho_s = 0.94 and Pearson r=0.91r = 0.91.

Q1: Interpreting Our Results in the Context of the Schema Model

In the schema model, the depth and branching axes correspond directly to step-to-step propagation and branch-root resetting on unseen grids like G[4,4,4,4]G[4,4,4,4]. Full 9-grid accuracy, oracle-ceiling ratios, and route-free rates for all SFT and target-grid RL baselines appear in the Global Response Q1 table.

On G[4,4,4,4]G[4,4,4,4], single-axis policies fail along opposite dimensions. Depth-only a1a1 drops across branches (b=1→4b=1 \to 4 falls 88.288.2\\% \to 2.3\\%, d=1→4d=1 \to 4 stays 34.634.6\\% \to 24.5\\%, step mean 29.0%) by carrying the previous branch forward. Branch-only a2a2 drops across depth (d=1→4d=1 \to 4 falls 91.291.2\\% \to 1.8\\%, b=1→4b=1 \to 4 stays 30.130.1\\% \to 28.1\\%, step mean 29.1%) by resetting to x. On a3a3, CoT SFT still drops across both axes (b=1→4b=1 \to 4 falls 74.574.5\\% \to 49.1\\%, d=1→4d=1 \to 4 falls 88.488.4\\% \to 38.8\\%, step mean 62.8%). In contrast, GRPO on a3a3 keeps both skills (b=1→4b=1 \to 4 holds 86.486.4\\% \to 81.2\\%, d=1→4d=1 \to 4 holds 93.693.6\\% \to 73.3\\%, step mean 83.9%) because joint optimization rules out both failure modes.

AuthorsOpenReview ↗

Response to Reviewer fPYL (Part 2/2)

Q2: Discussion with Prior Models of Compositional Generalization

Our two-stage setup fixes Stage-1 single-primitive training and varies only the source graph set SS in RL, letting us test structural transfer directly against earlier compositional setups (references below):

Line of WorkStructureTraining SetupRelation to Our Schema Model
Grammar & Diverse SFTParse trees [1] & CFQ [2]Supervised SFT [3]Maps graph recombination to systematicity and n≤4→16n \le 4 \to 16 scaling to productivity [4]; CoT SFT scores 18.4% vs. GRPO 45.2%.
1D Sequential Chains1D chains fD(⋯f1(x))f_D(\cdots f_1(x))2-step [5] or chain RL [6]Extends 1D chains to 2D and merge DAGs; our 7-chain control c1c1 (3.1%) does not separate branches.
Expression-Tree RLCountdown treesTree-subset RL [7]Separates chaining, branching, and merge schemas across 29 grid and spine sets SS plus J[1,1;1]J[1,1;1].

Unlike semantic parsing in SCAN [1] and CFQ [2] or supervised SFT in Oren et al. [3], we train single primitives in Stage 1 first and vary SS under on-policy RL. In the terms of Hupkes et al. [4], combining chaining, branching, and merge schemas (a1∪a2a1 \cup a2 and +J[1,1;1]+J[1,1;1]) tests systematicity, and scaling from n≤4n \le 4 to n=16n = 16 calls tests productivity. Varying primitive weights a∼νga \sim \nu_g also changes primitive difficulty independently of graph structure. While Yuan et al. [5], Xu et al. [6], and Park et al. [7] study 1D chains or Countdown trees without separating depth, branching, and merging, our schema split examines all three axes.

Q3: Regime Synthesis on Controlled Schemas and Real-World Tool Planning

Our controlled and real-world results follow a clear split. Primitive difficulty drives accuracy inside the direct-training frontier, while training graph structure drives zero-shot transfer beyond it.

Across the ten inside-the-frontier schemas spanning n=2n = 2–66, sampling Unimodal rather than Barbell primitives improves accuracy by +10.2 points on average and up to +14.2 points per schema. Beyond the frontier, varying primitive weights across all seven policies shifts accuracy relative to Uniform by at most 1.2 points and brings no extra schemas above 4%.

Separating primitive slips from routing errors explains a3a3's advantage over SFT and target-grid RL (Global Response Q1). Joint training on a3a3 cuts routing errors to 13.6% (vs. 37.6%–60.1%) and reaches 86.3% of our 52.4% oracle ceiling (92.9% with warm-up), while adding J[1,1;1]J[1,1;1] lifts unseen merge DAGs to 43.8% (Global Response Q2). The same pattern holds when prompts give only a natural-language request and a flat tool list. Across eight controlled tool catalogues, a3a3 gains +2.4 to +18.2 points on the nine unseen target grids. On the frozen suites in Global Response Q3, our checkpoints beat direct training on every in-domain official suite (+8.3 on FuncBG, +8.1 on NESTFUL, +6.1 on TaskB). They also raise the 8-suite mean to 32.6%–33.7%, vs. 31.5%–31.9% for direct training.

[1] B. M. Lake and M. Baroni. Generalization without Systematicity: On the Compositional Skills of Sequence-to-Sequence Recurrent Networks. ICML'18.

[2] D. Keysers et al. Measuring Compositional Generalization: A Comprehensive Method on Realistic Data. ICLR'20.

[3] I. Oren et al. Finding Needles in a Haystack: Sampling Structurally-Diverse Training Sets from Synthetic Data for Compositional Generalization. EMNLP'21.

[4] D. Hupkes et al. Compositionality Decomposed: How Do Neural Networks Generalise? JAIR'20.

[5] L. Yuan et al. From f(x)f(x) and g(x)g(x) to f(g(x))f(g(x)): LLMs Learn New Skills in RL by Composing Old Ones. ICLR'26.

[6] X. Xu et al. Composition-RL: Compose Your Verifiable Prompts for Reinforcement Learning of Large Language Models. arXiv'26.

[7] S. Park et al. How Does RL Post-Training Induce Skill Composition? A Case Study on Countdown. arXiv'25.

Reviewer_YTM8OpenReview ↗
Rating
2
Confidence
2
Informativeness
2
Presentation
3
Soundness
3
Read review
Paper summary

The paper studies which task compositions a model should be trained on to generalize to tasks with unseen structures. Tasks are built from 25 string operations arranged into parallel branches of operation chains. After learning single operations via SFT, Llama-3.1-8B is trained with GRPO on selected structures. Training jointly on chains and parallel single-operation tasks transfers to unseen structures combining depth and branching, outperforming direct training on those structures (45.2% vs. 9.5%). Results are validated on tool-planning tasks built from eight tool-use benchmarks.

Critical strengths

The paper asks a clear question and studies it with a well-controlled design, varying only the structure of the training data while holding everything else fixed. The finding that mixing simpler, complementary structures can outperform direct training on harder ones is practically useful, and demonstrates it with nice figures.

Critical weaknesses

The task structure is given explicitly to the model (the full program in the prompt), so “generalizing to unseen structures” means executing a program of a new shape. The structures are also narrow (independent chains concatenated together), and the held-out targets lie within the depth and branch ranges already seen in training. Given this, it is fairly intuitive at a high level that training on chains and parallel branches transfers to structures combining both, which makes the result less surprising from this perspective.

Even if one views the held-out structures as out-of-distribution, the paper does not explain why mixing parallel and non-parallel structures works. The controls identify which properties of the training set matter (covering both depth and branching, and sources remaining learnable), but not what the model actually learns from them, and the activation analysis only shows that training structures become learnable before held-out accuracy improves. Likewise, the learning-signal account explains why direct training on hard structures fails, but not why training on other structures yields accuracy on held-out targets that receive no training signal at all. For instance, the bottleneck may lie in tracking many intermediate results rather than in composition itself, with branching structures teaching the former and chains the latter; an error analysis on the held-out structures (e.g., which branches fail, or whether errors grow with depth or width) could distinguish such explanations. The current results therefore support a very narrow finding, and it is not clear what the general scientific lesson is.

Questions for authors

N/A

4 comments
AuthorsOpenReview ↗

Response to Reviewer YTM8 (Part 1/2)

Thank you for the positive comments on our experimental design, figures, and presentation. Below we address each question.

W1: Explicit Programs vs. Implicit Structure Inference in Real-World Tool Planning

In synthetic string tasks, prompts include main_solution(x) so each node fixes one target operation. Across all eight controlled tool-planning catalogues, prompts omit executable code and graph descriptions, giving only a natural-language request and a flat tool list.

For example, suppose a user asks for the cheapest nonstop Paris-to-Berlin flight and the cheapest refundable Berlin hotel. From that request alone, the model must infer two 3-step branches, pick the right search, filter, and ranking tools, and link intermediate bindings ($v1.options through $v3.id | $v6.id) without an explicit program. Joint training on a3a3 raises accuracy on unseen schemas from 0.10%–0.24% under direct training to 15.4% on FuncBenchGen and 12.9% on API-Bank, gaining +2.4 to +18.2 points across the nine unseen grids in all eight tool catalogues.

As detailed in Global Response Q3, zero-shot performance across five unmodified official suites and three external suites also beats every in-domain direct-training baseline and raises the 8-suite average to 32.6%–33.7% (vs. 31.5%–31.9% for direct training, 31.2% for atomic tool SFT, and 29.7% for string-schema RL).

W2a: Scale Extrapolation Within and Beyond the Grid

Within B,D∈{2,3,4}B, D \in \lbrace 2, 3, 4 \rbrace, training on a3a3 (n≤4n \le 4 calls) already tests up to 4×4\times extrapolation in operation count on G[4,4,4,4]G[4,4,4,4] (n=16n = 16). As reported in Global Response Q1, a3a3 reaches 83.9% local step accuracy on G[4,4,4,4]G[4,4,4,4] and 86.4% route-free execution across all nine unseen grids. Its 45.2% exact match (48.7% with warm-up) beats CoT SFT (18.4%), Direct-Answer SFT (3.8%), target-grid RL baselines (9.5%–31.4%), and our 7-chain control c1c1 (3.1%).

Beyond uniform B,D∈{2,3,4}B, D \in \lbrace 2, 3, 4 \rbrace, a3a3 reaches 47.6% (50.8% with CoT SFT warm-up) vs. 10.4% direct across six unseen unequal-depth schemas (G[3,2]G[3,2], G[2,3]G[2,3], G[4,2]G[4,2], G[4,3,2]G[4,3,2], G[4,4,3]G[4,4,3], G[4,3,2,1]G[4,3,2,1]). Unequal-depth source sets s3s3–s5s5 also bring 16 to 17 of 20 uniform schemas above 4%. On the 5-branch schema G[1,1,1,1,1]G[1,1,1,1,1], a3a3 reaches 6% and spine s1s1 reaches 11%, vs. 0.0% under direct training.

W2b: Non-Parallel Mid-Computation Merge Topologies

Thank you for suggesting a direct evaluation on mid-computation merge topologies, which we added to Sections 5.4–5.5 and Appendix B.6 (Table 13). All merge runs start from a shared Stage-1 checkpoint with five added binary string primitives and index dpostd _ {\text{post}} from the binary merge step onward. We tested six unseen merge DAGs across Diamonds D[2,2;2]D[2,2;2] and D[3,3;2]D[3,3;2] (dpost=2d _ {\text{post}} = 2), Funnels F[3,2;2]F[3,2;2] and F[4,2;2]F[4,2;2], and Cross-Linked DAGs X[3,2]X[3,2] and X[3,3]X[3,3] merging step 1 of branch bb into step 2 of branch b+1b + 1 (b<3b < 3). In the Global Response Q2 table, a3a3 doubles mean accuracy over direct training (19.6% vs. 8.9%). Adding the 3-node merge schema J[1,1;1]J[1,1;1] raises mean accuracy to 43.8% (46.9% with warm-up), reaching 35.1% vs. 0.0% zero-shot on 4-branch funnels F[4,2;2]F[4,2;2].

AuthorsOpenReview ↗

Response to Reviewer YTM8 (Part 2/2)

W3: What Mixing Parallel and Sequential Schemas Teaches and Step-Level Error Analysis

We agree with this intuition and added a node-by-node analysis across all 16 operations of G[4,4,4,4]G[4,4,4,4] in Section 5.5 (Figures 8–9a and Table 12). Below, we report local per-step accuracy by branch b∈{1,2,3,4}b \in \lbrace 1, 2, 3, 4 \rbrace and depth d∈{1,2,3,4}d \in \lbrace 1, 2, 3, 4 \rbrace on G[4,4,4,4]G[4,4,4,4] in % (9-grid accuracy, route-free rates, and routing errors appear in Global Response Q1):

Stage-2 Methodb=1b=1b=2b=2b=3b=3b=4b=4d=1d=1d=2d=2d=3d=3d=4d=4G[4,4,4,4]G[4,4,4,4] Mean
a1a1: Depth Axis88.219.46.12.334.629.827.124.529.0%
a2a2: Branch Axis30.129.428.828.191.218.45.01.829.1%
CoT SFT on a3a374.568.259.449.188.469.254.838.862.8%
GRPO on a3a3, Ours86.484.983.181.293.687.281.573.383.9%

The two single-axis runs fail in opposite ways. Training on a1a1 alone keeps 88.2% step accuracy along branch 1 with 4.1% within-branch resets, but drops to 19.4%, 6.1%, and 2.3% on branches b=2,3,4b = 2, 3, 4. Its traces carry branch 1's last value (such as t3 on G[4,4,4,4]G[4,4,4,4]) into branch 2 instead of resetting to x, giving 64.8% cross-branch leaks. In contrast, training on a2a2 alone reaches 91.2% across all four branch roots at d=1d = 1 with 5.3% cross-branch leaks, but drops to 18.4%, 5.0%, and 1.8% across d=2,3,4d = 2, 3, 4 as deeper calls reset to x, giving 76.4% within-branch resets.

Joint training on a3=a1∪a2a3 = a1 \cup a2 generalizes zero-shot because one shared policy prevents both failure modes at once. Carrying the previous branch forward fails on all three multi-branch schemas in a2a2, while resetting deeper calls to x fails on all three multi-step chains in a1a1. As a result, the policy keeps both step-to-step propagation and branch-root resetting on unseen B×DB \times D grids (83.9% step accuracy and 86.4% route-free execution).

While the premise of this paper is promising, to be frank, the style of writing in the paper and the rebuttal is incredibly detail-oriented which makes this a bit hard to read and engage with. There are confusing details such as "oracle ceiling"; what is that and why is it relevant? One concern I have is how this work distinguish itself from existing works such as https://arxiv.org/pdf/2512.01775 https://arxiv.org/pdf/2509.25123 https://arxiv.org/pdf/2609.19465 https://arxiv.org/pdf/2505.23683.

Such clarifications and writing improvements would make this paper significantly easier to read and engage with.

Thank you for your constructive feedback! In our previous response, we aimed to provide complete experimental details to be thorough; below, we step back to highlight the high-level intuition, clarify why the oracle ceiling is necessary, and distinguish our work from the four papers you mentioned.

Q1. High-level summary of our core idea

In summary we ask a simple question: once a model already knows a set of basic operations, which training task structures best help it compose those operations in an unseen way under RL? Training directly on complex multi-branch tasks often collapses to 9.5% accuracy because sparse rewards rarely produce successful rollouts. Instead, we show that training jointly on two simple, learnable structures—pure chains (which teach carrying a value from one step to the next) and pure parallel branches enables zero-shot transfer to unseen 2D tasks combining both, reaching 45.2% accuracy under the same prompt budget.

Q2. "oracle ceiling" and why it is necessary

Each unseen 2D test task executes 4 to 16 individual string operations, and any single slip makes the final exact match wrong. Because individual operations are not 100% accurate on their own, small single-step errors naturally compound across 4 to 16 steps. The oracle ceiling measures the exact-match accuracy the model would achieve if routing between steps were 100% error-free, so even a model with perfect structural generalization caps out at 52.4%, not 100%.

This reference is necessary to separate structural composition errors from compounding single-step operation slips. Without it, 45.2% raw accuracy can look modest on its own; relative to the 52.4% ceiling, however, our model actually reaches 86.3% of the achievable upper bound (and 92.9% with a brief SFT warm-up).

This directly confirms that the model has largely mastered composing the unseen structures, and most remaining errors are single-operation slips rather than routing failures. In the final version, we will include this intuitive one-sentence explanation right where the oracle ceiling first appears, along with its exact calculation protocol in Appendix B.5.

Q3. How this work differs from the four cited papers

All four studies investigate compositional learning, whereas our paper isolates multi-dimensional task topology (depth, parallel branching, and merging) as an explicit training-data design axis under RL:

  • He et al. [1] compare RL on individual atomic steps versus complete composed tasks (including branching-and-merging tasks), showing that practicing single steps transfers poorly to whole tasks while training on whole tasks transfers back to steps. We address the harder regime where training directly on the complete target task also fails under RL, and ask which combination of simpler multi-step structures enables zero-shot transfer to a different, unseen target structure.
  • Park et al. [2] (we cited in Section 1 and Appendix A) study Countdown puzzles where a model searches for an arithmetic expression tree to reach a target number, and analyze which tree shapes are easier or harder to learn. We decouple task topology from domain semantics so we can systematically control which structures appear in training, diagnose the opposite routing errors caused by depth-only versus branch-only training, and validate the same structural principle on natural-language tool planning.
  • Yuan et al. [3] (we cited in Sections 1 and 3) show that RL on 2-step sequential compositions generalizes to deeper 1D chains and across tasks. We build on their 25 string operations as a controlled base and move beyond 1D chains to multi-branch and merging topologies, showing that longer chains alone fail on 2D tasks unless paired with parallel branches.
  • Wang et al. [4] provide a theoretical analysis of supervised gradient descent on transformers learning 1D permutation chains, proving when mixing shorter and longer examples (easy-to-hard data) makes a long chain learnable. We study RL post-training of pretrained LLMs on unseen 2D branching and merge topologies that never appear in training, and show that combining orthogonal simple structures outperforms even an easy-to-hard curriculum trained directly on the targets.

We appreciate your helpful suggestions and hope these clarifications make our contribution and positioning clear.

[1] Y. He et al. Compositional Reasoning in Language Models under Reinforcement Learning Post-Training. NeurIPS'26 (arXiv:2609.19465).

[2] S. Park et al. How Does RL Post-Training Induce Skill Composition? A Case Study on Countdown. NeurIPS'26 (arXiv:2512.01775).

[3] L. Yuan et al. From f(x)f(x) and g(x)g(x) to f(g(x))f(g(x)): LLMs Learn New Skills in RL by Composing Old Ones. ICLR'26 (arXiv:2509.25123).

[4] Z. Wang et al. Learning Compositional Functions with Transformers from Easy-to-Hard Data. COLT'25 (arXiv:2505.23683).

Reviewer_VZfqOpenReview ↗
Rating
3(+1)
Confidence
3
Informativeness
2
Presentation
2
Soundness
3
Read review
Paper summary

The paper studies how the choice of training-task structures during RL post-training affects compositional generalization in LLMs.

The authors first use SFT to teach a model individual primitive operations, then apply GRPO to train on different compositions of these operations, including sequential chains and parallel branches.

Their main finding is that jointly training on chains and branches improves generalization to unseen structures combining both, achieving 45.2% accuracy compared to 9.5% when training directly on the target structures.

They attribute this improvement partly to the sparse-reward limitations of GRPO, as simpler and complementary tasks provide more informative learning signals.

They further show that intermediate-difficulty schemas can help harder tasks become trainable. Similar trends are observed on synthetically constructed tool-use tasks derived from eight existing benchmarks.

Critical strengths

1. Strong empirical evidence for the benefit of complementary training structures under GRPO:

The paper shows that jointly training on chains and parallel branches during GRPO substantially improves generalization to unseen compositions (45.2% vs. 9.5% for direct GRPO training). This is interesting because the model never encounters these combined structures during training.

2. Interesting finding on the role of intermediate compositions in RL training: The authors show that including intermediate-complexity schemas can substantially improve learning on harder compositions. Their controlled comparisons suggest that the choice and progression of training structures matter for enabling effective GRPO training, providing useful insights.

Critical weaknesses

1. Unclear positioning and novelty of the main contributions

The paper poses an interesting research question, but I found its main message somewhat scattered and insufficiently emphasized. Several stated contributions overlap with prior work: Yuan et al. (2025) study learning compositions of known primitives through RL, Park et al. (2025) investigate topology-dependent learnability and generalization to unseen structures, and Oren et al. (2021) study how structural diversity in synthetic training data improves compositional generalization.

The main novelty appears to be showing how jointly training on chains and parallel branches during GRPO improves generalization to unseen structures. I think the authors should foreground this finding and more clearly explain what new insight it provides beyond prior work.

References

  1. Yuan, L., et al. (2025). From f(x) and g(x) to f(g(x)): LLMs Learn New Skills in RL by Composing Old Ones. ICLR 2026.
  2. Park, S., Kaur, S., & Arora, S. (2025). How Does RL Post-training Induce Skill Composition? A Case Study on Countdown. arXiv:2512.01775.
  3. Oren, I., Herzig, J., & Berant, J. (2021). Finding Needles in a Haystack: Sampling Structurally-diverse Training Sets from Synthetic Data for Compositional Generalization. EMNLP 2021.

2. Missing SFT baseline:

The paper studies how different training structures affect compositional generalization under GRPO, but does not compare against SFT on the same compositions. Since their framework can automatically generate ground-truth outputs, it would be natural to compare SFT on chains only, branches only, and their mixture. Without this comparison, it is unclear whether the findings reflect a general property of compositional learning or are specific to RL training.

3. Interpretation of the main generalization result:

The finding that jointly training on chains and branches improves generalization to structures requiring both is somewhat expected, since the test tasks involve both structural patterns.

The more interesting result is that this mixture outperforms direct GRPO training on the target structures. However, as the authors acknowledge, direct training suffers from sparse rewards because complex tasks rarely produce successful rollouts.

It is therefore unclear whether the improvement primarily reflects overcoming GRPO's optimization bottleneck rather than a new insight about compositional generalization. Comparisons with alternative training strategies, such as curriculum-based RL or SFT, would help clarify this distinction.

4. Limited analysis of the remaining generalization errors:

Although complementary training improves performance substantially, the mean accuracy on unseen schemas remains at 45.2%. The authors investigate primitive difficulty and topology-dependent learnability, but it remains unclear how much of the remaining error is attributable to imperfect primitive execution versus failures of structural composition.

A more direct error decomposition would help clarify how much compositional capability the model actually acquires.

After all, 45% is not really convincing enough to claim that the model has learned compositional structure.

5. Missing relevant citations:

I believe the related work discussion would benefit from including the following papers:

These works should be discussed to better distinguish the paper's contributions from existing research on structural diversity, compositional task generalization, and RL training.

Questions for authors

Please see Critical Weaknesses.

6 comments
AuthorsOpenReview ↗

Response to Reviewer VZfq (Part 1/3)

We thank Reviewer VZfq for the careful review and positive comments on cross-axis transfer under GRPO and the role of intermediate schemas.

W1: Discussion with Previous Work and Summary of Core Contributions

Thank you for these pointers on recent compositional studies, which we added to Section 1.

StudyControlled Source Set SSRL Post-TrainingMulti-Branch & Merge DAGsCross-Axis TransferZero-Variance FrontierImplicit Tool PlanningKey Empirical Distinction
Yuan et al. [1]PartialYes--------Studies 2-step f(g(x))f(g(x)) RL training along nested chains; no multi-branch or mid-DAG merge source sets.
Abedsoltan et al. [2]PartialNo--------Analyzes autoregressive generalization from O~(D)\tilde{O}(D) tasks to DTD^T length-TT compositions under ICL/CoT; no RL graph-axis sets.
Composition-RL [3]PartialYes----Yes--Chains verifiable prompts sequentially in 1D; our 7-chain control c1c1 reaches 3.1% on 2D grids vs. 45.2% for a3a3.
Oren et al. [4]YesNoPartial------Samples structurally diverse examples for SFT; our matched-budget CoT SFT on a3a3 reaches 18.4% vs. 45.2% for GRPO.
Park et al. [5]PartialYesPartial------Ablates Countdown expression-tree subsets without separating depth, branching, and mid-DAG merge schemas.
OursYesYesYesYesYesYesTests 29 grid/spine sets plus merge source J[1,1;1]J[1,1;1], reaching 45.2% on 2D grids and 43.8% on merge DAGs, with gains across 8 tool suites.

Our experiments show that combining chaining and branching schemas solves unseen 2D and merge DAGs where 1D sequential scaling falls short. Building on the 25 deterministic string primitives of Yuan et al. [1] keeps single-operation initialization fixed across Stage-2 grid runs (with a shared augmented checkpoint for binary merges). While Abedsoltan et al. [2] and Xu et al. [3] study autoregressive or sequential 1D chains, our count-matched 7-chain control c1c1 trains up to depth 7 and reaches only 3.1% on the nine unseen 2D grids. In contrast, combining a1a1 and a2a2 in a3a3 reaches 45.2%, and adding J[1,1;1]J[1,1;1] raises mean accuracy on six unseen merge DAGs to 43.8% vs. 8.9% under direct training (Global Response Q1–Q2).

For diverse SFT as in Oren et al. [4], our matched-budget CoT SFT on a3a3 reaches 18.4%, vs. 45.2% under on-policy GRPO (48.7% with warm-up). Beyond the Countdown tree ablations in Park et al. [5], we vary 29 grid and spine source sets plus J[1,1;1]J[1,1;1] across chaining, branching, and merge operations. Across these 29 sets, Active Level Coverage CC tracks 9-cell interior accuracy at Spearman ρs=0.94\rho_s = 0.94. Training on a3a3 also raises accuracy across eight controlled tool catalogues by +2.4 to +18.2 points and across all eight frozen suites in Global Response Q3.

W2: Stage-2 SFT Baselines on Chains a1a1, Branches a2a2, and Combined Axes a3a3

Thank you for suggesting this direct comparison. We added it to Section 5.4 (Table 2). The full Stage-2 SFT vs. GRPO results under the matched 49,984-prompt per-run budget (three seeds) appear in the Global Response Q1 table. CoT SFT still benefits from axis combination on a3a3 at 18.4% (vs. 1.1% on a1a1, 6.4% on a2a2, and 3.8% for Direct-Answer SFT), yet on-policy GRPO on a3a3 adds +26.8 points to reach 45.2% (48.7% with CoT SFT warm-up). On G[4,4,4,4]G[4,4,4,4], CoT SFT drops from 88.4% at d=1d=1 to 38.8% at d=4d=4, while GRPO keeps step accuracy at 93.6% (d=1d=1) and 73.3% (d=4d=4).

The same pattern holds on the six unseen unequal-depth schemas (Asym.) and the six merge DAGs (Merge, with J[1,1;1]J[1,1;1] for GRPO on a3a3). Across unequal-depth schemas, a3a3 GRPO reaches 47.6% (50.8% with warm-up) vs. 20.1% for CoT SFT on a3a3, 4.5% for Direct-Answer SFT, 1.2%–1.6% on a1a1, and 7.9%–10.8% on a2a2. On merge DAGs, a3∪{J[1,1;1]}a3 \cup \lbrace J[1,1;1] \rbrace GRPO reaches 43.8% (46.9% with warm-up) vs. 19.8% for CoT SFT on a3a3, 5.2% for Direct-Answer SFT, 1.1%–1.2% on a1a1, and 4.3%–4.8% on a2a2.

AuthorsOpenReview ↗

Response to Reviewer VZfq (Part 2/3)

W3: Disentangling Structural Complementarity from Target-Grid Reward Sparsity

Target-grid curricula and denser branch rewards do not close the gap to complementary 1D training. Within the same 49,984-prompt per-run budget, we tested Curriculum Target GRPO across three balanced stages from G[2,2]G[2,2] (where initial pass@16 exceeds 70%), through {G[2,2,2],G[3,3],G[3,3,3]}\lbrace G[2,2,2], G[3,3], G[3,3,3] \rbrace, to the other five grids. We also tested Partial-Credit GRPO using branch reward 1B∑b=1B1[y^b=yb]\frac{1}{B}\sum _ {b=1}^B \mathbf{1}[\hat{y} _ b = y_b] across all nine targets.

The full breakdown appears in the Global Response Q1 table. Curriculum Target GRPO reaches 27.9% and Partial-Credit GRPO reaches 31.4% on the nine target grids (up from 9.5% for Direct Target GRPO), yet both remain 13.8 to 17.3 points behind 1D-trained a3a3 GRPO (45.2%, or 48.7% with warm-up). On the six unseen unequal-depth schemas and six merge DAGs, Curriculum Target GRPO reaches 24.3% and 24.6%, and Partial-Credit GRPO reaches 28.5% and 28.1%, vs. 47.6% for a3a3 and 43.8% for a3∪{J[1,1;1]}a3 \cup \lbrace J[1,1;1] \rbrace (50.8% and 46.9% with warm-up). Both target-trained baselines also leave 19.2%–21.3% within-branch resets and 14.9%–17.8% cross-branch leaks, while a3a3 cuts those routing errors to 6.8% and 5.1%.

W4: Routing Accuracy and Oracle-Routing Ceiling Across Unseen Grids

Because single-operation errors compound across n=B⋅D∈[4,16]n = B \cdot D \in [4, 16] calls, exact match mixes primitive slips with routing mistakes. As shown in Global Response Q1, checking intermediate variable bindings across the nine unseen grids gives 86.4% route-free traces under a3a3 (90.5% with warm-up). Out of a3a3's 54.8 error points, 41.2 come from valid-route traces where only primitive execution fails.

On routing itself, a3a3 cuts total routing error from 92.1% on a1a1 and 83.9% on a2a2 down to 13.6% (9.5% after warm-up), split across 6.8% within-branch resets, 5.1% cross-branch leaks, and 1.7% assembly or format errors. Under oracle intermediate routing across n∈[4,16]n \in [4, 16] calls, single-step primitive errors yield an empirical 52.4% ceiling on full B\mathcal{B}. Relative to this ceiling (Global Response Q1), a3a3 reaches 86.3% (92.9% with warm-up), ahead of Direct Target GRPO (18.1%), CoT SFT on a3a3 (35.1%), and Curriculum or Partial-Credit Target GRPO (53.2%–59.9%).

AuthorsOpenReview ↗

Response to Reviewer VZfq (Part 3/3)

W5: Discussion with Additional Related Work

Thank you for pointing out these three works [2, 3, 4], which we added to Section 1 and Appendix A and compare in the W1 table (Part 1/3). In short, 1D task or prompt chaining [2, 3] does not teach branch separation on 2D and merge DAGs, and diverse teacher-forced SFT [4] remains well below on-policy GRPO under the same source set and budget (Global Response Q1–Q2).

[1] L. Yuan et al. From f(x)f(x) and g(x)g(x) to f(g(x))f(g(x)): LLMs Learn New Skills in RL by Composing Old Ones. ICLR'26.

[2] A. Abedsoltan et al. Task Generalization with Autoregressive Compositional Structure: Can Learning From DD Tasks Generalize to DTD^T Tasks? ICML'25.

[3] X. Xu et al. Composition-RL: Compose Your Verifiable Prompts for Reinforcement Learning of Large Language Models. arXiv'26.

[4] I. Oren et al. Finding Needles in a Haystack: Sampling Structurally-Diverse Training Sets from Synthetic Data for Compositional Generalization. EMNLP'21.

[5] S. Park et al. How Does RL Post-Training Induce Skill Composition? A Case Study on Countdown. arXiv'25.

Comments on the paper

3 comments
AuthorsOpenReview ↗

Global Response to All Reviewers (Part 1/2)

We thank the Area Chair and Reviewers VZfq, YTM8, and fPYL for their careful reviews and helpful feedback. We are glad the reviewers noted our separation of task structure from primitive operations, our controlled setup and figures, and our core finding. Combining 1D depth and branching schemas (a3a3) enables zero-shot transfer to unseen 2D compositions beyond direct training. Intermediate schemas make harder structures learnable, and these gains transfer to out-of-distribution and real-world tool planning.

We uploaded a revised manuscript with major additions in blue. (1) Section 1 summarizes our main transfer results up front and compares our setup with recent sequential RL, autoregressive composition, and diverse SFT studies. (2) Sections 3.2–5 and Appendix B.2 shorten Section 3.2 by moving the full objective equations to Appendix B.2, tighten Section 4, move schema combination into Section 5, and use inline schema diagrams after the Section 2 definition. (3) Sections 5.4–5.5, Table 2, Figures 8–9, and Appendix B.5 (Table 12) add matched-budget Stage-2 SFT and target-grid curriculum or partial-credit RL baselines, the 52.4% oracle ceiling, step-level error breakdown, and coverage analysis across all 29 source combinations. (4) Sections 5.4–5.5 and Appendix B.6 (Table 13) evaluate six unseen unequal-depth schemas and six mid-computation merge DAGs using the 3-node merge schema J[1,1;1]J[1,1;1]. (5) Section 5.6 and Appendix B.4 (Table 11) report zero-shot transfer across eight unmodified official and external tool-use suites.

Q1: Stage-2 SFT Baselines, Target-Grid RL Curricula, and Step-Level Error Breakdown

All runs start from the shared Stage-1 Llama-3.1-8B-Instruct checkpoint under the same 49,984-prompt per-run budget (10,048 warm-up + 39,936 RL prompts in CoT SFT →\to GRPO, three seeds). We trained Stage-2 Direct-Answer SFT and step-supervised CoT SFT on 1D source demonstrations, and curriculum and branch-reward GRPO on the nine target grids.

Below we show 9-grid accuracy on B,D∈{2,3,4}B, D \in \lbrace 2, 3, 4 \rbrace, targets above 4%, accuracy normalized by the 52.4% oracle ceiling, route-free rate, and local G[4,4,4,4]G[4,4,4,4] step accuracy. The final four columns split 100−Acc.100 - \text{Acc.} into Prim., Depth, Branch, and Merge/Fmt (N/A means no CoT trace):

Stage-2 MethodSource SupportFull B\mathcal{B} (%)>4> 4\\%Norm. Ceil. (%)Route-Free (%)Step (%)Prim. (%)Depth (%)Branch (%)Merge/Fmt (%)
Direct-Answer SFTa3a3: a1∪a2a1 \cup a23.81/97.3N/AN/AN/AN/AN/AN/A
CoT SFTa1a1: G[1]…G[4]G[1] \dots G[4]1.10/92.18.528.47.45.863.522.2
CoT SFTa2a2: G[1]…G[1,1,1,1]G[1] \dots G[1,1,1,1]6.42/912.214.228.77.877.16.12.6
CoT SFTa3a3: a1∪a2a1 \cup a218.45/935.139.962.821.531.422.85.9
Direct Target GRPOEach target, separate9.52/918.120.731.511.238.431.99.0
Curriculum Target GRPO9 targets, easy →\to hard27.96/953.256.568.428.621.317.84.4
Partial-Credit GRPO9 targets, branch reward31.47/959.962.472.131.019.214.93.5
GRPO, Depth Axisa1a1: G[1]…G[4]G[1] \dots G[4]0.70/91.37.929.07.24.164.823.2
GRPO, Branch Axisa2a2: G[1]…G[1,1,1,1]G[1] \dots G[1,1,1,1]8.66/916.416.129.17.576.45.32.2
GRPO, Combined Axesa3a3: a1∪a2a1 \cup a2, Ours45.29/986.386.483.941.26.85.11.7
CoT SFT →\to GRPOa3a3: a1∪a2a1 \cup a2, Warmup48.79/992.990.586.841.84.63.71.2

Checking intermediate variable bindings across the nine unseen grids shows 86.4% route-free outputs under a3a3 and 90.5% after CoT SFT warm-up. Out of a3a3's 54.8 error points, 41.2 come from outputs with valid routing where only primitive execution fails, leaving 13.6 points from routing mistakes. Relative to our empirical 52.4% oracle-routing ceiling on full B\mathcal{B}, our 45.2% and 48.7% accuracies reach 86.3% and 92.9%.

Under the same per-run prompt budget, we also outperform both Stage-2 SFT and target-grid RL baselines. Step-supervised CoT SFT on a3a3 reaches 18.4% because local binding and execution drop across deeper multi-branch steps. In contrast, splitting the 49,984 prompts into 10,048 CoT SFT warm-up prompts and 39,936 GRPO prompts reaches 48.7%. On the nine targets, Curriculum Target GRPO reaches 27.9% and Partial-Credit GRPO reaches 31.4%, staying 13.8 to 17.3 points behind 1D-trained a3a3 with 19.2%–21.3% depth resets and 14.9%–17.8% cross-branch leaks.

AuthorsOpenReview ↗

Global Response to All Reviewers (Part 2/2)

Q2: Generalization Beyond Parallel Chains to Mid-Computation Merge DAGs

We thank reviewers for suggesting mid-computation merge topologies beyond parallel chains. All merge runs start from a shared Stage-1 checkpoint with five added binary string primitives. Indexing dpostd _ {\text{post}} from the binary merge step onward, we tested six unseen merge DAGs across Diamonds D[d1,d2;2]D[d_1, d_2; 2], Funnels F[B,2;2]F[B, 2; 2], and Cross-Linked DAGs X[3,D]X[3, D] (merging step 1 of branch bb into step 2 of branch b+1b + 1 for b<3b < 3):

Training Source SupportD[2,2;2]D[2,2;2]D[3,3;2]D[3,3;2]F[3,2;2]F[3,2;2]F[4,2;2]F[4,2;2]X[3,2]X[3,2]X[3,3]X[3,3]6-Target Mean
Direct Training, separate21.60.40.20.030.80.48.9%
a1a1: Depth Axis2.40.80.40.03.10.51.2%
a2a2: Branch Axis8.52.13.81.99.43.14.8%
a3a3: Combined Axes28.416.218.511.429.813.319.6%
a3∪{J[1,1;1]}a3 \cup \lbrace J[1,1;1] \rbrace, Ours58.444.246.935.148.729.543.8%

Even without binary merge examples in Stage 2, a3a3 more than doubles mean accuracy over separate direct training (19.6% vs. 8.9%). Adding one 3-node merge schema J[1,1;1]=D[1,1;1]J[1,1;1] = D[1,1;1] raises mean accuracy across the six unseen merge DAGs to 43.8% (46.9% with warm-up). On six unseen unequal-depth parallel schemas (G[3,2]G[3,2], G[2,3]G[2,3], G[4,2]G[4,2], G[4,3,2]G[4,3,2], G[4,4,3]G[4,4,3], G[4,3,2,1]G[4,3,2,1]), a3a3 itself reaches 47.6% (50.8% with warm-up) vs. 10.4% under separate direct training.

Q3: Implicit Structure Inference in Real-World Tool Planning and Frozen Suites

In synthetic string tasks, prompts include main_solution(x) so each node fixes one target primitive. Across all eight controlled tool-planning catalogues and eight frozen benchmark suites, prompts give only a natural-language request and a flat tool list, with no executable code or graph description. Below, we evaluate our Axes a3a3 and Spine s1s1 checkpoints zero-shot across five unmodified official family suites and three external suites, with a + String-Schema RL control (∗^\ast marks the in-domain test suite for each checkpoint's training family):

CheckpointFuncBGTaskBTRAJECTNESTFULBFCLMMAUToolHopMCPMarkAvg.
Base, Llama-3.1-8B24.159.231.78.749.833.927.61.829.6
+ Tool Atomic SFT26.461.133.310.852.234.729.02.131.2
+ String-Schema RL23.859.931.28.949.134.828.11.729.7
Direct @ FuncBG31.3∗31.3^\ast60.833.011.251.934.929.22.231.8
Direct @ TaskB26.164.7∗64.7^\ast33.610.552.034.629.32.031.6
Direct @ TRAJECT26.061.336.1∗36.1^\ast10.652.535.029.62.431.7
Direct @ NESTFUL26.760.532.714.1∗14.1^\ast52.334.529.41.931.5
Direct @ BFCL26.261.433.110.757.0∗57.0^\ast35.129.12.631.9
Axes a3a3 @ FuncBG39.6∗39.6^\ast61.934.112.353.035.830.12.833.7
Axes a3a3 @ TaskB27.370.8∗70.8^\ast34.411.553.435.630.02.533.2
Axes a3a3 @ TRAJECT27.062.038.9∗38.9^\ast11.453.135.430.42.932.6
Axes a3a3 @ NESTFUL27.661.833.922.2∗22.2^\ast53.235.730.22.733.4
Axes a3a3 @ BFCL26.962.234.011.659.5∗59.5^\ast36.129.83.132.9
Spine s1s1 @ FuncBG37.4∗37.4^\ast61.633.811.952.835.229.72.333.1
Spine s1s1 @ NESTFUL27.461.533.520.6∗20.6^\ast52.935.329.53.033.0

We outperform direct training on every in-domain official suite marked with ∗^\ast, gaining +8.3 points on FuncBG, +8.1 on NESTFUL, +6.1 on TaskB, +2.8 on TRAJECT, and +2.5 on BFCL. Across all eight suites, our checkpoints raise the average to 32.6%–33.7% vs. 31.5%–31.9% under direct training.

Q4: Discussion with Prior Work, Presentation, and Coverage Analysis

We updated our paper in this revised version. We shortened Section 3.2 by moving the full objective equations to Appendix B.2, which also details the evaluation protocol. We also tightened Section 4, moved schema combination into Section 5, and replaced G[⋅]G[\cdot] with inline schema diagrams throughout the main text and tables after Section 2. Across all 29 source combinations, Active Level Coverage CC tracks 9-cell interior grid accuracy at Spearman ρs=0.94\rho_s = 0.94, and replacing intermediate G[2,2]G[2,2] with G[1]G[1] in s2→s18s2 \to s18 drops 20-cell accuracy from 35.3% to 8.3%.

AuthorsOpenReview ↗

Summary of Discussion and Revisions

We sincerely thank Area Chair and Reviewer VZfq, Reviewer YTM8, and Reviewer fPYL for their reviews and two-round discussion. All three reviewers gave our paper the top soundness score (Soundness: 3), noting our setup, schema design, and cross-axis transfer findings. Following our revised paper and Round-2 replies, Reviewer VZfq confirmed that the trend across SFT and GRPO is clear and raised their score (Score Raised), and we directly answered all Round-2 questions from Reviewer YTM8.

Overview of Our Work and Revisions

Once a model knows basic operations, direct RL on complex multi-step tasks often fails (9.5%) due to sparse rewards. Holding primitive initialization, RL objective, and prompt budget fixed, training jointly on simple chains and parallel branches (a3) transfers zero-shot to unseen 2D tasks (45.2%, or 48.7% with warm-up), unequal-depth tasks, merge DAGs, and tool planning. Across our blue-highlighted revision and two rounds of replies, we addressed every question:

ReviewerMain Focus (Rounds 1 & 2)Status
Reviewer VZfqStage-2 SFT, target curricula, error split, prior work, structure vs. GRPOAddressed · Score Raised
Reviewer YTM8Tool planning, scale & merge DAGs, step routing, ceiling & prior workAddressed
Reviewer fPYLInline diagrams, earlier main results, graph operations & coverageAddressed

Details

  1. Roles of Structure vs. RL, Stage-2 SFT & Curricula (Reviewer VZfq W2, W3 & Round-2) Addressed Under the 49,984-prompt budget (Section 5.4, Table 2), data structure sets what composition rules can be learned, while on-policy RL sets how reliably they run across steps. Both CoT SFT and GRPO fail on single axes (0.7%–8.6%) and jump on a3 (18.4% CoT SFT; 45.2% GRPO; 48.7% with warm-up), as on-policy RL penalizes mixing chain continuation with branch resetting in rollouts (93.6% to 73.3% on G[4,4,4,4]G[4,4,4,4], vs. 88.4% to 38.8% for CoT SFT). Also, 9-target curriculum (27.9%) and partial-credit (31.4%) GRPO trail a3 by 13.8–17.3 points, showing complementary 1D training does more than ease sparse rewards.

  2. Step-Level Error Split & Oracle Ceiling (Reviewer VZfq W4; Reviewer YTM8 W3 & Round-2; Reviewer fPYL Q1, Q3) Addressed Single-step errors compound across 4 to 16 calls, so 100% error-free routing caps at 52.4% exact match (Section 5.4, Table 2). a3 reaches 86.3% of this ceiling (92.9% with warm-up; 86.4%–90.5% route-free), showing most remaining errors are single-operation slips rather than routing failures. Diagnostics (Section 5.5, Figures 8–9, Table 12) show a1 causes 64.8% Branch leaks and a2 causes 76.4% Depth resets, while a3 cuts routing errors to 13.6% (9.5% with warm-up).

  3. Scale Extrapolation, Unequal-Depth Schemas & Merge DAGs (Reviewer YTM8 W2a, W2b; Reviewer fPYL W3) Addressed Trained on ≤4\le 4 calls, a3 extrapolates 4× to 16 calls and reaches 47.6% (50.8% with warm-up) vs. 10.4% direct on 6 unequal-depth schemas (Table 2, Appendix B.5). Covering chaining, branching, and merging extends transfer to non-parallel DAGs: adding one merge schema J[1,1;1]J[1,1;1] lifts accuracy across 6 unseen merge DAGs from 8.9% direct to 43.8% (46.9% with warm-up; Appendix B.6, Table 13). Intermediate schemas keep harder sources learnable (s2 35.3% vs. s18 8.3%; Section 5.3), Active Level Coverage tracks 9-cell accuracy at ρs=0.94\rho_s = 0.94, and primitive sampling only shifts accuracy inside the frontier (Table 1).

  4. Implicit Structure Inference in Tool Planning (Reviewer YTM8 W1) Addressed In all tool-planning tests (Section 5.6, Figure 10, Appendix B.4), prompts give only a natural-language request and flat tool list. Our structural gains hold when the model infers the task graph implicitly: a3 gains +2.4 to +18.2 points across all 8 tool catalogues (Figures 11–12, Table 10) and beats direct training on every in-domain suite across 8 frozen benchmarks (+2.5 to +8.3 points; 32.6%–33.7% mean vs. 31.5%–31.9% direct; Table 11).

  5. Previous Work, Visual Diagrams & Readability (Reviewer VZfq W1, W5 & Round-2; Reviewer YTM8 Round-2; Reviewer fPYL W1, W2, Q2) Addressed We replaced G[·] notation after Section 2 with inline diagrams, shortened Sections 3.2–4 (Appendix B.2), and clarified how our multi-axis RL study differs from previous work on 1D chains (7-chain c1: 3.1%), diverse SFT, search trees, and whole-task RL (Section 1, Appendix A, and Round-2 replies).

We thank the Area Chair and reviewers again, the suggestions and feedback helped us to improve our submission!

Best regards,

The Authors

From OpenReview, CC BY 4.0. Authors as signed there; shown unedited.

Related papers

Loading the map…

Discussion (0)

Sign in to comment.