- Rating
- 2
- Confidence
- 2
- Informativeness
- 2(+1)
- Presentation
- 2
- Soundness
- 2
Read review
Paper summary
The paper studies whether computation planning and model allocation can be learned separately in heterogeneous LLM orchestration. The authors propose SReF, which combines a model-identity-agnostic Graph Planner with a descriptor-conditioned Expert Router. Using this design, a frozen verifier evaluates executed segments and can trigger history-preserving modifications of pending computation. The Planner is then trained using dependency-aware segment feedback attributed to the source of executed node versions, while the Router learns from matched-context comparisons between candidate experts. The empirical results validate the improvements on four benchmarks and necessary ablations and analysis.
Critical strengths
S1. The manuscript successfully demonstrates a substantial technical improvement to an existing methodological design.
S2. The writing is clear and readable.
S3. The paper presents encouraging empirical results and useful controls. Specifically, the reported improvements span various benchmarks and the ablations are technically informative.
Critical weaknesses
W1. The authors should state that why should this particular separation of planning, routing, verification, and credit assignment be preferable to simpler alternatives with comparable supervision and computation. The proposed SReF’s more distinctive contribution appears to be its dependency- and provenance-aware credit under graph repair. However, Table 2 changes local reward availability as well as credit construction, leaving unclear whether the proposed propagation mechanism is responsible for the gains rather than intermediate supervision more generally.
W2. The justification of factorization and structural credit remains limited. Though Appendix C.3 establishes that the dependency trace is well-defined, while Eq. 35 establishes conditional Planner invariance by construction. This is not a requirement for a universal optimality theorem, but the paper would benefit from clearer assumptions and targeted empirical evidence for these design choices.
W3. End-to-end efficiency and resource matching should be taken into consideration. As presented in the manuscript, SReF relies on an external verifier with substantial invocation overhead, while the presneted cost breakdown omits Planner calls and does not provide corresponding totals for the baselines. It is also unclear whether the closest baselines receive comparable verification, repair, and training-supervision resources.
Questions for authors
See the weaknesses above.
5 comments
Response to Reviewer rSPA(2/4)
Adding segment-level supervision without credit propagation already improves accuracy by 1.2 percentage points, showing that intermediate supervision effectively improves training outcomes in the current setting. The full method provides a further 0.2-percentage-point improvement, but this difference is small relative to the evaluation variability. We will investigate these issues further in subsequent experiments. Taken together, dependency propagation and provenance attribution provide a way to organize feedback using the executed graph and Patch history. We will accordingly distinguish the design motivation of these mechanisms from their empirically demonstrated performance contribution, rather than attributing all gains to propagation.
Revision. We have added the segment-supervision control without propagation to Appendix G.4.
W2: Assumptions, theoretical scope, and empirical evidence for factorization and structural credit
The justification of factorization and structural credit remains limited. Though Appendix C.3 establishes that the dependency trace is well-defined, while Eq. 35 establishes conditional Planner invariance by construction. This is not a requirement for a universal optimality theorem, but the paper would benefit from clearer assumptions and targeted empirical evidence for these design choices.
Our theoretical results establish the mathematical properties of the credit recurrence and the conditional invariance of the Planner interface, respectively. They do not prove that structure–resource separation is universally optimal.
Theoretical scope of dependency credit. The result in Appendix C.3 is based on a finite executed segment-level DAG. Given rewards, execution durations, edge-attribution coefficients, and old critic predictions, it establishes that the dependency-credit recurrence has a unique finite solution that can be computed in reverse topological order. Historical execution records and retry nodes remain in the executed graph, so the result also applies to trajectories containing Patches. This ensures that credit computation is well-defined; it does not imply an unbiased estimate of the causal contribution of structural decisions, nor does it guarantee superiority over other credit-assignment methods.
Conditions for Planner invariance. Equation 35 applies with the same Planner input, fixed parameters, and identical decoding settings. The Planner’s input excludes expert identities and resource descriptors, so changing only the expert pool does not change the conditional planning policy for a given input. However, different expert outputs can change verification feedback and subsequent Patch contexts, so the executed graph may still change. This interface property does not imply that a frozen Planner remains optimal for the changed expert pool, nor does it guarantee that Router-only adaptation is equivalent to joint training.
The practical premise of this design is that existing task decompositions and dependency structures retain value after an expert-pool change, and that some differences in execution capability can be handled through resource reallocation or feedback-driven Patch. If an expert-pool change makes the original decomposition unsuitable, updating the Planner may still be necessary.
Targeted supplementary experiments. We added an experiment removing Qwen3.5-27B, the expert with the highest mean calibrated capability in the original pool, and a joint-policy control trained on randomized pools. Below, we report the mean ± sample standard deviation of Pass@1 (%) across three evaluation runs:
| Method | Pre | Post-R | Post-All |
|---|---|---|---|
| SReF | 92.8 ± 0.2 | 92.8 ± 0.2 | 92.9 ± 0.3 |
| Fixed-ID Router | 90.1 ± 0.1 | 92.0 ± 0.4 | 93.0 ± 0.2 |
| Planner-Integrated | 91.7 ± 0.3 | N/A | 92.0 ± 0.0 |
| Planner-Integrated + randomized-pool training | 92.7 ± 0.2 | N/A | 92.6 ± 0.1 |
SReF’s direct-transfer performance is close to its Router-only and joint-adaptation results, supporting the feasibility of retaining the Planner in this expert-removal setting. Meanwhile, randomized-pool training improves the joint policy’s direct-transfer performance from 91.7% to 92.7%, close to SReF. This shows that joint policies can also acquire robustness through training. We therefore limit our conclusion to the effectiveness of the factorized design under the tested conditions, without claiming that it universally outperforms joint policies.
We will explicitly distinguish these theoretical properties from the empirical conclusions in the revised manuscript. The contribution of dependency propagation itself is addressed separately in W1.
Response to Reviewer rSPA(3/4)
W3: Complete inference overhead and resource matching
End-to-end efficiency and resource matching should be taken into consideration. As presented in the manuscript, SReF relies on an external verifier with substantial invocation overhead, while the presneted cost breakdown omits Planner calls and does not provide corresponding totals for the baselines. It is also unclear whether the closest baselines receive comparable verification, repair, and training-supervision resources.
We thank the reviewer for pointing out the insufficient reporting of cost breakdowns and resource allocation. SReF’s segment-level verification and Planner Patch are part of its complete execution procedure and do introduce additional overhead. We have added inference token statistics that include Planner calls and a GraphPlanner control with verification and repair.
Planner token usage
| Dataset | Mean input tokens/question | Mean output tokens/question |
|---|---|---|
| MATH-500 | 790.0 | 143.0 |
| MMLU-Pro | 900.1 | 140.0 |
| HumanEval | 861.3 | 236.7 |
| GSM-Hard | 778.7 | 144.1 |
Mean token usage by method
We must emphasize that these statistics were collected from our local implementations using the same accounting convention as SReF. Each method is accounted for using the calls actually made during inference, including the inputs and outputs of planning, verification, repair, retry, and reward-model calls where applicable. Some methods were also adapted to our small-model pool; without such adaptation, their direct execution performed substantially worse than running an individual model. SReF’s total includes Planner, expert, and verifier usage. Verification accounts for a large share of our token consumption, so finding an inexpensive and accurate verifier is a direction for our next stage of work.
| Method | Mean tokens/question |
|---|---|
| Phi-4 | 604.8 |
| Qwen3.5-27B | 486.3 |
| DeepSeek V4.0 Pro | 379.0 |
| RouterDC | 543.8 |
| GraphRouter | 533.2 |
| Router-R1 | 1423.1 |
| xRouter | 1392.6 |
| R2-Reasoner | 3205.6 |
| MasRouter | 3964.7 |
| GraphPlanner | 5767.4 |
| GraphPlanner + verification and repair | 8916.1 |
| Puppeteer | 3751.3 |
| Uno-Orchestra | 2761.3 |
| SReF | 4,777.8 |
Verification and repair resources. The original baselines retain their respective execution procedures, without additional access to SReF’s segment-level verifier and Planner Patch. To examine the effect of the additional verification and repair resources, we integrated the same verifier used by SReF after GraphPlanner’s final output, keeping the verification threshold and maximum number of repairs identical. If verification fails, the relevant context is passed to a designated model to regenerate the answer. Evaluation uses the same expert pool.
Response to Reviewer rSPA(1/4)
Thank you for your careful review and constructive comments. We appreciate your positive assessment of the technical improvement, writing clarity, and empirical results. We respond to your concerns and questions point by point below.
W1: Design motivation and the independent contribution of credit propagation
SReF addresses two related issues: execution failures may arise from subtasks and their dependencies or from the choice of executing experts; after graph repair, the final result may also be jointly produced by the initial plan and subsequent Patches. We therefore separate planning from routing and construct different feedback for these two types of decisions.
For the Router, we fix the node and its dependency inputs and compare outputs from different experts to learn model selection for the same subtask. For the Planner, we reuse the segment-level verification scores collected during execution to provide more local structural feedback than terminal rewards alone.
Our credit mechanism further accounts for execution over a dependency graph whose computation structure may be modified by Patch. Nodes at different positions in the graph have different responsibilities: upstream nodes provide the basis for subsequent computation, while downstream nodes use, check, or integrate existing results. A segment’s local quality and the outcomes of the downstream execution it supports are therefore related but distinct feedback signals. Assigning each segment only the terminal reward and its own verification score provides intermediate supervision, but does not use inter-segment dependencies to distinguish the paths along which downstream feedback should propagate. The full method therefore propagates credit along actual execution dependencies, allowing upstream structural decisions to receive feedback from downstream segments that depend on them.
Meanwhile, Patch changes pending computation, so a single executed segment may mix nodes from the initial plan and later repairs. We attribute each node’s credit to the planning output that last created or modified its actually executed version: retained nodes continue to update their original source, while newly created or modified nodes update the corresponding Patch output. In this way, feedback is not assigned wholesale to the most recent Patch merely because the nodes appear in the latest graph.
Thus, segment-level scoring provides local quality feedback, dependency propagation organizes feedback across segments using the executed graph, and provenance attribution returns feedback to the planning decisions that actually produced the node versions. Together, these designs address the training requirements of graph execution and history-preserving repair.
To distinguish the contributions of local supervision and dependency propagation, we added a control that retains the same segment-level supervision but removes inter-segment credit propagation. Each segment directly combines the terminal structural signal with its own local verification reward to construct training credit, without aggregating feedback from other segments. The verification and Patch protocol, reward weights, structural regularization terms, and provenance-attribution rules are unchanged. Given the rebuttal time constraints, we added retraining and evaluation in the MATH-500 setting. Below, we report the mean ± sample standard deviation of Pass@1 (%) across three evaluation runs: