acceptodds
Under review as a conference paper at ICLR 2027

E-Orch: Towards Effective, Efficient, and Extensible Agentic Orchestration with Reinforcement Learning

Abstract

Agentic orchestration enables multiple autonomous agents to collaborate on complex tasks through adaptive task decomposition, delegation and execution. However, existing orchestrators largely rely on hand-crafted orchestration logic and prompting strategies, limiting their ability to adapt and generalise across tasks and executor configurations. In this work, we propose E-Orch, a reinforcement learning framework for learning effective, efficient, and extensible agentic orchestration through a workflow. Rather than planning all subtasks upfront, E-Orch organises execution around , a mid-level abstraction that scopes planning and execution around meaningful intermediate objectives, enabling the orchestrator to adapt its decisions as the task evolves. For each milestone, the orchestrator constructs a dependency-aware plan, delegates subtasks to suitable executors, and parallelises independent subtasks for efficient execution. We learn the orchestration policy from execution feedback, using milestone and plan decisions as natural units for fine-grained credit assignment. To estimate their downstream contributions, we construct tree-structured rollouts that compare alternative decisions under shared execution histories. The policy is jointly optimised for task performance and efficiency through complementary rewards for performance, execution cost, and planning completeness, including an uncertainty-aware performance reward that accounts for stochastic downstream outcomes. Across seven benchmarks spanning diverse domains, E-Orch achieves the best task performance under different agent configurations, improving on the strongest baselines by - points, and achieves - higher intelligence efficiency. The learned policy also transfers to new agent configurations introduced only at evaluation time, supporting the extensibility of agentic orchestration.

Primary area:
foundation or frontier models, including LLMs
Keywords:
LLM, Multi-Agent, Agentic Orchestration

Reviews

Among:
Shape:
1234this paper, Top 49%
Figure 1. Mean review rating of the reviewed papers at ICLR 2027. This paper averages 2.33 over 3 reviews, above 51% of the others.
Reviewer_iof2OpenReview ↗
Rating
3
Confidence
2
Informativeness
3
Presentation
3
Soundness
2
Read review
Paper summary

This paper proposes E³-Orch, a reinforcement learning framework for adaptive multi-agent orchestration. It introduces a milestone-plan-act workflow that decomposes complex tasks into intermediate objectives, constructs dependency-aware execution graphs, and enables parallel execution and dynamic replanning. To improve decision-level credit assignment, the method employs tree-structured rollouts that compare alternative orchestration decisions under shared execution histories, together with separately normalized rewards for task performance, execution cost, and planning completeness. Experiments across seven benchmarks spanning mathematical reasoning, embodied decision-making, and deep research demonstrate consistent improvements in task performance and token efficiency over existing baselines. Additional experiments and ablations support the effectiveness of the proposed components and their generalization to unseen executor configurations.

Critical strengths
  1. Well-motivated and coherent framework. The milestone-plan-act workflow provides a meaningful intermediate abstraction for adaptive task decomposition, dependency-aware execution, and decision-level credit assignment. Integrating this workflow with tree-structured rollouts and multi-objective reinforcement learning offers a practical approach to improving both orchestration effectiveness and efficiency.

  2. Consistent empirical improvements with supporting ablations. E³-Orch achieves the best task performance across seven benchmarks under two executor configurations, while also improving token efficiency. The ablation studies systematically examine the contributions of the milestone abstraction, tree-structured rollouts, and reward components, providing evidence that the proposed design choices contribute to the observed improvements.

Critical weaknesses
  1. Insufficient justification of decision-level credit assignment. Although tree-structured rollouts are motivated by comparing alternative decisions under shared execution histories, the performance reward evaluates each decision against a zero-success reference posterior rather than its sibling decisions. Moreover, advantage normalization is performed across the rollout forest rather than within shared-history groups. This raises questions about whether the proposed formulation effectively isolates individual decision contributions, which is central to the paper's methodological motivation.

  2. Insufficient isolation of the proposed RL framework's contribution. While the paper provides extensive ablations, it lacks a training-free baseline using the identical milestone-plan-act workflow. Consequently, it remains unclear whether the improvements primarily arise from the hierarchical orchestration structure or the proposed RL training procedure. This limits the evidence supporting the necessity of the learning framework.

Questions for authors
  1. Credit Assignment: Why does the performance reward compare each decision against a zero-success reference posterior rather than sibling decisions sharing the same execution history? How does this design effectively isolate decision-level contributions?

  2. Contribution of RL: How does E³-Orch compare against a training-free baseline using the identical milestone-plan-act workflow? This comparison is important for distinguishing improvements from the orchestration structure itself versus the proposed RL training procedure.

  3. Executor Configuration: In Appendix A.7, model_a and model_b use the same Qwen2.5-7B-Instruct backbone but have different capability rankings and a 1:2 cost ratio. What empirical evidence supports these assignments, and how sensitive are the results to the assumed cost ratios?

3 comments
AuthorsOpenReview ↗

Response to Reviewer iof2 (Part 1 / 2)

Thank you for your thoughtful review and for recognizing the well-motivated design of our milestone-plan-act framework, its consistent improvements in performance and token efficiency across seven benchmarks, and our systematic ablation studies. We address your concerns point by point below.

W1 & Q1: Justification of credit assignment

Advantage normalization is not performed only at the forest level. As described in Lines 262–269, the cost reward is first normalized among siblings that share the same execution history (Equation (2)) and then across the forest (Equation (4)). Cost depends strongly on context (e.g., a difficult milestone requires more tokens), so forest-level normalization alone would attribute context-induced cost differences to the decision itself.

We do not normalize the performance reward within siblings, for two reasons:

  • Sibling differences are already preserved. Rperf(ν)R_{\text{perf}}(\nu) is conditioned on the execution state of ν\nu, like an action value Q(s,a)Q(s,a). Siblings share this state and decision type, so Equation (4) applies the same affine transformation to them and preserves their differences, as in the group normalization of GRPO [3]. Subtracting a sibling mean would only add a state-dependent baseline, which reduces variance but does not change the expected gradient [1, 2].
  • Sibling z-scores would discard the sample-size information in Equation (1). Equation (1) places all performance rewards on a common probability scale that accounts for sample size. With two siblings, z-scoring maps the pairs 2/8 vs. 3/8 (0.90 vs. 0.96) and 0/8 vs. 8/8 (0.50 vs. 1.00) to identical values.

Empirically, normalizing the performance reward within siblings reduces performance on all three task families (by 1.4, 3.1, and 3.9 points) at a similar token cost, as shown in the table below. We have added the new ablation study in Section 4.3.

MethodMathematical Perf.Mathematical Tokens (K)Embodied Perf.Embodied Tokens (K)Deep Research Perf.Deep Research Tokens (K)
E³-Orch52.3 ± 0.46.2 ± 0.566.5 ± 0.53.3 ± 0.414.2 ± 0.410.4 ± 0.4
w/ performance-reward normalization within siblings50.9 ± 0.65.9 ± 0.363.4 ± 0.43.5 ± 0.210.3 ± 0.510.6 ± 0.6

[1] Williams, Ronald J. "Simple statistical gradient-following algorithms for connectionist reinforcement learning." Machine Learning 8.3 (1992): 229–256.
[2] Sutton, Richard S., et al. "Policy gradient methods for reinforcement learning with function approximation." Advances in Neural Information Processing Systems 12 (1999).
[3] Shao, Zhihong, et al. "DeepSeekMath: Pushing the limits of mathematical reasoning in open language models." arXiv preprint arXiv:2402.03300 (2024).

AuthorsOpenReview ↗

Response to Reviewer iof2 (Part 2 / 2)

W2 & Q2: Contribution of RL

Thank you for the suggestion. We add results for a training-free variant of E³-Orch, which uses the identical milestone-plan-act workflow and the same orchestrator backbone but no RL training. The results are shown in the table below. All rows except the training-free variant are reproduced from Table 1. We have added the experiments of RL in the Appendix B.6.

MethodTrainingAIME24/25MATH500ALFWorldWebShopGAIAHLE
AutoGen✗16.7 ± 0.3 / 14.8 ± 0.479.5 ± 0.552.4 ± 0.441.9 ± 0.68.3 ± 0.52.7 ± 0.4
MAS-ZERO✗29.5 ± 0.6 / 25.0 ± 0.585.1 ± 0.558.2 ± 0.649.7 ± 0.411.5 ± 0.53.0 ± 0.4
E³-Orch (training-free)✗18.2 ± 0.3 / 15.6 ± 0.481.4 ± 0.553.6 ± 0.444.1 ± 0.69.0 ± 0.54.3 ± 0.4
Best Training Baseline✓33.5 ± 0.4 / 28.0 ± 0.488.1 ± 0.468.8 ± 0.558.9 ± 0.417.2 ± 0.56.6 ± 0.3
E³-Orch✓35.0 ± 0.4 / 31.5 ± 0.590.4 ± 0.470.2 ± 0.562.7 ± 0.620.5 ± 0.47.8 ± 0.3

First, even without training, E³-Orch outperforms AutoGen on all benchmarks by 0.7–2.2 points, indicating that the milestone-plan-act workflow is a reasonable structure on its own. However, the training-free variant falls behind MAS-ZERO on all benchmarks except HLE. Note that MAS-ZERO iteratively designs, evaluates, and refines a multi-agent system for each problem at inference time and selects among candidate answers via self-verification, which uses additional test-time computation, whereas the training-free E³-Orch generates each milestone and plan in a single pass.

Second, RL training accounts for most of the improvement. Training improves E³-Orch by 3.5–18.6 points across benchmarks, enabling it to surpass MAS-ZERO by 4.8–13.0 points and the best training-based baseline on each benchmark by 1.2–3.8 points. These results indicate that the milestone-plan-act structure alone is insufficient for the best performance: the improvements come primarily from the proposed RL training framework, with the structure providing an effective foundation for it.

Q3: Executor configuration

The orchestrator never observes a 1:2 cost ratio. It receives only capability and cost rankings (intelligence_rank and cost_rank), not cost values (Appendix A.7). These rankings are specified in a YAML configuration file, a flexible interface that supports diverse rankings and measurements: executors can be ranked by performance, token cost, wall-clock time, or other metrics, and new ranking attributes can be added without modifying the method.

In our experiments, the two executors share the same backbone and differ in maximum generation length (2,048 vs. 4,096 tokens, the context_windows field in Appendix A.7). This follows commercial platforms such as ChatGPT and Claude, which offer the same model at different capability and cost levels through reasoning-effort settings (low/medium/high). Empirically, we evaluate these two settings on AIME24 / GPQA as following table, which demonstrates that model_b is strong than model_a.

model_amodel_b
AIME248.310.7
GPQA29.834.2

The relative costs κ = (1.0, 2.0) in Appendix A.13 quantify the different sources of executor tokens. These weights apply solely to executor tokens and are excluded from the orchestrator's input, as the orchestrator conditions on rankings rather than raw values. We have a premise: since model_a is the cheaper executor, the cost weight assigned to model_b must exceed that of model_a. It then follows from the properties of weighted sums that, provided E³-Orch uses less of model_b than the baselines do, the resulting ordering remains invariant, as demonstrated in the following table, which lists the model_b usage of E³-Orch and the baselines across all evaluation runs.

E³-OrchPuppeteerMasRouterDAAO
# Tokens of model_b54760.73k70902.3k60245.4k65328.2k

Moreover, E³-Orch transfers to new executor configurations without retraining (Appendix B.3), including a three-executor pool with κ = (1.0, 2.0, 3.0) and enriched, domain-specific profiles. Importantly, when the cost ratio is κ = (1.0, 2.0, 3.0), E³-Orch still outperforms baselines on both performance and cost.


We hope our responses could address your concerns. If you have any further questions, please do not hesitate to let us know.

Dear Reviewer iof2,

Thank you for your positive assessment and for the time you have worked on our paper. With only one day left in the discussion period, we would like to kindly ask whether our response has addressed your concerns. We sincerely appreciate your consideration.

Best,

Authors

Reviewer_hLo5OpenReview ↗
Rating
2
Confidence
4
Informativeness
2
Presentation
2
Soundness
2
Read review
Paper summary

This paper proposes E^3-Orch, which decomposes multi-agent tasks into dynamically adjustable stage-wise goals through a "milestone–plan–act" workflow, and generates for each goal an execution graph containing executor assignments and dependency relations. The method leverages tree-based rollouts over a shared execution history to provide fine-grained feedback for milestone and planning decisions, and trains the orchestrator with rewards combining performance, cost, and plan completeness. In the experiments, the paper exhibits consistent gains in performance and efficiency, and further examines scaling behavior with respect to executor configurations.

Critical strengths
  1. Milestones offer a sensible intermediate granularity between one-shot global planning and step-by-step execution, while supporting dynamic replanning, dependency management, and parallel execution.
  2. Comparing different decisions under a shared history helps reduce interference from prior-state differences in credit assignment; separately handling task success, execution cost, and structural validity also makes the optimization objectives clearer.
  3. Beyond the reward and rollout ablations, it also examines the effects of removing milestones and the completeness reward, and reports executor-scaling experiments.
Critical weaknesses
  1. Equation (1) compares a node's success probability against a reference posterior of "zero successes under the same number of attempts." As the number of samples grows, the reference posterior concentrates toward zero, so the reward can increase even when the empirical success rate is unchanged. The differing numbers of descendants across nodes and the shared partial paths among trajectories are not sufficiently analyzed.
  2. Appendix B.4 treats the entire task as a single milestone while simultaneously removing stage-wise replanning and milestone-level branching. The ablation therefore supports the effectiveness of the overall workflow, but cannot verify whether the gains come from the milestone representation, dynamic replanning, or additional credit-assignment locations.
Questions for authors
  1. After fully accounting for orchestrator, executor input/output, and tool costs, does the efficiency improvement still hold? Can the authors provide a performance comparison under matched total cost?
  2. For decision nodes with the same success rate but different numbers of descendants, what reward differences does Equation (1) produce? How is the sample correlation caused by shared paths handled, and how is the reward prevented from favoring more-sampled nodes?
  3. Can a control condition be added that retains dynamic replanning but omits explicit milestones, with matched training tokens, executor call counts, and branching amounts, in order to isolate the contribution of the milestone abstraction itself?
  4. Please clarify the evaluation split, sample counts, deduplication, and final scorer for each benchmark, and explain the unit of the standard deviations in Table 1. Do the embodied tasks also use the official environment success metrics?
6 comments
AuthorsOpenReview ↗

Response to Reviewer hLo5 (Part 1 / 2)

Thank you for your thoughtful review and for recognizing our milestone-based task decomposition, the shared-history credit assignment with decomposed rewards, and our ablation and executor-scaling experiments. We address your concerns point by point below.

W1 & Q2: The role of Equation (1)

Thank you for the careful analysis of Equation (1). You are correct that, for a fixed success rate, Equation (1) increases with the number of samples, and this is intended: it rewards stronger evidence of success, not more samples. Shared paths do not bias any of the three rewards.

Equation (1) and sample size. In tree-structured rollouts, decision nodes have different numbers of descendant trajectories. A node with more successes over more attempts can rank higher despite a slightly lower success rate, which is the intended preference for stronger evidence. The empirical success rate assigns the same reward to 2/4 and 4/8, ignoring the difference in evidence, whereas Equation (1) assigns 0.92 vs. 0.99 to encourage. It does not reward samples alone: additional failures lower the reward (0.80 for 1/2 vs. 0.76 for 1/8; 0.92 for 2/4 vs. 0.90 for 2/8). Replacing Equation (1) with the empirical success rate (w/ Mean Performance Reward, Table 2) reduces performance by 2.2, 3.3, and 1.7 points on the Mathematical, Embodied, and Deep Research task families, without reducing token cost.

Shared paths for different reward designs.

  • Performance reward: the prefix shared above a node ν is the execution state in which ν is made, so it does not bias the estimated success after taking ν. Descendants of ν may share sub-paths, so the Beta posterior is an approximation. Equation (1) is used to rank evidence across decisions rather than to produce calibrated probabilities, and the ablation above supports this use.
  • Cost reward: we exploit the shared path on purpose. Siblings share the same execution state and have disjoint subtrees, so comparing them controls for execution history.
  • Completeness reward: it depends only on the format validity and executability of each node's own output, so shared paths do not affect it.

W2 & Q3: A control that retains dynamic replanning but omits explicit milestones

Dynamic replanning recovers part of the performance gain, but the milestone abstraction still adds 0.8–3.4 points and accounts for the efficiency gain: without milestones, token cost remains 33–76% higher even with replanning.

We agree that the ablation in Appendix B.4 removes the milestone representation, stage-wise replanning, and milestone-level branching together, and therefore cannot isolate a single factor. We add a control, w/o Milestones but w/ Dynamic Replanning, in which the orchestrator generates no explicit milestones but revises its remaining plan whenever intermediate execution results return. Training tokens, executor call budget, and branching factor match those of E³-Orch, and all other settings are unchanged. The results are shown below, the first two rows are reproduced from Table 2 and Table 8 of the submission.

MethodMathematical Perf.Mathematical Tokens (K)Embodied Perf.Embodied Tokens (K)Deep Research Perf.Deep Research Tokens (K)
E³-Orch52.3 ± 0.46.2 ± 0.566.5 ± 0.53.3 ± 0.414.2 ± 0.410.4 ± 0.4
w/o Milestones46.3 ± 0.48.5 ± 0.760.7 ± 0.56.1 ± 0.410.3 ± 0.313.2 ± 0.7
w/o Milestones but w/ Dynamic Replanning48.9 ± 0.59.1 ± 0.363.8 ± 0.45.8 ± 0.513.4 ± 0.613.8 ± 0.5
  • Replanning recovers part of the performance gain. Adding dynamic replanning to w/o Milestones improves performance by 2.6, 3.1, and 3.1 points on the Mathematical, Embodied, and Deep Research task families, respectively. It still remains 3.4, 2.7, and 0.8 points below E³-Orch.
  • The efficiency gain comes from milestones. Replanning barely changes token cost relative to w/o Milestones (9.1K vs. 8.5K, 5.8K vs. 6.1K, and 13.8K vs. 13.2K), so its cost remains 47%, 76%, and 33% above E³-Orch, reducing intelligence efficiency to 0.64×, 0.55×, and 0.71× that of E³-Orch. Milestones re-scope execution at milestone boundaries instead of repeatedly revising a single long-horizon plan.

We have added the results of w/ Dynamic Replanning in the revised Appendix B.4.

AuthorsOpenReview ↗

Response to Reviewer hLo5 (Part 2 / 2)

Q1: Comparison under matched total cost

Thank you for the question. E³-Orch achieves both higher performance and lower total cost than strong baselines DAAO, Puppeteer, and MASRouter on every benchmark, so it Pareto-dominates them.

We use wall-clock time as a unified cost measure. Tool costs (e.g., web search, code execution, and environment interaction) and token costs cannot be placed on a single scale, so we use end-to-end wall-clock time (Appendix B.2), measured under identical hardware and serving settings. It covers orchestrator inference, executor prefilling and decoding, and tool execution. E³-Orch achieves lower cost on every benchmark. Relative to the fastest of these baselines, E³-Orch reduces per-instance time by 7.5%–17.0% and full-run time by 4.7%–26.1% (Figure 8, we list the original results in the following tables). Orchestrator tokens account for only 2–3% of executor tokens.

Per-instance wall-clock time (s)

MethodAIME24AIME25MATH500ALFWorldWebShopGAIAHLE
DAAO91.6104.651.779.270.7129.6132.3
Puppeteer89.4124.055.084.371.7139.3164.8
MasRouter98.998.948.774.773.7152.0142.4
E³-Orch76.182.443.962.059.4119.9112.6

Full-run wall-clock time (h)

MethodAIME24AIME25MATH500ALFWorldWebShopGAIAHLE
DAAO1.531.741.351.551.932.113.45
Puppeteer1.492.071.431.691.872.204.29
MasRouter1.651.651.271.522.012.313.71
E³-Orch1.131.221.041.131.452.012.98

Q4: Evaluation settings

The table below details the evaluation benchmarks. Each subscript in the results tables is the standard deviation of the benchmark-level score across independent runs. We have added the details of evaluation settings in the Appendix A.14.

Benchmark# TasksRunsScorerMetric
AIME24 / AIME253032Math-VerifyAccuracy
MATH5005003Math-VerifyAccuracy
ALFWorld1343Environment SuccessSuccess rate
WebShop5003Environment SuccessSuccess rate
GAIA1653CompassVerifierAccuracy
HLE25003CompassVerifierAccuracy

We hope our responses could address your concerns and we are open for further discussion.

AuthorsOpenReview ↗

Follow-up of Rebuttal

Dear Reviewer hLo5,

As the discussion period is about to close, we would like to kindly draw your attention to our response above. In response to your reviews, we have clarified that Equation (1) rewards stronger evidence of success rather than more samples and that shared paths do not bias any of the three rewards, added a control that retains dynamic replanning but omits milestones, showing that the milestone abstraction still adds 0.8–3.4 points and accounts for the efficiency gain, shown that E³-Orch achieves both higher performance and lower end-to-end wall-clock time than the compared baselines on every benchmark, and added the full evaluation settings in Appendix A.14.

Could you please let us know whether these additions have addressed your concerns regarding the reward design, the role of milestones, the cost comparison, and the evaluation settings? If so, we would appreciate it if you could consider updating your score.

If any concerns remain, we would be glad to clarify them further while the discussion period is still open.

Sincerely,

Authors

Reviewer_GdJmOpenReview ↗
Rating
2
Confidence
3
Informativeness
3
Presentation
3
Soundness
2
Read review
Paper summary

This paper investigates the task orchestration problem for agents and proposes E³-ORCH, a reinforcement learning framework for training an agentic orchestrator to dynamically allocate tasks and manage intermediate results, rather than relying on manually specified rules or rigid, predefined workflows. The main contributions are threefold: (1) introducing a milestone-plan-act abstraction to formulate dynamic task orchestration; (2) proposing a training method based on tree-structured rollouts; and (3) evaluating the effectiveness, efficiency, and extensibility (E³) of the proposed approach across seven benchmarks: AIME24/25, MATH500, ALFWorld, WebShop, GAIA, and HLE.

Critical strengths

S1. Clear problem formulation and practical importance. \textbf{S1. Clear problem formulation and practical importance. }The paper clearly formulates an important problem. In emerging application domains such as code agents, long-horizon tasks are becoming increasingly valuable from a commercial perspective, making efficient and effective orchestration particularly important.

S2. Well-motivated methodological design.\textbf{S2. Well-motivated methodological design.} The proposed milestone concept and tree-structured rollout method are well motivated and appear theoretically reasonable. Both are also well aligned with the requirements of agent orchestration.

S3. Broad experimental evaluation.\textbf{S3. Broad experimental evaluation.} The authors conduct experiments across a relatively diverse set of benchmarks and task categories. Comparisons in terms of cost, completeness, and steps used provide evidence of the method's generality. In addition, replacing the Qwen2.5-7B executor used during training with GPT-5 mini at test time provides preliminary evidence of the method's generalization and transferability across executor models.

Critical weaknesses

W1. Insufficient diversity among executor models during training. \textbf{W1. Insufficient diversity among executor models during training. } As described in Appendix A.7, both models A and B are based on Qwen2.5-7B and differ only in their context window lengths. To establish the robustness and reliability of the proposed method more convincingly, the authors should evaluate training with a more diverse set of executor models, ideally covering multiple models with substantially different capabilities and characteristics.

W2. Insufficient discussion of practical deployment considerations. \textbf{W2. Insufficient discussion of practical deployment considerations. } Once trained, the orchestrator must be deployed in an inference-serving environment, yet the paper provides limited discussion of the deployment architecture and its associated overhead. For example, how can newly introduced models be mapped to the proposed executor profiles based on their capabilities and pricing? What is the inference cost of the orchestrator itself? Furthermore, switching executors may invalidate the existing context and incur additional prefill costs. These practical deployment considerations are not adequately addressed.

W3. Limited experimental repetitions and statistical reliability.\textbf{W3. Limited experimental repetitions and statistical reliability.} For the main results in Table 1, most benchmarks are evaluated using only three runs, with AIME24/25 being the exception at 32 runs. With so few repetitions, both the estimated means and standard deviations may be sensitive to randomness, making the reported results less convincing. More repetitions and a more rigorous statistical analysis are needed to establish the reliability of the findings.

Overall, the paper presents an interesting problem formulation and several promising ideas. However, the experimental evaluation remains somewhat preliminary, and further validation is needed to substantiate the proposed method's robustness and practical utility.

Questions for authors

Q1. Executor diversity:\textbf{Q1. Executor diversity:} As discussed in W1, do the results remain consistent when training with multiple, more diverse executor models?

Q2. Deployment and executor profiles:\textbf{Q2. Deployment and executor profiles:} As discussed in W2, how should the orchestrator be integrated into an inference-serving system? How can newly introduced models be characterized in terms of capabilities and pricing and mapped to the proposed executor profiles?

Q3. Orchestrator overhead:\textbf{Q3. Orchestrator overhead:} As discussed in W2, what is the inference cost of the orchestrator itself, particularly given that the models used are 3B or 7B parameters (Appendix B.1)? How should this overhead be weighed against the resulting benefits?

Q4. Statistical significance:\textbf{Q4. Statistical significance:} As discussed in W3, are the results statistically significant given the limited number of experimental runs? Could the authors provide additional experiments or a more rigorous statistical analysis to support their conclusions?

5 comments
AuthorsOpenReview ↗

Response to Reviewer GdJm (Part 1 / 2)

We thank the reviewer for the thoughtful review and for finding the problem formulation clear and important and the design well motivated.

W1 & Q1: Executor diversity when training

E³-Orch is not tied to specific executors . We first justify this from a design perspective, and then support it with empirical results that combine those in the original submission with new experiments on Qwen3-4B-Instruct-2507.

  • By design. The orchestrator observes only an anonymous identifier for each executor (e.g., model_a) together with its capability and cost rankings (Appendix A.7). Information needed only to invoke a model, such as its real name, URL, and context window, is never exposed. The interface is therefore identical between training and testing, and executors can be substituted freely in both stages (Appendix B.3).

  • Empirically. Firstly, in the original submission, with a different executor pool not used during training, it still achieves the best results on all benchmarks, outperforming the strongest included baseline, Puppeteer while using fewer tokens. Moreover, we replaced model_b with Qwen3-4B-Instruct-2507 during training, so the pool consists of Qwen2.5-7B-Instruct and Qwen3-4B-Instruct-2507, and kept all other settings identical to the paper. Owing to limited time and computation, we compare against representative baselines only. We have added these additional experiments in the revised Appendix B.3.

MethodTrainingAIME24/25MATH500ALFWorldWebShopGAIAHLE
MAS-ZERO✗40.4±0.3/32.3±0.488.6±0.664.3±0.454.1±0.515.2±0.64.3±0.3
Puppeteer✓42.7±0.4/35.6±0.691.2±0.573.1±0.564.8±0.419.6±0.55.5±0.4
E³-Orch✓45.5±0.4/38.6±0.493.7±0.676.0±0.367.3±0.524.2±0.39.0±0.5

W2 & Q2: Deployment and executor profiles

In E³-Orch, adding a new executor requires no retraining. Users only update the rankings in the YAML profile (Appendix A.7), and the orchestrator uses the new executor directly (Appendix B.3). The YAML profile is a flexible interface: executor metadata can be added or removed freely as long as the format is followed.

To add a new model to the pool, user could follow the simple steps:

  1. Evaluate it on the target benchmarks.
  2. Re-rank all executors in the pool by performance and inference cost (e.g., number of tokens or wall-clock time).
  3. Write the resulting ranks into the intelligence_rank and cost_rank fields of the configuration file.

W2 & Q3: Orchestrator overhead

In multi-agent systems, the vast majority of tokens are generated by the executors, and the orchestrator overhead has only a marginal effect on the results. The table below reports the total number of tokens generated by the orchestrator and by the executors across all evaluation runs. Orchestrator tokens amount to only 2–3% of executor tokens, and this ratio is similar across methods. Our efficiency analysis therefore focuses on executor tokens. Moreover, E³-Orch 7B generates fewer orchestrator tokens and fewer executor tokens than Puppeteer 7B, so including orchestrator tokens does not change the efficiency comparison with Puppeteer in Figure 4, which counts executor tokens only.

Orchestrator# Orchestrator Tokens# Executor TokensRatio (Orch. / Exec.)
E³-Orch 3B2148.5K102753.6K2.09%
E³-Orch 7B3377.4K124456.2K2.71%
Puppeteer 7B4896.5K178690.2K2.74%
AuthorsOpenReview ↗

Response to Reviewer GdJm (Part 2 / 2)

W2: Switching executors may invalidate the existing context and incur additional prefill cost

Switching executors invalidates little context and adds little prefill cost, because each sub-goal starts from a short, clean context rather than a long accumulated history.

In agentic orchestration, we decompose a complex task into several relatively independent sub-goals. Our approach executes each sub-goal with a short, clean context. As a result, switching executors at sub-goal boundaries discards little reusable context, and the prefill cost of each call is bounded by this short context. In contrast, a single agent solving a long-horizon task accumulates context at every step, so its efficiency relies heavily on KV-cache reuse [1, 2].

Prior work has shown that giving each agent a short context reduces the encoding complexity from O(n2)O(n^2) to O(nk) [3], and that isolating sub-tasks keeps the active context up to an order of magnitude smaller than that of lone sequential reasoning [4]. Consistent with this, E³-Orch has the lowest end-to-end wall-clock time, which includes executor prefilling, among the methods compared in Appendix B.2 on every benchmark (Figure 8, we list the initial results in the following tables).

Per-instance wall-clock time (s)

MethodAIME24AIME25MATH500ALFWorldWebShopGAIAHLE
DAAO91.6104.651.779.270.7129.6132.3
Puppeteer89.4124.055.084.371.7139.3164.8
MasRouter98.998.948.774.773.7152.0142.4
E³-Orch76.182.443.962.059.4119.9112.6

Full-run wall-clock time (h)

MethodAIME24AIME25MATH500ALFWorldWebShopGAIAHLE
DAAO1.531.741.351.551.932.113.45
Puppeteer1.492.071.431.691.872.204.29
MasRouter1.651.651.271.522.012.313.71
E³-Orch1.131.221.041.131.452.012.98

[1] Ji, Yichao. "Context engineering for AI agents: Lessons from building Manus." Manus Blog (2025).
[2] Zhou, Zijian, et al. "MEM1: Learning to synergize memory and reasoning for efficient long-horizon agents." International Conference on Learning Representations (2026).
[3] Zhang, Yusen, et al. "Chain of agents: Large language models collaborating on long-context tasks." Advances in Neural Information Processing Systems 37 (2024): 132208–132237.
[4] Sun, Weiwei, et al. "Scaling Long-Horizon Agent via Context Folding."

W3 & Q4: Statistical significance

We agree that reducing the effect of randomness is essential for reliable results, which is why we run every benchmark multiple times. The number of runs follows common practice in prior work. For AIME24/25, which contain few problems, we perform 32 independent evaluations, following the common protocol [1]. For the other benchmarks, we average over 3 runs [2]. We use fewer runs on these benchmarks for two reasons:

  • Most of them contain substantially more problems, which makes the results more stable.
  • Running massive evaluations on such large datasets would exceed the computational budget.

To check this choice, we evaluated E³-Orch on MATH500 and GAIA with 3, 5, and 10 runs and the mean scores are shown in the table below. Performance varies by at most 0.4 points across the number of runs, far less than the margin of E³-Orch over the strongest baseline. This suggests that 3 runs are sufficient for stable conclusions on these benchmarks.

BenchmarkBest BaselineE³-Orch (3 runs)E³-Orch (5 runs)E³-Orch (10 runs)Range
MATH50088.190.490.190.40.3
GAIA17.220.520.720.30.4

[1] Yu, Qiying, et al. "DAPO: An open-source LLM reinforcement learning system at scale." arXiv preprint arXiv:2503.14476 (2025).
[2] Zhang, Jiayi, et al. "AFlow: Automating agentic workflow generation." International Conference on Learning Representations (2025).


We hope our responses could address your concerns and we are open for further discussion.

AuthorsOpenReview ↗

Follow-up of Rebuttal

Dear Reviewer GdJm,

As the discussion period is about to close, we would like to highlight our responses above. In response to your reviews, we have shown that E³-Orch is not tied to specific executors, including new results with a different executor pool during training, that new executors can be added through the YAML profile without retraining, that orchestrator tokens account for only 2–3% of executor tokens and executor switching adds little prefill cost, and that our results are stable across the number of evaluation runs, varying by at most 0.4 points.

Could you please let us know whether these additions have addressed your concerns regarding executor diversity, deployment, efficiency, and statistical significance? If so, we would appreciate it if you could consider updating your score.

If any concerns remain, we would be glad to clarify them further while the discussion period is still open.

Sincerely,

Authors

From OpenReview, CC BY 4.0. Authors as signed there; shown unedited.

Related papers

Loading the map…

Discussion (0)

Sign in to comment.