acceptodds
Under review as a conference paper at ICLR 2027

MAO-ARAG: Multi-Agent Orchestration for Agentic Retrieval-Augmented Generation

Abstract

Retrieval-Augmented Generation (RAG) has become pivotal in Question Answering (QA) for enhancing accuracy and reducing hallucinations. While diverse RAG architectures—ranging from single-round to iterative and reasoning-based models—have been developed, selecting the optimal pipeline for individual queries with varying complexity remains a challenge. Existing adaptive methods typically rely on classifiers to select among a limited set of predefined workflows; however, this selection-based approach is inherently coarse-grained and restricted by the static scope of candidate workflows. Consequently, neither static workflows nor selection-based strategies can reliably balance effectiveness (answer quality) and efficiency (e.g., token or retrieval cost) in real-world scenarios. To address this, we propose MAO-ARAG, a fine-grained adaptive framework based on multi-agent orchestration. Conceptualizing RAG as a multi-turn decision-making process, we define a set of atomic Executor Agents corresponding to standard RAG modules (e.g., query reformulation, retrieval, generation). A Planner Agent dynamically orchestrates these executors to construct a query-specific workflow, aiming to maximize answer quality while minimizing operational costs. The Planner is optimized via Reinforcement Learning using a composite reward of F1 score and cost penalties. Extensive experiments across multiple QA benchmarks demonstrate that MAO-ARAG outperforms baselines by dynamically tailoring workflows, and achieves a superior trade-off between answer quality and cost.

Primary area:
applications to computer vision, audio, language, and other modalities
Keywords:
Multi-Agent RL, RAG, Dynamic Workflow

Reviews

Among:
Shape:
1234this paper, Bottom 27%
Figure 1. Mean review rating of the reviewed papers at ICLR 2027. This paper averages 2.00 over 3 reviews, above 27% of the others.
Reviewer_WLVgOpenReview ↗
Rating
1
Confidence
4
Informativeness
1
Presentation
2
Soundness
1
Read review
Paper summary

This paper proposes a multi-agent orchestration framework for Retrieval-Augmented Generation (RAG). The authors claim that static RAG workflows and classifier-based adaptive methods, which select among a small set of predefined pipelines, cannot balance answer quality and cost across queries of varying complexity, and thus propose a planner that constructs a query-specific workflow from a set of executor modules. The Planner is trained with PPO using a reward of answer F1 minus cost and format penalties. The experiments are conducted on seven open-domain QA datasets with baselines including eight baselines.

Critical strengths
  1. The problem of balancing the answer quality and the inference cost in RAG systems is well-motivated.
Critical weaknesses
  1. The paper does not justify the benefits of multi-agent orchestration beyond end-to-end results. Moreover, empirically, without training the proposed method performs similarly with Vanilla RAG.
  2. The baseline comparison is not rigorous, where several baselines include a trained component, but is replaced with GPT-4o-mini. Since GPT-4o-mini cannot be trained, their reported performance likely understates the original methods.
  3. The novelty is limited, by stacking a trained planner with multiple standard RAG modules with fixed prompts.
Questions for authors
  1. Can the authors provide justification for the multi-agent orchestration beyond end-to-end experiments, e.g., a comparison against the best single fixed workflow or a simple router over the same executors?
  2. How were the baselines that require training implemented with GPT-4o-mini, and were they trained on the same data as MAO-ARAG?
Reviewer_CxCCOpenReview ↗
Rating
3
Confidence
2
Informativeness
2
Presentation
3
Soundness
3
Read review
Paper summary

This paper proposes MAO-ARAG, a multi-agent framework that dynamically constructs query-specific RAG workflows by orchestrating atomic executor modules. A Planner is trained using PPO with answer-quality rewards and cost penalties to balance effectiveness and efficiency. Experiments across seven QA benchmarks demonstrate improved overall F1 performance and a favorable quality–cost trade-off compared with existing RAG baselines.

Critical strengths

The paper addresses the limitations of fixed RAG pipelines and coarse-grained adaptive routing by enabling more flexible, query-specific workflow orchestration. The decomposition of RAG operations into atomic executors provides a structured action space for learning adaptive workflows, while the Planner–Executor architecture supports modularity and different executor backbones. In addition, the proposed method achieves the highest average F1 across seven QA benchmarks and demonstrates a favorable quality–cost trade-off. The evaluation also examines different Planner sizes and executor backbones, supporting the framework's flexibility.

Critical weaknesses
  1. While the proposed fine-grained workflow orchestration achieves promising results, the experiments do not sufficiently isolate the contributions of its two central design choices: dynamic atomic workflow composition and PPO-based optimization. First, it remains unclear whether a simpler Planner trained with the same PPO reward to select among a limited set of predefined workflows could achieve comparable performance, which would help establish the necessity of fine-grained composition. Second, Table 2 shows that the smaller Planners already achieve competitive performance after SFT, with only marginal improvements from subsequent PPO training. Since the construction and source of the SFT supervision are not clearly specified, the independent contribution of PPO remains difficult to assess. Together, these issues leave the necessity and relative benefits of the proposed design choices insufficiently established.
  2. While MAO-ARAG achieves strong overall performance, its improvement over Search-o1 is substantially larger on single-hop QA (+6.58 F1) than on multi-hop QA (+0.45 F1). Given that fine-grained orchestration is motivated by handling queries of varying complexity, the paper would benefit from further analysis of why the gains differ across these settings. In particular, examining the learned workflow selection patterns and the effectiveness of query decomposition could help clarify when the proposed orchestration strategy provides the greatest benefit.
Questions for authors
  1. Could a simpler Planner trained with the same PPO reward, backbone, and training budget, but restricted to selecting among a small set of predefined workflows, achieve comparable performance and quality–cost trade-offs? Such a comparison would help clarify whether the improvements come from fine-grained atomic workflow composition or from RL-based workflow selection.
  2. Table 2 shows that the 1.5B and 0.5B Planners achieve performance close to the 7B PPO-trained Planner after SFT alone, while subsequent PPO training provides only marginal F1 improvements. Could the authors clarify how the SFT training data and target workflows are constructed, particularly whether they depend on trajectories generated by the PPO-trained Planner? Why does PPO provide such limited additional benefit after SFT, and what does this imply about its contribution to learning effective workflow orchestration?
  3. Since the proposed method achieves marginally improvements on multi-hop QA benchmarks and given that fine-grained orchestration is motivated by adapting to questions of varying complexity, could the authors explain why the gains are relatively limited on multi-hop benchmarks? In particular, could the authors analyze the learned workflow selection patterns, such as the frequency and effectiveness of query decomposition and retrieval across different datasets, to clarify when fine-grained orchestration is most beneficial?
  4. The paper models workflow orchestration as an SMDP, where each workflow is a macro-action with variable execution duration. However, it is unclear how this duration is incorporated into PPO optimization, particularly in return estimation and GAE. Could the authors clarify whether variable action durations are explicitly accounted for during optimization, and what role the SMDP formulation plays beyond representing workflow-level decisions?
Reviewer_H2zFOpenReview ↗
Rating
2
Confidence
3
Informativeness
2
Presentation
3
Soundness
2
Read review
Paper summary

MAO-ARAG proposes a fine-grained, modular agentic RAG system in which a PPO-trained Planner autoregressively composes workflows from seven fixed executor modules (serial/parallel decomposition, rewriting, retrieval, document selection, answer generation, and summarization). The Planner is formulated as a semi-Markov high-level decision-maker and optimized using terminal answer F1 together with workflow-format and optionally weighted inference-cost penalties. Across seven QA benchmarks, the reported F1-only variant achieves 52.91 average F1 compared with 49.83 for Search-o1 and 46.72 for Adaptive RAG. The paper separately explores cost-aware variants, planner sizes, and alternative executor backbones. The central question is whether learned module orchestration, rather than backbone differences or simple training gains, explains the claimed quality/cost improvements.

Critical strengths
  1. Practical modular formulation. Composing a query-specific workflow from interpretable RAG executors is a useful alternative to choosing among entire fixed pipelines. The serial/parallel decomposition and re-planning interface can accommodate heterogeneous QA requirements (Sections 3.1-3.2).

  2. Promising empirical performance and cost instrumentation. The paper evaluates seven single-/multi-hop QA benchmarks, reports three inference cost metrics, studies separately trained cost weights, and explores smaller Planners and replaceable executor APIs (Table 1, Figure 4, Table 2, Appendices E-G). In particular, the F1-only trained Planner substantially improves on the untrained counterpart (52.91 vs. 44.38 overall).

Critical weaknesses
  1. Incomplete isolation of the core architectural gain. Most reimplemented baselines use GPT-4o-mini, whereas MAO-ARAG uses a Qwen2.5-7B Planner plus GPT-4o-mini Executors. Search-R1 explicitly uses a different backbone/training setup (Section 4.1 and Appendix E). The authors correctly identify the Search-R1 comparison as a system-level contrast, but the experiments do not cleanly establish how much improvement comes from learned fine-grained composition versus an extra trained Planner, executor access, workflow flexibility, or extra computation. The untrained-Planner comparison demonstrates the benefit of PPO but does not isolate the proposed action space. A matched planner-based fixed-workflow/router baseline and controlled module-level ablations are important to the central novelty claim.

  2. Quality-cost conclusions require more careful separation and stronger evidence. Table 1's headline 52.91 F1 is the alpha=0 model, trained without cost penalization; Figure 4's alpha sweep covers NQ/HotpotQA only, and separate alpha settings require separate PPO runs (Sections 4.1-4.2, Appendix F). The authors acknowledge that one-time training costs are excluded from the inference-cost axis (Section 4 / Appendix G). Thus the reported results support an inference-time trade-off on evaluated workloads, not a universal, deployment-level optimality statement. Cost-matched quality comparisons, per-dataset trade-offs, and uncertainty across repeated training runs would materially strengthen the claim.

Questions for authors
  1. Could the authors supply a controlled comparison using the same Planner and Executors, training data, inference budgets, and execution rules, but restricting the Planner to (a) selecting predefined pipelines and (b) composing atomic executor sequences? Please additionally report ablations for decomposition (QDS/QDP), rewriting, selection, and multi-turn replanning to identify the source of gains.

  2. For the quality-cost claim, could the authors provide seven-dataset F1 and cost tables for each separately trained alpha (rather than only the NQ/HotpotQA frontier) and cost-matched baseline comparisons? How would amortized Planner training cost alter deployment break-even points?

  3. Can the authors report variation across independent PPO runs and clarify whether the semi-Markov duration T explicitly affects PPO/GAE discounting or whether the SMDP is mainly a modeling abstraction? In Eq. (8), the same terminal F1 is assigned to every Planner invocation, an ablation or analysis of long-rollout credit assignment could materially inform soundness.

From OpenReview, CC BY 4.0. Authors as signed there; shown unedited.

Related papers

Loading the map…

Discussion (0)

Sign in to comment.