acceptodds
Under review as a conference paper at ICLR 2027

APWA: A Distributed Architecture for Parallelizable Agentic Workflows

Abstract

Autonomous multi-agent systems based on large language models (LLMs) have demonstrated remarkable abilities in independently solving complex tasks in a wide breadth of application domains. However, these systems hit critical reasoning, coordination, and computational scaling bottlenecks as the size and complexity of their tasks grow. These limitations hinder multi-agent systems from achieving high-throughput processing for highly parallelizable tasks, despite the availability of parallel computing and reasoning primitives in the underlying LLMs. We introduce the Agent-Parallel Workload Architecture (APWA), a distributed multi-agent system architecture designed for the efficient processing of heavily parallelizable agentic workloads. APWA facilitates parallel execution by decomposing workflows into non-interfering subproblems that can be processed using independent resources without cross-communication. It supports heterogeneous data and parallel processing patterns, and it accommodates tasks from a wide breadth of domains. In our evaluation, we demonstrate that APWA can dynamically decompose complex queries into parallelizable workflows and scales on larger tasks in settings where prior systems fail completely. We show experimentally that APWA scales to over 4,100 agent invocations per second on a 30-node cluster, performs real tasks with over 10,000 independent subtasks, and is the only fully-automated system to complete every task at every input scale across 5 task families.

Primary area:
infrastructure, software libraries, hardware, systems, etc.
Keywords:
multi-agent systems, distributed systems, large language models

Reviews

Among:
Shape:
1234this paper, Bottom 27%
Figure 1. Mean review rating of the reviewed papers at ICLR 2027. This paper averages 2.00 over 3 reviews, above 27% of the others.
Reviewer_peCWOpenReview ↗
Rating
2
Confidence
3
Informativeness
2
Presentation
2
Soundness
2
Read review
Paper summary

This paper proposes APWA, a distributed architecture for highly parallelizable agentic workflows, aiming to address the scalability bottlenecks of existing multi-agent systems in large-scale task decomposition, coordination, and execution.

APWA introduces abstractions such as Data Tables, Subtask Templates, and dynamic agent configurations, allowing a manager agent to automatically organize large numbers of independent subtasks without delegating them individually. The system leverages Ray for distributed execution, task scheduling, and result management.

The authors evaluate APWA on five task families, including hierarchical summarization, PII redaction, structured information extraction, multilingual translation, and web report generation. The results demonstrate strong scalability on large parallel workloads where several existing multi-agent frameworks struggle, with ablation studies supporting the importance of the proposed subtask delegation mechanism.

Critical strengths

S1. Practical abstractions for large-scale parallel agent execution.

APWA introduces useful abstractions, particularly Data Tables and Subtask Templates, that reduce the context and coordination overhead of delegating large numbers of tasks. These abstractions allow the manager to specify batches of subtasks without constructing individual execution requests, providing practical value for large-scale independent data processing.

S2. Complete system implementation with encouraging scalability results.

The authors implement a distributed agent system using Ray and evaluate it across multiple task families and input scales. APWA successfully processes large workloads that several existing multi-agent frameworks struggle to complete. The ablation studies also demonstrate the importance of efficient batch delegation, supporting the system's practical effectiveness within its target setting.

Critical weaknesses

W1. Limited applicability to complex and dynamically coordinated agentic workflows.

APWA heavily relies on the independence of subtasks and explicitly prohibits direct communication between workers within a batch. While this design facilitates scalable execution, it limits the framework's applicability to more complex workflows involving dynamic dependencies, shared states, or continuous inter-agent coordination. Furthermore, most evaluated benchmarks naturally exhibit parallel structures, such as independent document processing, batch translation, and report generation on separate topics. Even hierarchical summarization follows a predefined data hierarchy. Although these experiments demonstrate the system's ability to execute large-scale parallel workloads, they provide limited evidence of its ability to handle more complex real-world workflows where the parallel structure is less obvious. This restricts the broader applicability and significance of the proposed approach.

W2. Insufficient validation of the reliability of automatic task decomposition.

Automatic task decomposition is a central capability of APWA, yet the decomposition process primarily relies on an LLM-based manager. Although the system introduces planning, delegation checks, and output contracts, these mechanisms mainly enforce execution consistency and structural requirements rather than verifying the semantic correctness of the decomposition itself. In particular, it remains unclear how the system detects missing subtasks, incorrect dependencies, or inappropriate data partitioning. Moreover, some large-scale experiments show substantial gaps between structural completeness and semantic quality, and the authors acknowledge failures involving incorrect input passing and result organization. Therefore, the current evaluation does not sufficiently establish that APWA can reliably generate correct task decompositions, rather than efficiently executing workloads with relatively obvious parallel structures. This limitation directly affects confidence in one of the paper's central contributions.

Questions for authors

Q1. Applicability to complex workflows.

APWA is primarily designed for independent parallel subtasks and does not support direct communication between workers. Could the authors clarify the extent to which the framework can handle workflows involving dynamic dependencies, shared states, or inter-agent coordination? What are its main limitations when the parallel structure is not immediately apparent?

Q2. Reliability of automatic task decomposition.

How does APWA detect errors in manager-generated task decompositions, such as missing subtasks, incorrect dependencies, or inappropriate data partitioning? Since output contracts mainly verify structural requirements, are there additional mechanisms or experimental results that demonstrate the semantic correctness and reliability of the decomposition process?

Q3. Effectiveness compared with manually designed workflows.

Some experiments show noticeable semantic quality gaps between APWA and manually designed parallel workflows. Could the authors clarify whether these differences mainly arise from task decomposition errors, worker execution quality, or result aggregation? How might these limitations affect the framework's practical usefulness in real-world applications?

2 comments

We thank the reviewer for the thoughtful questions. We respond to each in order.

Q1 / W1. Applicability to complex workflows

APWA explicitly targets workloads that are decomposable into parallelized execution paths and thus assumes latent parallelism in the task. However, this assumption does not mean dependency-free:

  • Dynamic dependencies. APWA’s manager inspects the results of each batch before planning the next round. The hierarchical summarization benchmark requires orchestrating results across dependent rounds (3, 4, and 5 rounds for the three corpora, each consuming the previous round's outputs), and the manager recognizes this decomposition automatically. In TranslationBench, the manager recognizes without instruction that it must first take the cross product of two input tables.
  • Shared state. State is shared across rounds through read-only, lineage-tracked data tables and a content-addressed object store, so concurrent workers cannot modify data that other workers read.
  • Inter-agent coordination. A worker can itself be a multi-agent team, e.g., independent teams of debate agents answering questions in parallel (this already captures one agent correcting another's work). Cross-subtask communication is not available. It could be supported via a shared message board or publish/subscribe model, but this re-introduces a coordination challenge and breaks desirable semantic properties (e.g., a failure of subtask A can now invalidate subtask B).

When the parallel structure is not apparent, APWA's benefit diminishes. Planning overhead grows primarily with the number of execution rounds, while the speedup comes from the independent work processed within each round. APWA helps most when each round contains many independent subtasks and least when the task is mostly sequential. Conversely, because the manager's specification cost is independent of round width, we expect the abstractions to apply well beyond our largest evaluated round (13,632 subtasks), to 100k or 1M subtasks. Our scalability study (Appendix C.6) already exercises the executor at roughly 1.46M subtasks per 30-node trial. Workloads requiring large-scale coordination among concurrent agents (e.g., developing a large code base) would need a different design, which we leave to future work.

Q2 / W2. Reliability of automatic task decomposition

APWA does not verify the semantic correctness of a decomposition. However, several classes of decomposition errors are structurally prevented or made directly inspectable:

  • Missing subtasks. A subtask template automatically expands over every row in a table, so within a batch the manager cannot omit subtasks. Result tables connect back to source rows via lineage, so missing outputs appear as missing rows to the orchestrator. Output contracts check row counts before the task can terminate.
  • Incorrect dependencies. The dependency structure of each round is explicit, inspectable state due to table lineage. A delegation step validates the trajectory against the plan and output contract before each batch or finalization (Appendix B.1.1).
  • Inappropriate partitioning. APWA does not detect this automatically, although the manager can inspect samples and statistics of intermediate tables between rounds.

Table 9 measures reliability directly by recording, per layer, whether each expected output was produced. With either GPT-5.4 manager, presence and URI resolution are 1.00 at every layer of every SummaryBench corpus, with 0% failure rate (Table 7). With a GPT-5.4 mini manager, some outputs go missing in later Roman rounds (URI resolution 0.71), which output contracts did not catch since they check row counts and schemas rather than contents.

Q3. Effectiveness compared with manually designed workflows

The gap mainly arises from task decomposition and result aggregation in the manager, not from worker execution. Holding the worker fixed, replacing a GPT-5.4 mini manager with GPT-5.4 raises the Roman semantic score from 0.23 to 0.37 (Table 4), and Table 9 shows that the mini manager's deficit comes from summaries missing in later rounds. With a GPT-5.4 manager, every expected output is present, and APWA (5.4×nano) matches the hand-written pipelines on Roman (0.40 vs. 0.39–0.40; Table 8).

Regarding practical usefulness: the hand-written decompositions roughly lower-bound end-to-end latency since decomposition is pre-specified by a human-authored script, whereas APWA performs both decomposition and execution. APWA targets the setting where a parallel decomposition does not yet exist, which otherwise requires hand-authoring a parallelized execution path per-task.

Thank you for your rebuttal. ( Although your response came a little late), it has largely addressed my concerns. I will consider raising my rating before the final decision.

Reviewer_cmqcOpenReview ↗
Rating
2
Confidence
3
Informativeness
2
Presentation
2
Soundness
1
Read review
Paper summary

This paper proposes APWA, an LLM-based distributed workflow execution system designed to handle large-scale parallelizable tasks. The authors identify key scalability challenges in existing multi-agent systems and introduce agent-facing abstractions to address these limitations. By combining LLM-driven task decomposition with distributed execution infrastructure, APWA enables the parallel execution of thousands of independent subtasks. Experiments across five benchmarks demonstrate promising scalability, execution efficiency, and task completion performance compared with existing multi-agent baselines.

Critical strengths
  • The paper proposes APWA, an LLM-based distributed workflow execution system designed to support large-scale parallelizable tasks. By combining LLM-driven planning with distributed execution infrastructure, the system enables efficient execution of thousands of independent subtasks.

  • The paper concludes scalability challenges in existing multi-agent systems, including metadata overflow and expensive per-task delegation. It introduces practical abstractions, such as data tables and subtask templates, to address these challenges and improve execution efficiency.

  • The authors evaluate APWA across five benchmarks covering different application domains. The results demonstrate promising scalability and task completion performance compared with the evaluated baselines, particularly on large-scale parallelizable workloads.

Critical weaknesses
  • In section 2.2, the authors claim that 'The architecture should partition workflows into non-interfering subproblems processed on independent resources without cross-communication'. This has over-simplified the complexity of task decomposition. As a lot of practical tasks need coordination between agents. Therefore, even though the authors try to focus on scalable multi-agent coordination, the problem formulation somewhat overstated, as it is largely limited to massively parallelizable, independent agentic workloads.

  • In Line 69, the author state: 'Existing multi-agent systems nonetheless work poorly ... They offer no automated task decomposition over distributed infrastructure..' I don't agree with this, as several works already explore task decomposition (MetaGPT), distributed execution (e.g. AutoGen), dynamic coordination, and heterogeneous agent roles. The authors should provide stronger evidence for this claim and clarify whether the actual limitation of existing approaches lies in their decomposition capabilities or in the scalability and efficiency of distributed execution, or substantiate these claims

  • The methodological novelty is limited. APWA has not altered the traditional Manager-Worker paradigm. It does not fundamentally resolve the issue of centralized coordination. Rather, it reduces the frequency and cost of invoking such coordination. Therefore, the paper's novelty appears to lie in the integration and adaptation of established distributed computing abstractions for LLM agents, rather than in a fundamentally new multi-agent coordination or task decomposition mechanism.

  • Although the benchmarks cover a diverse range of application domains, the underlying tasks appear relatively straightforward in terms of decomposition complexity and fit to APWA design naturally. In most cases, the decomposition strategy is largely dictated by the input data structure, requiring little sophisticated reasoning about task dependencies or coordination. Consequently, these benchmarks provide limited evidence of the system's ability to perform non-trivial automated task decomposition.

.

Questions for authors
  • My central question is, why do we need APWA as a new architecture, rather than simply equipping an existing LLM code agent with Ray tools? A comparison against an LLM code agent with access to the same distributed computing infrastructure is essential.

  • In Table 2, the method doesn't show superior performance in the R&J dataset. Could the authors provide more elaborations on this? Also, it seems most of the experiments only serve to validate large-scale workload execution, rather than the innovation of the decomposition algorithm itself, which corresponds to my first and last weakness concern.

2 comments

We thank the reviewer for the detailed review. We answer the points below.

Q1. Why not equip an existing LLM code agent with Ray tools?

Ray gives only scheduling and placement, so a code agent with Ray tools must rebuild the relevant structures itself in addition to solving the task, including an organization scheme for intermediate outputs, an exhaustive enumeration of work units per round with coverage checks, a record of intermediate state, per-item status and retry logic, and concerns such as batch sizing and rate limits. Re-inventing these per task is possible, but prone to mistakes that are hard to detect, and it competes for the same reasoning budget as the task itself.

Our experiments already include code-capable agents. MegaAgent equips every agent with subagent delegation, shared file system access, and Python execution, yet still reasons about data through file paths and about delegation through recruited subagents, and fails on our tasks. Our LangChain code agents and Magentic-One's Coder and CodeExecutor agents also fail on large inputs. Our hand-written Ray and asyncio decompositions show the outcome of a correct fan-out script, and roughly lower-bound end-to-end latency since decomposition is pre-written.

Q2. R&J results; experiments validate execution rather than decomposition

R&J is our smallest input, which existing baselines comfortably process. Matching their quality is the intended outcome: APWA's semantic score (0.42 ± 0.14) overlaps within one standard deviation with Direct (0.43) and LangChain (0.41–0.46). On Dynasts the baselines produce lower-quality answers (at most 0.26, vs. 0.45 for APWA), and on Roman they fail to generate sensible output (at most 0.03).

APWA has no separate decomposition algorithm. The manager LLM performs the decomposition using our abstractions. The template ablation (removing only the delegation abstraction causes failure on every SummaryBench corpus), the manager comparison in Table 4 (with a fixed worker, quality changes with the manager), per-layer output presence in Table 9, and TranslationBench and ReportBench (discovering a cross product and selecting a capability) all test this directly.

W1. Coordination between agents

APWA targets highly parallelizable workloads; we do not claim it is better on mostly sequential tasks. "Non-interfering" does not mean "free of dependencies": in SummaryBench each round consumes the previous round's outputs (3–5 rounds), and in TranslationBench the manager finds without instruction that it must take a cross product of two tables. A worker can also itself be a multi-agent team (e.g., one debate team per question).

APWA deliberately disallows direct communication between concurrent workers for these reasons: independent subtasks can be retried safely, a failure in subtask A cannot invalidate subtask B, and lineage-based recomputation remains deterministic. A shared message board or publish/subscribe channel would remove these properties and reintroduce the coordination cost APWA removes.

W2. "No automated task decomposition over distributed infrastructure" (line 69)

The systems listed examples systems each have only one of automated decomposition or distributed infrastructure. MetaGPT decomposes a task into roles through a developer-written SOP, but does not partition data or schedule work on a cluster. AutoGen provides a distributed agent runtime, but developers must define each agent and the team structure. Magentic-One, built on AutoGen, decomposes automatically but executes workers serially through its orchestrator. MegaAgent and LangChain sub-agents decompose dynamically, but create each sub-agent with a separate tool call, and execution is not distributed.

The limitation is in the agent-facing abstractions. Through the standard tool-calling interface, the number of subtasks an orchestrator can issue is bottlenecked by LLM generation latency, restricting per-subtask delegation to tens to hundreds of subtasks. With a file-system interface, the LLM must track and replicate paths across hundreds of tool calls, and at larger scales, merely enumerating the data items hits the context window limit (Magentic-One's FileSurfer consumed 6.5M input tokens paging through 2,500 files). Neither limit depends on the execution layer: in the template ablation, with Ray execution still available, the manager without templates fails on every SummaryBench corpus.

W3. Novelty: APWA keeps the manager–worker paradigm

We agree, and we do not claim a new coordination topology. Our contribution lies in our diagnosis of why existing agent-based orchestrators fail at scale, and in the agent-facing abstractions that resolve those failures. In existing systems, the orchestrator processes each worker's report individually, so its reasoning load grows with the number of agents. In APWA, a single template stages an entire batch, and results return as tables the manager reasons over in aggregate, so the manager's load grows with the number of batches rather than workers (over 2.5k concurrent workers on Roman).

MapReduce and Spark also use a centralized master or driver, and distributed programs could be written before either existed. Their contribution was a set of intuitive and expressive abstractions that greatly simplified the development of distributed applications. We pursue the analogous goal in the agentic setting, where metadata overflow and inverted specification cost, which classical dataflow engines do not face, motivate the data table and subtask templates. An arbitrarily large number of subtasks can be defined with a single tool call: our largest task issues 13,632 subtasks, we expect the abstractions to apply at 100k or 1M, and our scalability study already exercises the executor at roughly 1.46M subtasks per 30-node trial (Appendix C.6).

W4. The benchmarks are simple to decompose

That the decompositions seem obvious is a credit to the abstractions we employ, not triviality of the tasks. Every baseline receives the same tasks and data, and none finds and executes the decomposition at scale (e.g., MegaAgent attempts purely role-based decomposition, and the LangChain supervisor fails to scale delegation with input size). The tasks also require more than a single fan-out: 3–5 dependent rounds in SummaryBench, a cross product in TranslationBench, and capability selection from the registry in ReportBench.

Reviewer_Ai8xOpenReview ↗
Rating
2
Confidence
1
Informativeness
2
Presentation
2
Soundness
2
Read review
Paper summary

This paper addresses why LLM multi-agent systems fail on large, parallelizable workloads, where data exceed the context window and delegating one subtask per tool call becomes a bottleneck (“inverted specification cost”). The authors attribute these failures to missing agent-facing abstractions and propose APWA, a manager/worker/executor architecture on Ray. The manager reasons over read-only data tables with constant-size metadata, issues subtask templates that expand over every table row, maintains a plan and an optional output contract, and assembles workers from a capability registry; subtasks run as sandboxed agents or single LLM calls. The evaluation covers five task families—hierarchical summarization, PII redaction, format conversion, translation, and web reports—with Direct prompting, Magentic-One, MegaAgent, LangChain subagents, and hand-written decompositions compared only on the first three, plus a 1–30-node executor scaling study.

Critical strengths

Coupling read-only tables to row-expanding subtask templates is a simple, reusable answer to a genuine agent-specific bottleneck: one tool call can specify thousands of subtasks. On Roman and PII-4096, all four baselines produce no output or semantic scores of at most 0.03 and 0.18, against APWA’s 0.23 and 0.54.

Critical weaknesses
  1. My primary concern is the practical value of this work. Mainstream harnesses such as Claude Code and Codex already support sub-agents and can script an LLM fan-out over input rows, yet none is evaluated. Such a script would resemble the paper’s hand-written asyncio pipeline, which attains higher semantic scores than APWA (mini×mini) and runs faster on every SummaryBench and PII-300k input (Tables 7–8, 10–11). Would APWA’s tables and templates, exposed as tools, improve these harnesses, and does the diagnosis of “inadequate agent-facing abstractions for large data” (Section 1) still hold under them?
  2. Quality is not “near the hand-authored decompositions” (Section 4.2) at the largest inputs, and the 0% failure rate hides collapsed trials. Against hand-written asyncio, APWA (mini×mini) trails in semantic score by 8–16% at small and medium scale but 43% on Roman (0.23 vs. 0.40; Table 8) and 35% on PII-4096 (0.54 vs. 0.83; Table 11). At PII-4096, the structural score of 0.90 ± 0.30 at 0% FR (Tables 10–11) sits at the largest SD possible for a [0, 1] score with mean 0.9, so trials are essentially all-or-nothing—roughly one in ten without a usable output table. Section 4.2 calls such empty-result trials “minor” but gives no counts. A GPT-5.4 manager narrows the Roman gap (0.37–0.40; Table 8), but PII-300k was run only with mini×mini, and the proposed harness fixes are untested. I therefore cannot determine how often APWA delivers hand-written-level output at the scales motivating the paper.
Questions for authors

N/A

1 comment

We thank the reviewer for their comments. We address the points below.

W1. Practical value compared with Claude Code, Codex, and similar harnesses

We agree that a fan-out script produced by such a harness would resemble our hand-written asyncio pipeline, which is why we report the hand-written decompositions. These roughly lower-bound end-to-end latency since decomposition is pre-specified by a human-authored script, whereas APWA performs both decomposition and execution.

This mirrors distributed programming before MapReduce: distributed programs could already be written, but each one managed partitioning, data movement, and failure handling by hand, and MapReduce's contribution was a set of abstractions that greatly simplified writing them. A harness with code execution and sub-agents is in an analogous position. It supplies lower-level primitives, so the agent must rebuild the relevant structures itself in addition to solving the task. Concretely, it must construct and maintain on every task:

  1. An ad hoc organization scheme for intermediate outputs, e.g., nested file paths.
  2. An exhaustive enumeration of work units per round, and verification of complete coverage of those items.
  3. A record of what intermediate state exists, implicitly (the context window) or explicitly (written and read back each round).
  4. Per-item success and failure status, and retry and failure handling logic.
  5. Other low-level concerns, like subbatch sizing, concurrency limits, and rate-limit backoff.

Re-inventing these structures per task is possible, but (1) prone to mistakes, (2) the resulting errors are hard to detect, (3) it introduces unnecessary variance in the execution path, and (4) it competes for the same reasoning budget as the task itself. In APWA, completeness of delegation, the correspondence between inputs and outputs, and the safety of retries are all maintained by the system and inspectable by the agent.

Our baselines already cover both paths such a harness could take. Per-subtask delegation (MegaAgent, LangChain code agents with sub-agents) is bottlenecked by LLM generation latency: on PII-4096, the LangChain supervisor requests between 1 and 8 workers per trial and writes a final answer containing at most 15 of 4,096 documents. Programmatic solutions (Magentic-One, with its Coder and CodeExecutor agents) fail through orchestration errors and context explosion, e.g., the FileSurfer agent paging through over 2,500 input documents and consuming over 6.5M input tokens. The template ablation isolates the delegation abstraction: all else being held equal, the manager without subtask templates fails to complete any SummaryBench task. Our largest evaluated task issues 13,632 subtasks, but in APWA an arbitrarily large number of subtasks can be defined with a single tool call, and our scalability study already exercises the executor at roughly 1.46M subtasks per 30-node trial (Appendix C.6).

W2. Quality at the largest inputs, and the 0% failure rate

The all-or-nothing scoring tends to arise due to the somewhat strict scoring of our metrics. If the manager fails to take a single important step (e.g., renaming a column), it scores a 0 in the corresponding part of the evaluation. We note from the manager ablation that the more powerful GPT 5.4 model makes such mistakes less often. We consider these mistakes as minor in the sense that we do not believe they are related to the scale of the task, but can happen with the same frequency at all task scales.

From OpenReview, CC BY 4.0. Authors as signed there; shown unedited.

Related papers

Loading the map…

Discussion (0)

Sign in to comment.