- Rating
- 2
- Confidence
- 3
- Informativeness
- 2
- Presentation
- 2
- Soundness
- 2
Read review
Paper summary
This paper proposes APWA, a distributed architecture for highly parallelizable agentic workflows, aiming to address the scalability bottlenecks of existing multi-agent systems in large-scale task decomposition, coordination, and execution.
APWA introduces abstractions such as Data Tables, Subtask Templates, and dynamic agent configurations, allowing a manager agent to automatically organize large numbers of independent subtasks without delegating them individually. The system leverages Ray for distributed execution, task scheduling, and result management.
The authors evaluate APWA on five task families, including hierarchical summarization, PII redaction, structured information extraction, multilingual translation, and web report generation. The results demonstrate strong scalability on large parallel workloads where several existing multi-agent frameworks struggle, with ablation studies supporting the importance of the proposed subtask delegation mechanism.
Critical strengths
S1. Practical abstractions for large-scale parallel agent execution.
APWA introduces useful abstractions, particularly Data Tables and Subtask Templates, that reduce the context and coordination overhead of delegating large numbers of tasks. These abstractions allow the manager to specify batches of subtasks without constructing individual execution requests, providing practical value for large-scale independent data processing.
S2. Complete system implementation with encouraging scalability results.
The authors implement a distributed agent system using Ray and evaluate it across multiple task families and input scales. APWA successfully processes large workloads that several existing multi-agent frameworks struggle to complete. The ablation studies also demonstrate the importance of efficient batch delegation, supporting the system's practical effectiveness within its target setting.
Critical weaknesses
W1. Limited applicability to complex and dynamically coordinated agentic workflows.
APWA heavily relies on the independence of subtasks and explicitly prohibits direct communication between workers within a batch. While this design facilitates scalable execution, it limits the framework's applicability to more complex workflows involving dynamic dependencies, shared states, or continuous inter-agent coordination. Furthermore, most evaluated benchmarks naturally exhibit parallel structures, such as independent document processing, batch translation, and report generation on separate topics. Even hierarchical summarization follows a predefined data hierarchy. Although these experiments demonstrate the system's ability to execute large-scale parallel workloads, they provide limited evidence of its ability to handle more complex real-world workflows where the parallel structure is less obvious. This restricts the broader applicability and significance of the proposed approach.
W2. Insufficient validation of the reliability of automatic task decomposition.
Automatic task decomposition is a central capability of APWA, yet the decomposition process primarily relies on an LLM-based manager. Although the system introduces planning, delegation checks, and output contracts, these mechanisms mainly enforce execution consistency and structural requirements rather than verifying the semantic correctness of the decomposition itself. In particular, it remains unclear how the system detects missing subtasks, incorrect dependencies, or inappropriate data partitioning. Moreover, some large-scale experiments show substantial gaps between structural completeness and semantic quality, and the authors acknowledge failures involving incorrect input passing and result organization. Therefore, the current evaluation does not sufficiently establish that APWA can reliably generate correct task decompositions, rather than efficiently executing workloads with relatively obvious parallel structures. This limitation directly affects confidence in one of the paper's central contributions.
Questions for authors
Q1. Applicability to complex workflows.
APWA is primarily designed for independent parallel subtasks and does not support direct communication between workers. Could the authors clarify the extent to which the framework can handle workflows involving dynamic dependencies, shared states, or inter-agent coordination? What are its main limitations when the parallel structure is not immediately apparent?
Q2. Reliability of automatic task decomposition.
How does APWA detect errors in manager-generated task decompositions, such as missing subtasks, incorrect dependencies, or inappropriate data partitioning? Since output contracts mainly verify structural requirements, are there additional mechanisms or experimental results that demonstrate the semantic correctness and reliability of the decomposition process?
Q3. Effectiveness compared with manually designed workflows.
Some experiments show noticeable semantic quality gaps between APWA and manually designed parallel workflows. Could the authors clarify whether these differences mainly arise from task decomposition errors, worker execution quality, or result aggregation? How might these limitations affect the framework's practical usefulness in real-world applications?
2 comments
Thank you for your rebuttal. ( Although your response came a little late), it has largely addressed my concerns. I will consider raising my rating before the final decision.
We thank the reviewer for the thoughtful questions. We respond to each in order.
Q1 / W1. Applicability to complex workflows
APWA explicitly targets workloads that are decomposable into parallelized execution paths and thus assumes latent parallelism in the task. However, this assumption does not mean dependency-free:
When the parallel structure is not apparent, APWA's benefit diminishes. Planning overhead grows primarily with the number of execution rounds, while the speedup comes from the independent work processed within each round. APWA helps most when each round contains many independent subtasks and least when the task is mostly sequential. Conversely, because the manager's specification cost is independent of round width, we expect the abstractions to apply well beyond our largest evaluated round (13,632 subtasks), to 100k or 1M subtasks. Our scalability study (Appendix C.6) already exercises the executor at roughly 1.46M subtasks per 30-node trial. Workloads requiring large-scale coordination among concurrent agents (e.g., developing a large code base) would need a different design, which we leave to future work.
Q2 / W2. Reliability of automatic task decomposition
APWA does not verify the semantic correctness of a decomposition. However, several classes of decomposition errors are structurally prevented or made directly inspectable:
Table 9 measures reliability directly by recording, per layer, whether each expected output was produced. With either GPT-5.4 manager, presence and URI resolution are 1.00 at every layer of every SummaryBench corpus, with 0% failure rate (Table 7). With a GPT-5.4 mini manager, some outputs go missing in later Roman rounds (URI resolution 0.71), which output contracts did not catch since they check row counts and schemas rather than contents.
Q3. Effectiveness compared with manually designed workflows
The gap mainly arises from task decomposition and result aggregation in the manager, not from worker execution. Holding the worker fixed, replacing a GPT-5.4 mini manager with GPT-5.4 raises the Roman semantic score from 0.23 to 0.37 (Table 4), and Table 9 shows that the mini manager's deficit comes from summaries missing in later rounds. With a GPT-5.4 manager, every expected output is present, and APWA (5.4×nano) matches the hand-written pipelines on Roman (0.40 vs. 0.39–0.40; Table 8).
Regarding practical usefulness: the hand-written decompositions roughly lower-bound end-to-end latency since decomposition is pre-specified by a human-authored script, whereas APWA performs both decomposition and execution. APWA targets the setting where a parallel decomposition does not yet exist, which otherwise requires hand-authoring a parallelized execution path per-task.