- Rating
- 2
- Confidence
- 3
- Informativeness
- 2
- Presentation
- 3
- Soundness
- 1
Read review
Paper summary
This paper investigates the serving efficiency of multi-agent LLM systems and highlights the gap between algorithm design and serving-system optimization. The authors propose a unified typed workflow graph representation, together with a phase-aware cost model that distinguishes prefill and decode costs. Based on this framework, the paper categorizes optimization techniques into two groups: static execution-level optimizations that preserve workflow semantics, and dynamic graph-level strategies that trade computational efficiency against task quality. The authors summarize six static patterns and five dynamic strategies, and evaluate several of them on mathematical reasoning and code generation benchmarks. The experiments show that prefix caching can improve time-to-first-token latency without necessarily providing meaningful end-to-end speedups, while strategies such as early exit and model downscaling can reduce computational costs with varying accuracy trade-offs. The paper also observes that some multi-agent workflows do not outperform a simple single-model baseline.
Critical strengths
S1. A useful perspective on algorithm–system co-design.
The paper provides a structured perspective on multi-agent serving by representing different workflows as typed graphs and analyzing their costs across prefill, decode, and execution dependencies. This framework offers a useful way to organize existing optimization techniques and understand their applicability to different workflow structures.
S2. Informative empirical observations, including negative results.
The paper evaluates both local optimization benefits and end-to-end performance, revealing that improvements in prefill efficiency may not translate into meaningful overall speedups. It also shows that some multi-agent configurations fail to outperform simpler single-model baselines. These observations provide useful practical insights and highlight the importance of evaluating actual serving costs rather than focusing exclusively on task accuracy or isolated performance metrics.
Critical weaknesses
W1. Limited technical novelty and lack of a substantial new optimization method.
The paper introduces a unified workflow graph and a phase-aware cost model, but most of the discussed optimization techniques, including prefix caching, early exit, heterogeneous model assignment, and dynamic routing, have already been explored in prior work. Representing workflows as computation graphs and applying Amdahl's Law to analyze performance improvements are also well-established ideas. More importantly, the experiments primarily apply existing techniques or simple hand-designed heuristics rather than introducing a new optimization algorithm or system. The paper does not demonstrate how its unified framework can automatically select, combine, or jointly optimize different strategies. The authors also acknowledge in Section 6 that the optimizations are manually designed, leaving automatic co-design as future work. Therefore, while the paper offers useful empirical observations, the independent technical contribution of the proposed framework remains unclear.
W2. Limited experimental validation of the broader full-stack co-design claims.
Although the paper discusses six static optimization patterns and five dynamic strategies, only a subset is directly evaluated, while the remaining techniques are mainly supported by existing literature and analytical arguments. Furthermore, the experiments focus on a single model family, mathematical reasoning and code generation benchmarks, and single-request serving. They do not evaluate multi-tenant concurrency, real tool interactions, or more complex production-level agent workflows. The limited end-to-end benefits of prefix caching are also closely related to the decode-dominated workloads used in the experiments. It remains unclear how broadly these findings generalize to workloads with different prefill/decode characteristics. Overall, the current experimental evidence is insufficient to establish the practical effectiveness and generality of the proposed framework for full-stack algorithm–system co-design.
Questions for authors
Q1. Novelty and independent contribution of the proposed framework.
Could the authors provide a concrete example where the unified graph representation and cost model enable a new optimization decision or capability that cannot be directly obtained from existing techniques? How does the proposed framework go beyond organizing and explaining established optimization methods?
Q2. Generalizability of the experimental findings.
How well do the reported findings extend to more realistic multi-agent serving scenarios involving concurrent requests, tool interactions, or more complex workflow dependencies? In particular, would the conclusions about prefix caching change substantially for workloads with a higher proportion of prefill computation?
Q3. Optimization strategy selection and quality trade-offs.
The experiments show that techniques such as early exit and model downscaling can reduce computational costs but may also degrade accuracy on harder tasks. How should the proposed framework guide the selection and configuration of these strategies under different workloads and accuracy requirements? Is there evidence that such decisions can be made more effectively using the proposed cost model than with existing heuristics?