- Rating
- 3(+1)
- Confidence
- 2
- Informativeness
- 2
- Presentation
- 2
- Soundness
- 2
Read review
Paper summary
The paper provides new exact thresholds for sizing networks optimally in DAG-based agentic prediction aggregation scenarios. The authors provide analyses under the oblivious and adaptive setting each, which can give insights for feasibility and generality of sizing networks. The analysis is rigorous and the contribution is theoretical. No experimental validation was performed.
Critical strengths
- The direction seems promising and potentially useful: Use the theoretical model of aggregation over predictors to apply in agentic AI
- The presentation of the work is technical and rigorous, and novelty is organized by stating new results in a table.
- The analysis via oblivious and adaptive settings appears important for the setting of designing an actual network in any more practical scenario.
Critical weaknesses
- The contribution is theoretical. There is no experimental section validating the theoretical contributions. Some simulation studies and discussion on realistic assumptions could help. Also, from the title, I would have expected some agentic AI use cases for validation, such as LLM agents feeding into each other.
- The theoretical framework itself is not significantly novel, given the work is building upon various related prior works.
- I think an experimental validation in the direction of use cases would be useful for the paper, as none is given currently.
- Presentation-wise, part of the paper is hard to place. Some overview figures could be helpful to easily understand the architectures of interest. Similarly, a discussion or conclusion would be helpful for less-familiar readers.
- The discussion of related work is a bit sparse, I wonder does this area of research allow a connection to recent work on circuit complexity / collaboration (for LLMs, e.g. [A, B]) or network coding?
[A] Feng et al. "Towards revealing the mystery behind chain of thought: a theoretical perspective." Advances in neural information processing systems 36 (2023): 70757-70798.
[B] Zhang et al. "Chain of agents: Large language models collaborating on long-context tasks." Advances in Neural Information Processing Systems 37 (2024): 132208-132237.
Questions for authors
- What use cases does this work provide? For agentic, I wonder if it is possible to consider LLM agents feeding into each other?
- What main limitations does this theoretical analysis have that could potentially be relaxed in future work (such as 1 feature per agent)?
- Does this area of research allow a connection to recent work on circuit complexity (for LLMs) or network coding?
2 comments
Thank you for the response. My concerns have been addressed. I think the theoretical results are of potential interest. Thus, in light of the provided discussions and clarifications, I increased my score in favor of acceptance.
We thank the reviewer for their time and careful review.
On the contribution being theoretical. Yes, this is true. However, the model of Kearns, Roth, and Ryu (SODA 2026) and its subsequent works are motivated by more practical scenarios such as agentic AI, where each agent sees part of the information and later agents build on the outputs of earlier ones. As we say in the introduction, this is our motivation as well. Our goal in this paper is to find the limits of their model: what the best network can do, and what no network can do.
On experiments and use cases. We agree that experiments in the direction of use cases, such as LLM agents feeding into each other, would be valuable. Our focus in this paper is on the limits of the model, and we leave its practical applications for future work. The model is well motivated by agentic AI, which makes this direction attractive.
On novelty. Yes, we do not introduce the model. There is an active line of work on it: Kearns, Roth, and Ryu (SODA 2026), Bateni et al. (2026a, 2026b), and Pal (2026), all discussed in Section 3. These works mostly analyze a given graph and feature allocation. Our framing is different: a designer chooses the network, and we ask what the best choice achieves. This perspective is new and sheds more light on the model.
On presentation. We thank the reviewer for these suggestions. In the revised version, we added a figure of the three-parent network in the adaptive designer setting (Figure 1), a figure for the two-parent replacement (Figure 2), and a conclusion with limitations and future work (Section 8). All changes are marked in blue.
On related work. We thank the reviewer for these papers. We added a discussion of [A] and [B] to the revised version, in the introduction and Appendix A. Systems like the one in [B] are the kind of practical setting that motivates this model. Its worker agents form a path with one parent per agent, which our Section 4 studies in a simpler form, with scalar predictions in place of text messages.
Again, the motivation is agentic AI, and LLM agents feeding into each other, as in [B], is the setting we have in mind. We hope that our theoretical results can help in that direction.
We thank the reviewer for pointing out the one-feature assumption. It is indeed easy to relax. If each agent may observe up to a constant number c of features, our upper bound constructions still apply and the same arguments give the lower bounds for exact aggregation up to constant factors. We added a brief discussion of this in Appendix G.
As the reviewer says, moving the model toward practical settings is the most important direction for future work. The main gaps are exact population fits, linear learners, and scalar messages.
Yes, our questions are motivated by and connected to both. As in circuit complexity, we measure depth and size under a bound on the number of inputs to each node, and as in network coding, intermediate nodes forward combinations of what they receive. The results are not directly applicable there right now, since each of our nodes must output its own least-squares prediction. Again, we leave moving the model of Kearns, Roth, and Ryu toward these more practical settings as future work.