- Rating
- 1
- Confidence
- 3
- Informativeness
- 1
- Presentation
- 1
- Soundness
- 1
Read review
Paper summary
The paper proposes HIVE, a graph-based estimator of intervention value, and HIVE-DP, a scheduler that incorporates review capacity and practice dependent reviewer competence.
Critical strengths
Allocating limited human oversight into multi-step workflows is an important and timely problem.
The authors specifies structural constraints and reports several architectural and sensitivity analyses.
Critical weaknesses
Weaknesses
-
Unclear methodological presentation. Essential details including how the hypothetical review representation is constructed and how intervention value connects to reviewer competence are scattered across the manuscript. The explanation is difficult to follow and understand.
-
Limited empirical support. The evaluation relies on hand-specified synthetic protection and competence functions, leaving effectiveness on realistic agent workflows unestablished. Moreover, HIVE does not consistently outperform the directly matched Free-Delta-GNN baseline reported in the appendix.
-
Unjustified restriction in dynamic planning. HIVE-DP only considers reviews with positive immediate estimated reward. However, a review with negative immediate reward may still improve future competence and total utility. This restriction can exclude optimal actions under the stated objective and requires justification or comparison with unrestricted planning.
Minor Comments
The citation for "Reflexion: Language Agents with Verbal Reinforcement Learning" incorrectly lists "Ashish Berman" and "Arnav Gopinath" as authors. These names do not match the official NeurIPS 2023 proceedings or the arXiv version of the paper.
Section 2 cites the paper CIGA as Wu et al. (2022), although that reference corresponds to DIR, a totally different paper.
Questions for authors
See the Weaknesses above