- Rating
- 2
- Confidence
- 2
- Informativeness
- 2
- Presentation
- 2
- Soundness
- 2
Read review
Paper summary
This paper proposes HIVE to address failures caused by insufficient decision-relevant information in LLM-based MAS. HIVE performs hierarchical sibling probing over parameters, tools, and subgoals, and updates role-specific knowledge online through sibling trajectory comparisons. Experiments report consistent improvements in mean performance.
Critical strengths
-
The paper provides a concrete approach to decision-level information acquisition for failure recovery. It expands probing from parameters to tools and subgoals, then incorporates the outcomes into Planner and Executor knowledge. This enables execution-grounded comparisons beyond a verbal explanation of failure.
-
The evaluation coverage and reported results (value itself - see weaknesses 3) are encouraging. HIVE achieves the highest mean accuracy across six benchmarks and additional LLM backbones, while the ablations in Table 2 support the usefulness of hierarchical sibling probing and online knowledge updating.
Critical weaknesses
-
The claims about identifying and disentangling information requirements exceed the demonstrated evidence. HIVE expands the scope of changes from parameters to tools and subgoals, rather than descending through successor states as in conventional DFS. However, successful hierarchical exploration alone does not establish the correct identification of missing information.
-
The experiments demonstrate improvements in mean performance but do not sufficiently establish the added value over generic exploration and reflection or statistical significance. Without budget-matched comparisons against DFS-like exploration with reflection memory, simple retries, and alternative probing orders, or independent validation of information requirements, the benefits of additional exploration remain difficult to separate from those of the proposed comparison and knowledge updating procedures.
-
The paper lacks statistical reporting. Providing the standard deviation for each reported metric alongside token usage would greatly benefit readers.
-
Some missing related work: REFLECT[1] and Causal Agent Replay[2] study attribution through intervention and replay, while ExpeL[3] updates knowledge extracted from trajectory comparisons.
-
There are some important discrepancies between the paper and the released implementation. 1)The implementation does not consistently restart each alternative from the same information state, as described in the paper. Information gathered during earlier attempts can remain in shared memory and influence later alternatives. It also records a changed subgoal as a sibling before executing and verifying it, so the recorded comparison is not necessarily between two executed alternatives starting from the same state. 2) The code counts more cases as redundant loops than the paper’s definition allows. The paper defines a loop as two consecutive execution steps that repeat essentially the same action without obtaining new target information. The implementation also counts candidates that were never executed and actions repeated after intervening steps, without checking whether new target information was obtained.
[1]: Lin, Xiaofeng, et al. "REFLECT: Intervention-Supported Error Attribution for Silent Failures in LLM Agent Traces." arXiv preprint arXiv:2606.09071 (2026).
[2]: Shah, Jaineet. "Causal Agent Replay: Counterfactual Attribution for LLM-Agent Failures." arXiv preprint arXiv:2606.08275 (2026).
[3]: Zhao, Andrew, et al. "Expel: Llm agents are experiential learners." Proceedings of the AAAI Conference on Artificial Intelligence. Vol. 38. No. 17. 2024.
Questions for authors
See weaknesses.