- Rating
- 1
- Confidence
- 3
- Informativeness
- 2
- Presentation
- 2
- Soundness
- 2
Read review
Paper summary
This paper proposes OverForge, a training-free hierarchical LLM agent that separates coordination strategies from tactical actions and uses partner-conditioned rollouts for decision-making. Experiments in OvercookedV2 evaluate coordination, partner adaptation, and cross-episode memory.
Critical strengths
S1. The strategy-tactic hierarchy provides an intuitive and interpretable framework for cooperative planning. S2. The paper includes component ablations and detailed behavioral analyses, showing improvements over flat LLM baselines in one kitchen layout.
Critical weaknesses
W1. Limited generalizability. Experiments are restricted to OvercookedV2, a single LLM backbone, and only five episodes per condition. Performance gains are also inconsistent across layouts, while the method relies on task-specific prompts. W2. Insufficient evidence for lifelong adaptation. OverForge largely builds on MindForge's memory mechanisms, while the memory-restart experiments do not isolate the hierarchy's contribution to cross-episode adaptation. W3. Inadequate baseline comparisons. The paper lacks comparisons with existing hierarchical agents. Existing baselines also differ in components and inference costs, with OverForge requiring roughly three times as many LLM calls as MindForge, making OverForge's advantages unclear.
Questions for authors
Q1. Can the authors provide results on other benchmarks or LLM backbones, along with independent runs to assess the robustness and generalizability of the improvements? Q2. Can the authors compare OverForge with its non-hierarchical variant under the same cross-episode memory setting? Q3. Can the authors compare against existing hierarchical methods while controlling for memory, partner modeling, and inference budget?