- Rating
- 1
- Confidence
- 2
- Informativeness
- 1
- Presentation
- 1
- Soundness
- 1
Read review
Paper summary
The paper proposes StructuredCache, an execution framework intended to reduce failures caused by outdated task states in embodied multi-agent collaboration. Agent-generated actions are converted into structured instructions containing operation semantics, read/write dependencies, state versions, and execution constraints. A three-level cache organizes local, shared, and globally verified states, while execution gating, outcome verification, and cache invalidation maintain consistency. Evaluation uses crafting, cooking, and construction tasks in the Minecraft-based MINDcraft platform, with three LLM backbones and teams of two or three agents. Comparisons with MINDcraft and VillagerAgent report improved task success and fewer environment-interacting actions. Component ablations and trajectory analyses provide additional evidence about the framework’s behavior.
Critical strengths
-
Relevant execution-level problem. Outdated shared information can cause redundant actions, resource conflicts, and unsuccessful coordination. Explicitly tracking action dependencies and verifying execution outcomes addresses a meaningful source of failure in multi-agent systems.
-
Encouraging results within the evaluated setting. The main results show improvements across three LLM backbones, particularly on crafting and cooking. The appendix provides operation-specific templates, runtime prompts, pseudocode, and behavioral analyses that help explain the implementation.
Critical weaknesses
-
The benchmark coverage is too narrow to support the broader embodied-collaboration claims. All experiments use a single Minecraft-based platform. Crafting, cooking, and construction provide task diversity within the same game, but do not establish generalization across environments. There is no validation in a physics-based robotics simulator or a real-world robotic setting. This limitation is especially consequential because the method relies on structured state extraction, explicit action dependencies, and reliable verification of inventory or block-state changes. These assumptions become substantially more challenging under noisy perception, uncertain manipulation, and incomplete observations. Evaluation with only two or three agents also provides little evidence about scalability.
-
The contribution of the proposed coherence mechanism is insufficiently isolated. StructuredCache combines structured state representations, verified shared information, action revision, execution gating, retry suppression, and hierarchical retrieval. Although Table 2 includes variants without the hierarchy or structured instructions, it does not separately establish the benefits of version validation, invalidation, write ownership, and execution receipts. Nor is it sufficiently demonstrated that the main baselines receive equivalent verified information and action-revision opportunities. Consequently, the results do not convincingly establish how much improvement comes from the cache-coherence design rather than additional state feedback and execution safeguards.
-
The evidence for consistently improved reliability and efficiency is incomplete. Construction success improves by only 1.6–3.1 percentage points in Table 1, without uncertainty estimates establishing robustness. The appendix also contains regressions requiring explanation: Table 9 reports a decrease from 66.7% to 33.3% on the six-task, two-agent long-crafting subset, while Table 10 shows worse invalid-action and duplicate-action rates on long three-agent cooking tasks. These results complicate the narrative of consistently more reliable execution. Furthermore, Completion Steps counts environment-interacting actions rather than total computational cost. The conditional timing analysis in Figure 3 does not replace a comprehensive accounting of LLM calls, revision tokens, cache operations, and synchronization overhead.
-
Figure 1 is visually cluttered and insufficiently informative. The large stylized Minecraft scene and decorative CPU illustrations occupy substantial space, while important text and screenshots are small. More importantly, the figure does not clearly explain which caches are private versus shared, how an action’s read/write dependencies are resolved, or how a verified update invalidates another agent’s stale state. The speech bubbles illustrate generic intentions rather than a concrete cache-coherence event. As the main method overview, the figure should be replaced with a precise schematic showing the information flow and a worked example with explicit state values and versions.
Questions for authors
Can the method be evaluated in an additional environment involving physical interaction, such as a robotics simulator or a real-world multi-agent task? How would state verification and version updates work with noisy or incomplete observations?