- Rating
- 3
- Confidence
- 2
- Informativeness
- 2
- Presentation
- 2
- Soundness
- 3
Read review
Paper summary
GameWISE helps a 3D-game agent reuse past experience by building a visual map and sharing skill-specific knowledge across places with similar outcomes. In its hybrid version, a VLM plans using this memory, while a smaller VLA executes the actions. The authors test three ViZDoom tasks and transfer to two other game engines, reporting improvements over memory-based and baselines, with ablations supporting the design.
Critical strengths
- The graph of visually recognized places and connecting interaction segments provides a useful structure for local retrieval, progress tracking, and visual arrival checks.
- The skill-specific experience-reuse mechanism and its integration with planning and control are valuable for game tasks that are interaction-rich.
Critical weaknesses
- Visual topological mapping is already well established in the previous papers. The novelty of the "world index" relative to existing representations needs clearer justification.
- The system’s complexity makes the main contribution difficult to identify. To me, the core contribution is a particular experience-reuse mechanism and its integration for game tasks, closer to an event-based memory/interaction system than a fundamentally new map representation.
- The presentation is kind of difficult to follow, particularly how the modules interact and how information flows through the full system.
Questions for authors
- How accurate is the VLM’s coordinate grounding, and how do grounding errors affect the VLA’s control performance? Please also specify the grounding prompt and exact coordinate-output format.
- Have you evaluated a larger VLM as the planner? If so, does it improve task performance or change the relative benefit of the proposed memory system?
2 comments
-
Evidence (grounding prompt and output format, to be added to the appendix): All coordinates use 0..1000 normalized image coordinates, with [0,0] at the top-left and [1000,1000] at the bottom-right. The combat grounding system prompt is:
You ground the active combat executor in the current ViZDoom RGB frame. Return only a current-frame enemy target. Use 0..1000 image coordinates, where [0,0] is the top-left and [1000,1000] is the bottom-right. If an enemy is visible, set target_category=enemy, provide target_point at the best aim point, provide target_box when visible, and decide whether firing is safe now. If no enemy is visible, set target_category=none, target_point=null, target_box=null, should_fire=false. Ignore HUD and the held weapon sprite. JSON only.
The VLM must return a JSON object with the following schema:
{ "target_category": "enemy | none", "target_point": "[x,y] in 0..1000 image coordinates or null; enemy aim point", "target_box": "[x1,y1,x2,y2] in 0..1000 image coordinates or null", "passable": bool, "should_fire": bool, "fire_mode": "hold | single | short_burst", "burst_ticks": "0..8", "tracking_ttl_steps": "2..120", "confidence": "0..1", "reason": "short string" }
Q2. Have you evaluated a larger VLM as the planner? If so, does it improve task performance or change the relative benefit of the proposed memory system?
Yes. We additionally evaluated GPT-Astra as the planner, extending the experiments beyond the paper's Qwen3.5-9B planner. With GPT-Astra, enabling spatial memory and SCOCS more than doubles the Deathmatch mean score and My Way Home SPL, and raises My Way Home success from 45% to 100%. The benefit of memory therefore persists with a stronger planner, although its magnitude depends on the task.
- Evidence (Qwen3.5-9B and GPT-Astra planners):
| Metric | Qwen3.5-9B: memory off | Qwen3.5-9B: memory on | GPT-Astra: memory off | GPT-Astra: memory on |
|---|---|---|---|---|
| Deathmatch mean score | 5.10 | 13.24 | 6.35 | 14.10 |
| My Way Home success rate | 0% (0/20) | 65% (13/20) | 45% (9/20) | 100% (20/20) |
| My Way Home SPL | 0.000 | 0.577 | 0.3597 | 0.9079 |
| Defend the Center mean score | 0.35 | 12.25 | 10.35 | 10.55 |
We sincerely thank Reviewer L2UK for the thoughtful review and valuable comments, which we address point by point below.
W1 & W2. Visual topological mapping is well established, so the novelty of the world index is unclear; the core contribution looks more like an event-based memory/interaction mechanism than a new map representation.
We agree with this reading. Visual topological mapping is not new in itself, and the world index is the substrate for our contribution, not the contribution itself. The core contribution is the skill- and outcome-conditioned experience-reuse mechanism and its integration for game tasks. Both components are designed for one criterion: usefulness for experience reuse. For the index, this means avoiding duplicate nodes, which fragment experience, and false merges, which corrupt it; place-recognition recall is not the target. For the reuse mechanism, which records are shared matters more than how many: with memory size held fixed, outcome-conditioned grouping selects more predictive records than random sampling from the same stratum.
W3. It is hard to follow how the modules interact and how information flows.
The system is a loop of four modules, each with one input and one output: place re-identification → state key → SCOCS retrieval summary → planner goal → executor. We will add an end-to-end information-flow figure and a running example that follows a single query through the system.
Q1. How accurate is the VLM's coordinate grounding, and how do grounding errors affect VLA control? What are the prompt and output format?
We measured grounding accuracy directly against engine ground-truth bounding boxes (engine state is used only for this offline metric). A prediction counts as a hit if the predicted point falls inside the ground-truth box of the target. The grounding prompt and the exact coordinate output format will be added to the appendix. Grounding is reliable for enemies (77.2% hit rate), the target that matters most for combat. The main factor behind grounding error is the target's on-screen size, not its category: very small, distant pickups (some only 2×3 pixels) are rarely hit, while large targets are hit 81.3% of the time. Grounding errors also affect control: after a hit on an enemy, the agent gets about 2.7× more kills in the next 8 steps than after a miss.
Stratified by on-screen target size (the strongest explanatory variable):