acceptodds
Under review as a conference paper at ICLR 2027

GameWISE: Learning World-Indexed Situated Experience for Adaptive Decision-Making in 3D Games

Abstract

Visual agents in unfamiliar 3D environments accumulate long interaction streams that are brittle to retrieve or replay as monolithic trajectories. We introduce GameWISE, a world-indexed situated experience reuse framework in which visually re-identifiable places partition long-horizon interaction into locally retrievable and verifiable segments. To determine where skill experience remains behaviorally reusable, Skill-Conditioned Cross-Place Outcome Compatibility Sharing (SCOCS) compares skill-conditioned empirical outcome distributions using optimal transport and recursively evaluates finite-depth successor compatibility under matched observable conditions. Screened successful segments adapt a visual–action controller to take over high-frequency low-level actions, while the VLM uses retrieved spatial and outcome evidence to issue persistent goals and replan on events. Under a common protocol in ViZDoom, GameWISE achieves the strongest system-level results. On held-out Xonotic and Sauerbraten, GameWISE acquires and reuses situated experience online, achieving the best task performance without target-engine policy training.

Primary area:
applications to computer vision, audio, language, and other modalities
Keywords:
Embodied AI, Visual Game Agents, Episodic Memory

Reviews

Among:
Shape:
1234this paper, Top 15%
Figure 1. Mean review rating of the reviewed papers at ICLR 2027. This paper averages 3.00 over 4 reviews, above 85% of the others.
Reviewer_L2UKOpenReview ↗
Rating
3
Confidence
2
Informativeness
2
Presentation
2
Soundness
3
Read review
Paper summary

GameWISE helps a 3D-game agent reuse past experience by building a visual map and sharing skill-specific knowledge across places with similar outcomes. In its hybrid version, a VLM plans using this memory, while a smaller VLA executes the actions. The authors test three ViZDoom tasks and transfer to two other game engines, reporting improvements over memory-based and baselines, with ablations supporting the design.

Critical strengths
  • The graph of visually recognized places and connecting interaction segments provides a useful structure for local retrieval, progress tracking, and visual arrival checks.
  • The skill-specific experience-reuse mechanism and its integration with planning and control are valuable for game tasks that are interaction-rich.
Critical weaknesses
  • Visual topological mapping is already well established in the previous papers. The novelty of the "world index" relative to existing representations needs clearer justification.
  • The system’s complexity makes the main contribution difficult to identify. To me, the core contribution is a particular experience-reuse mechanism and its integration for game tasks, closer to an event-based memory/interaction system than a fundamentally new map representation.
  • The presentation is kind of difficult to follow, particularly how the modules interact and how information flows through the full system.
Questions for authors
  • How accurate is the VLM’s coordinate grounding, and how do grounding errors affect the VLA’s control performance? Please also specify the grounding prompt and exact coordinate-output format.
  • Have you evaluated a larger VLM as the planner? If so, does it improve task performance or change the relative benefit of the proposed memory system?
2 comments

We sincerely thank Reviewer L2UK for the thoughtful review and valuable comments, which we address point by point below.

W1 & W2. Visual topological mapping is well established, so the novelty of the world index is unclear; the core contribution looks more like an event-based memory/interaction mechanism than a new map representation.

We agree with this reading. Visual topological mapping is not new in itself, and the world index is the substrate for our contribution, not the contribution itself. The core contribution is the skill- and outcome-conditioned experience-reuse mechanism and its integration for game tasks. Both components are designed for one criterion: usefulness for experience reuse. For the index, this means avoiding duplicate nodes, which fragment experience, and false merges, which corrupt it; place-recognition recall is not the target. For the reuse mechanism, which records are shared matters more than how many: with memory size held fixed, outcome-conditioned grouping selects more predictive records than random sampling from the same stratum.

W3. It is hard to follow how the modules interact and how information flows.

The system is a loop of four modules, each with one input and one output: place re-identification → state key → SCOCS retrieval summary → planner goal → executor. We will add an end-to-end information-flow figure and a running example that follows a single query through the system.

Q1. How accurate is the VLM's coordinate grounding, and how do grounding errors affect VLA control? What are the prompt and output format?

We measured grounding accuracy directly against engine ground-truth bounding boxes (engine state is used only for this offline metric). A prediction counts as a hit if the predicted point falls inside the ground-truth box of the target. The grounding prompt and the exact coordinate output format will be added to the appendix. Grounding is reliable for enemies (77.2% hit rate), the target that matters most for combat. The main factor behind grounding error is the target's on-screen size, not its category: very small, distant pickups (some only 2×3 pixels) are rarely hit, while large targets are hit 81.3% of the time. Grounding errors also affect control: after a hit on an enemy, the agent gets about 2.7× more kills in the next 8 steps than after a miss.

  • Evidence (localization error, predicted point vs. ground-truth box):
CategorynHit rate (point inside GT box)95% CINormalized L2L2 / box diagonalBox IoU
enemy2,1300.772[0.754, 0.790]0.0660.400.524
health4,2600.674[0.660, 0.688]0.0370.860.435
ammo8750.645[0.613, 0.677]0.0370.390.413
weapon800.212[0.125, 0.312]0.0952.990.158
Overall7,3450.694[0.683, 0.704]0.0460.690.455

Stratified by on-screen target size (the strongest explanatory variable):

Size binnHit rateL2 / box diagonal
tiny (< 0.1% of screen area)6310.2824.24
small (0.1%–1%)3,6550.6660.45
large (> 1%)3,0590.8130.25
  • Evidence (grounding error vs. control quality, enemy targets, step level):
Enemy groundingnKills in the next 8 steps95% CI
Hit1,6220.103[0.087, 0.120]
Miss4780.038[0.021, 0.059]
  • Evidence (grounding prompt and output format, to be added to the appendix): All coordinates use 0..1000 normalized image coordinates, with [0,0] at the top-left and [1000,1000] at the bottom-right. The combat grounding system prompt is:

    You ground the active combat executor in the current ViZDoom RGB frame. Return only a current-frame enemy target. Use 0..1000 image coordinates, where [0,0] is the top-left and [1000,1000] is the bottom-right. If an enemy is visible, set target_category=enemy, provide target_point at the best aim point, provide target_box when visible, and decide whether firing is safe now. If no enemy is visible, set target_category=none, target_point=null, target_box=null, should_fire=false. Ignore HUD and the held weapon sprite. JSON only.

    The VLM must return a JSON object with the following schema:

    {
      "target_category": "enemy | none",
      "target_point": "[x,y] in 0..1000 image coordinates or null; enemy aim point",
      "target_box": "[x1,y1,x2,y2] in 0..1000 image coordinates or null",
      "passable": bool,
      "should_fire": bool,
      "fire_mode": "hold | single | short_burst",
      "burst_ticks": "0..8",
      "tracking_ttl_steps": "2..120",
      "confidence": "0..1",
      "reason": "short string"
    }
    

Q2. Have you evaluated a larger VLM as the planner? If so, does it improve task performance or change the relative benefit of the proposed memory system?

Yes. We additionally evaluated GPT-Astra as the planner, extending the experiments beyond the paper's Qwen3.5-9B planner. With GPT-Astra, enabling spatial memory and SCOCS more than doubles the Deathmatch mean score and My Way Home SPL, and raises My Way Home success from 45% to 100%. The benefit of memory therefore persists with a stronger planner, although its magnitude depends on the task.

  • Evidence (Qwen3.5-9B and GPT-Astra planners):
MetricQwen3.5-9B: memory offQwen3.5-9B: memory onGPT-Astra: memory offGPT-Astra: memory on
Deathmatch mean score5.1013.246.3514.10
My Way Home success rate0% (0/20)65% (13/20)45% (9/20)100% (20/20)
My Way Home SPL0.0000.5770.35970.9079
Defend the Center mean score0.3512.2510.3510.55
Reviewer_YdLvOpenReview ↗
Rating
3
Confidence
2
Informativeness
2
Presentation
3
Soundness
2
Read review
Paper summary

GAMEWISE helps agents reuse past experience to make decisions in 3D games using only visual observations. It organizes experience by location and shares it across places when the same skill produces similar outcomes under matching conditions. A VLM uses summaries of these experiences to plan, while a VLA carries out its instructions. The authors evaluate the system on navigation and combat tasks in ViZDoom and test transfer to two other game engines without additional policy training.

Critical strengths
  1. A strength of the paper is its distinction between recognizing a place and deciding whether past experience applies there. An agent may need to distinguish two locations for navigation while still drawing on similar combat or resource-gathering experiences from both. GAMEWISE captures this distinction naturally by keeping the spatial map separate from skill-specific experience sharing.

  2. The ablations show that sharing more experience is not always helpful. Selective sharing performs better than unconditional pooling, and both looking beyond immediate outcomes and handling missing evidence conservatively improve performance (Tables C.1–C.2). This provides useful evidence that the choice of which experiences to reuse matters, beyond simply giving the planner access to memory.

Critical weaknesses
  1. The added value of optimal transport over simpler experience-sharing rules remains unclear.

    1. Table 5 shows that SCOCS outperforms local state-graph retrieval and semantic clustering, but these methods also differ in how they account for the agent’s current state. As Section 5.4.2 explains, the semantic baseline groups similar places without preserving the same state distinctions. This supports the overall retrieval design, but does not isolate the benefit of optimal transport over a simpler comparison of outcomes under matching state conditions.

    2. The component ablations in Table C.2 provide evidence for selective sharing and recursive successor comparisons. The remaining question is whether a simpler outcome-distance measure could achieve similar results with the same state-matching and summary-generation rules. Reporting performance across comparable levels of supporting evidence would further clarify where the additional complexity pays off.

  2. The hybrid results do not separate the effects of its individual design choices.

    1. Enabling the “VLM–VLA interface” in Table 3 changes both the executor and the scheduled planning interval: Sections 4.4 and 5.1 describe replacing rule-based execution with a learned VLA and increasing the interval from 4 to 16 environment steps. The reduction from 5.52 to 1.30 seconds in Table 1’s latency metric is a valid system-level result, but does not separate the effects of learned execution, persistent goals, and less frequent planner calls. Comparisons at matched planning intervals would make this attribution clearer.

    2. Table 3 also contains numerical reversals that are not discussed. With spatial indexing alone, adding the interface lowers mean Deathmatch reward from 6.67 to 5.67 and Defend the Center reward from 7.20 to 6.65. With both spatial indexing and SCOCS, adding it improves both rewards. This suggests a possible interaction between memory and execution, but paired confidence intervals and trajectory-level analysis are needed to distinguish a consistent pattern from variation across runs.

Questions for authors
  1. Contribution of optimal transport

    1. Beyond the ablations in Table C.2, have you tested a simpler outcome-distance measure without optimal transport, while retaining the same state-matching rules and summary-generation procedure?

    2. If so, how does its task performance compare with SCOCS?

    3. For that comparison, how does the average number of records supporting each retrieved summary compare with SCOCS?

  2. Contribution of the hybrid design

    1. Given that the reported comparison evaluates the combined system, do you also have results comparing rule-based execution and VLA execution at the same scheduled planning interval and with the same memory configuration?

    2. Have you evaluated the hybrid with scheduled planning intervals of both 4 and 16 steps, keeping the controller checkpoint and memory configuration fixed? If so, how does performance change?

    3. If timing logs are available, could you report component-level timings and actual planner-call rates for each scenario in Table 1, distinguishing steps with and without a VLM call?

  3. Performance reversals in Table 3

    1. With spatial indexing alone, adding the interface reduces mean Deathmatch reward from 6.67 to 5.67. What is the paired 95% confidence interval for this change?

    2. The corresponding Defend the Center reward falls from 7.20 to 6.65. What is the paired 95% confidence interval for this change?

    3. If these decreases are supported by the paired analysis, do the recorded trajectories reveal behavioral differences that could explain why adding the interface helps when SCOCS is enabled but hurts when it is absent?

2 comments

We sincerely thank Reviewer YdLv for the thorough review and helpful suggestions, which we address point by point below.

W1 & Q1. The added value of OT over a simpler outcome-comparison rule is unclear. Have you tested a simpler outcome distance with the same state matching and summarisation, and how do its performance and support compare with SCOCS?

Yes, we ran exactly the baseline the reviewer suggested. The simple total-variation (TV) outcome distance is identical to SCOCS except for the pairwise distance: TV gives the planner more records per query than SCOCS but no better reward, and it is worse at future prediction: more support does not mean better reuse. The gain of SCOCS comes from recursive successor comparison, which requires OT: TV does not use a ground metric between successor states and therefore cannot recursively lift successor-state distances to the distribution level, whereas OT (Wasserstein) defines exactly this bisimulation-style recursive metric. This is also why Immediate OT (γ=0) performs on par with Simple TV, while Full SCOCS is consistently better.

  • Evidence (online, Deathmatch):
MethodRewardKillsNon-empty query rateAvg. support records per query
Simple TV11.134.7593.5%9.87
Immediate OT (γ=0)10.124.0090.7%10.92
Full SCOCS13.245.7587.4%8.46
  • Evidence (prediction error, 1 − p(observed class), lower is better; CI is the paired bootstrap 95% CI of SCOCS − Simple TV):
Eventsupport count kFull SCOCSImmediate OT (γ=0)Simple TVRandom same-stratum95% CI (SCOCS − Simple TV)
Killk<=40.6230.6810.6830.781[−0.084, −0.023]
4<k<=80.6160.6710.6700.792[−0.078, −0.028]
k>80.5860.6300.6290.769[−0.061, −0.027]
Armor gaink<=40.6110.6970.6960.790[−0.110, −0.045]
4<k<=80.5940.6740.6710.792[−0.106, −0.051]
k>80.5780.6320.6300.770[−0.073, −0.036]
Ammo gaink<=40.5750.6610.6650.772[−0.109, −0.044]
4<k<=80.5700.6530.6510.780[−0.109, −0.054]
k>80.5480.6030.6030.754[−0.074, −0.035]
Health pickupk<=40.6370.7030.7070.802[−0.088, −0.030]
4<k<=80.6400.6910.6920.808[−0.074, −0.027]
k>80.6210.6620.6620.783[−0.058, −0.024]

W2 & Q2. The hybrid results do not separate learned execution, persistent goals and planning frequency. How do rule-based and VLA execution compare at a matched interval, how does interval 4 compare with 16 with checkpoint and memory fixed?

We decoupled executor type from planning interval in a 2 × 2 design with the same VLA checkpoint and memory configuration; all cells use the same seeds with spatial indexing and SCOCS enabled. The two components play complementary roles. The VLA, pre-trained on human gameplay, gives closed-loop, reactive local control at every step (e.g., aiming and shooting), but it cannot make long-horizon decisions on its own. The VLM provides those decisions (where to go, which skill to pursue) and keeps them as persistent goals. Because the VLA handles local control between planner calls, hybrid performance is insensitive to the planning interval, and we use the 16-step interval for inference efficiency.

  • Evidence (reward / kills / efficiency in seconds; reward and kills on Deathmatch seeds 700–719):
ExecutorVLM Interval 4Interval 16
VLM + Rule13.24 / 5.75 / 5.522.22 / 1.15 / 1.01
VLM + VLA14.45 / 6.00 / 5.8015.33 / 6.45 / 1.30

W3 & Q3. Table 3 contains undiscussed reversals. What are the paired 95% CIs for the two reversals, and what do trajectories reveal?

The apparent performance reversals in Table 3 are not statistically significant under our evaluation budget. As discussed above, the hybrid design is intended to reduce inference latency. The VLA can take over local control from the VLM, allowing the VLM planning interval to increase from 4 to 16 steps and substantially reducing inference time. Comparing VLM-only execution (interval 4) with VLM–VLA execution (interval 16) without SCOCS, the hybrid-minus-VLM reward differences are −1.00 in Deathmatch and −0.55 in Defend the Center, while both SCOCS interaction CIs also include zero.

  • Evidence (Table 3 mean reward, spatial indexing enabled, SCOCS off and on; difference = with interface − without interface):
ScenarioSCOCSWithout interfaceWith interfaceReward differencePaired difference 95% CI
DeathmatchOff6.675.67−1.00[-2.82, 1.96]
Defend the CenterOff7.206.65−0.55[-1.45, 1.10]
  • Evidence (reward DiD for Deathmatch and Defend the Center, derived from the rounded Table 3 means; DiD = SCOCS-on interface effect − SCOCS-off interface effect):
ScenarioReward DiD (SCOCS on − off)Paired bootstrap 95% CI
Deathmatch+3.09[-7.17, +9.91]
Defend the Center+2.95[-4.85, +5.05]
Reviewer_KA9cOpenReview ↗
Rating
3(+1)
Confidence
3
Informativeness
2
Presentation
3
Soundness
2
Read review
Paper summary

GameWISE organizes an RGB-only game agent's experience around visually re-identifiable places while treating the applicability of a skill as dependent on observable conditions and outcomes. A pose-free world index divides interaction into local segments with visual checks for starting conditions, progress, and endpoints. Skill-Conditioned Cross-Place Outcome Compatibility Sharing (SCOCS) compares empirical outcome distributions using optimal transport and finite-depth successor comparisons. Skill-specific complete-link groups allow records to be consulted across places without merging the navigation graph. Missing observations are handled pessimistically, and the VLM uses the resulting summaries alongside current RGB to choose local goals. Screened source-game segments also train a small visual-action controller, enabling a hybrid system with less frequent VLM calls and persistent goal-conditioned execution. Experiments cover navigation and combat in ViZDoom and transfer to Xonotic and Sauerbraten without policy training on those target engines. The target engines still require online memory acquisition and the shared, calibrated observation interface. The paper also reports spatial-index controls, all combinations of three system modules, threshold sensitivity, and paired SCOCS component ablations.

Critical strengths

The separation of spatial identity from skill-specific applicability is both well motivated and tested. SCOCS keeps place identities and source records intact while allowing different sharing relations for different skills. Table 4 provides a useful spatial control: generic keyframe topology has higher place R@1 than the proposed index, but substantially more duplicate nodes and lower navigation SPL. Table 5 then holds the library, spatial index, planner, executors, and retrieval cap fixed while varying experience organization. SCOCS increases supporting records per query from 2.68 to 8.46 relative to direct state-graph retrieval, with higher mean reward. Together, these results substantiate the proposed organization of experience rather than merely showing that a larger combined system works.

The paper includes consequential controls for the distinctive sharing criterion and separates several system-level interpretations. On the same frozen library and 20 paired evaluation seeds, unconditional pooling, immediate-outcome-only comparison, and optimistic-bound grouping reduce reward by 4.24, 3.12, and 4.86 respectively. The shared-planner VLM comparisons, full module-combination table, and source-only adaptation/cross-engine evaluation further support practical relevance. The authors explicitly disclose the hybrid's different call frequency and the target engines' online memory acquisition, which helps avoid attributing all system gains to one component.

Critical weaknesses

The acquisition/reuse protocol is not specified concretely enough to reproduce or fully interpret the main results. Appendix B.2.3 states that each environment starts empty, receives a fixed interaction budget, and is then evaluated on held-out views or changed conditions; it also distinguishes frozen memory from online updates. However, the numerical acquisition budget, memory capacity, reset unit across maps/episodes/seeds, and which main-table results use frozen versus updating memory are not stated. The cross-engine map identities, episode horizon, and exact coverage computation are also missing. It is also unclear whether the simulation clock advances while a blocking VLM request is pending, which is needed to interpret performance alongside the different planner frequencies and step latencies. These details determine how much prior experience is available and what is being measured as transfer or online acquisition. The 20 source-game collection episodes in B.2.4 do not specify this separate test-time memory-building budget. Likewise, B.1 describes pretraining but omits the post-training dataset size, update schedule, and optimizer settings needed to reproduce the adapted controllers. This is the most consequential reservation: the controlled comparisons are promising, but their operating conditions need a reproducible specification.

The deployed depth fallback leaves the force of the conservative compatibility test unclear. Equation (6) requires every pair in a sharing group to pass the chosen depth's upper-bound test, while A.2.3 tries a depth-2 group, then a depth-1 group, then local records (p. 15). The manuscript does not define when a depth-2 group is considered unusable, how singleton groups are handled, or whether a depth-1 group may be consulted after depth-2 rejection caused by observed successor-skill disagreement rather than missing evidence. In that case, depth-1 compatibility of outcomes and observable successor states could allow consultation of records whose subsequent skill outcomes disagree. This is a clarification about the effective deployed rule, not a claim that the finite-depth equations are invalid. The component ablations show that recursion helps overall, but do not reveal how often the fallback bypasses deeper evidence. The distinction matters to the central claim about when experience is considered reusable.

Questions for authors

Please specify the acquisition-step budget, memory limits/eviction, collection seeds, and reset unit, and label the frozen-memory versus online-update setting for Tables 1-5. For Xonotic and Sauerbraten, which maps, horizons, and coverage definition were used? Does the simulation/episode clock advance during blocking VLM requests? Please also provide the retained adaptation-data counts and post-training schedule. A compact configuration table or the exact evaluation configurations would resolve much of the principal reproducibility concern.

What triggers the depth-2 to depth-1 fallback? Does observed incompatibility in subsequent skill outcomes block borrowing at depth 1, or is it intentionally allowed? How are singleton groups treated, and what proportion of queries obtains depth-2, depth-1, local-only, or no evidence? These answers would determine how strongly the recursive compatibility criterion constrains the evidence presented to the planner.

Is there retained query-level evidence showing whether accepted cross-place groups predict subsequent skill outcomes on held-out interactions, preferably stratified by source support count and fallback depth? A small diagnostic on the existing runs would help distinguish increased support from genuinely reliable transfer under sparse observations. The missing-observation bounds are explicitly not sampling-error bounds, so the aggregate gains alone do not establish held-out predictive reliability of individual sharing decisions.

3 comments

We sincerely thank Reviewer KA9c for the detailed review and insightful questions, which we address point by point below.

W1 & Q1. The acquisition/reuse protocol is not specified concretely enough to reproduce the main results (acquisition budget, memory limits, seeds, reset unit, frozen vs online labels, cross-engine maps, simulation clock, adaptation data).

For Tables 1 and 2, each VLM baseline starts from empty memory in each environment, runs 20 acquisition episodes, and builds its own memory. For the ablation studies, memory variants are constructed from a shared frozen recorded history. In both settings, each method is then evaluated on 20 episodes using a frozen persistent memory library. During evaluation, memory is updated within each episode, but these updates are not persisted to the library or carried over to subsequent episodes. Every evaluation episode starts from the same frozen library for the corresponding method or variant, and end-of-episode reflections and insights are not written back to it. Memory is never transferred between environments.

Key quantities (full details will be added to Appendix B):

ItemValue
Step budgetHorizon 4,200 tics (Deathmatch), 2,100 tics (My Way Home, Defend the Center,Xonotic, Sauerbraten); 20 FPS
SeedsAcquisition 600–619 (20 episodes); evaluation 700–719 (20 episodes), frozen persistent library with episode-local memory updates
Memory cap / evictionno capacity limit, no eviction, retrieval ≤ 3 summaries, 768 tokens
Cross-engine mapsXonotic: Stromkeep; Sauerbraten: metl4; coverage = visited grid cells / reachable grid cells; each map contains 100 reachable grid cells.
Planner / VLM blockingQwen3.5-9B, temperature 0; every 4 steps (16 for Hybrid); clock paused while waiting
Post-trainingWarm-started from the pre-adaptation checkpoint (weights only); 427 screened segments from 20 acquisition episodes; AdamW, lr 6e-6, wd 1e-4, β = (0.9, 0.999), 500 warmup steps, min-lr ratio 0.1; 1,000 steps, effective batch 8; 16 frames at 512×512, 70% augmentation.

W2 & Q2. With the depth fallback, it is unclear how much the conservative test actually constrains borrowing. What triggers the fallback and how often does each level occur?

The fallback never borrows across observed incompatibility. Falling back from depth 2 to depth 1 is triggered only when the evidence is insufficient to confirm compatibility, meaning some candidate has lower ≤ ε < upper. Any record pair observed to be incompatible at depth 2 remains excluded at depth 1. If every candidate's lower bound already exceeds ε, the incompatibility is observed and borrowing is blocked.

As defined in the manuscript (line 395), a group is a "non-singleton skill-sharing group at the stated depth." Singleton cases therefore fall under "Local only."

The 2,566 queries in the SCOCS run break down by retrieval level as follows:

LevelCountPercentage
Depth-2 group203779.4
Depth-1 group1867.2
Local only200.8
No evidence32312.6

Q3. Is there query-level evidence that accepted groups predict held-out outcomes, stratified by support and depth?

Yes. We added a held-out diagnostic on real evaluation queries, using only the frozen library, which was collected before any of these queries. SCOCS groups predict held-out outcomes better than both simpler sharing rules for all four event types, and much better than random same-stratum records.

  • Evidence (held-out joint (event, duration) prediction error, 1 − p(observed class), lower is better; CI is the paired bootstrap 95% CI of SCOCS − Simple TV):
Event @ durationsupport count kFull SCOCSImmediate OT (γ=0)Simple TVRandom same-stratum95% CI (SCOCS − Simple TV)
Killk<=40.6230.6810.6830.781[−0.084, −0.023]
4<k<=80.6160.6710.6700.792[−0.078, −0.028]
k>80.5860.6300.6290.769[−0.061, −0.027]
Armor gaink<=40.6110.6970.6960.790[−0.110, −0.045]
4<k<=80.5940.6740.6710.792[−0.106, −0.051]
k>80.5780.6320.6300.770[−0.073, −0.036]
Ammo gaink<=40.5750.6610.6650.772[−0.109, −0.044]
4<k<=80.5700.6530.6510.780[−0.109, −0.054]
k>80.5480.6030.6030.754[−0.074, −0.035]
Health pickupk<=40.6370.7030.7070.802[−0.088, −0.030]
4<k<=80.6400.6910.6920.808[−0.074, −0.027]
k>80.6210.6620.6620.783[−0.058, −0.024]

Thank you for the detailed response. It has addressed my concerns. I will increase my score.

Reviewer_KurMOpenReview ↗
Rating
3(+1)
Confidence
3
Informativeness
2
Presentation
3
Soundness
3
Read review
Paper summary

The paper studies how an RGB-only agent in an unfamiliar 3D game can reuse its own accumulated interaction experience. The authors argue that whole trajectories are too brittle to retrieve or replay, and that place identity alone is an insufficient context for reuse (the same room calls for different behavior at low vs. full health).

The authors propose to use a world-index: a pose-free topological graph whose nodes are visually re-identifiable places (multi-view RGB scans, re-identified via ORB matching and RANSAC verification) and whose edges are the action–observation segments between them, with step-wise correction and endpoint verification during reuse. SCOCS: a bisimulation-inspired criterion for sharing a skill's experience across places. States are (place, z), where z contains discretized HUD values (health, armor, ammunition) read by OCR plus auxiliary visual features. For each skill, empirical distributions of (outcome, successor) pairs are compared with optimal transport, recursively to finite depth; per-skill complete-link groups (all pairwise distances ≤ ε) determine which records are pooled, without merging places. A frozen Qwen3.5-9B planner selects skills and grounds targets using retrieved evidence; screened successful segments are used to adapt a 0.8B VLA via goal-conditioned behavior cloning. In GameWISE-Hybrid the VLM is called every 16 steps and the VLA executes in between.

Critical strengths
  1. Most experience-reuse methods for LLM agents are developed in text environments with discrete, easily matched states. Extending experience reuse to pixel-only, continuous 3D worlds without pose or maps is a meaningful direction.
  2. Table 4 shows nicely that place-retrieval accuracy alone is a poor proxy for index usefulness (duplicate nodes fragment experience, false merges corrupt it). Table 3 shows task-dependent contributions (spatial index for navigation, SCOCS for combat). Appendix C provides seed-paired ablations of the SCOCS criterion with paired confidence intervals and a threshold-sensitivity sweep showing a plateau.
  3. Experiments required a lot of engineering to get this to work on these games.
Critical weaknesses
  1. The paper combines a topological place graph, a bisimulation-style sharing rule, and a hierarchical VLM/VLA controller, plus active scanning, handoff rules, OCR, data screening, and retrieval fallback logic. The headline results (Tables 1 and 2) evaluate the full system, which differs from every baseline in several respects simultaneously. As a result, it is difficult to identify what the core scientific contribution is or what a reader should take away beyond "this engineered system works well on these games."
  2. The paper is dense and heavy on terminology ("situated experience," "world-indexed," "skill-conditioned cross-place outcome compatibility sharing," "persistent VLM–VLA execution interface"), and many essential details (state and outcome definitions, d_0 and δ_ω, the conservative bound, the handoff rules) are deferred to the appendix. Figure 1 is visually busy and hard to follow. A simple running example in the main text, showing a concrete state key, two places, and how SCOCS decides whether to pool, would make the method much easier to understand.
  3. Limited evaluation: the evaluation is small (20 episodes per cell, few environments, no variance in main tables). I believe the work needs a sharper contribution, fairer comparisons, and a broader evaluation before it is ready for ICLR.

I am willing to change my rating if my issues are addressed.

Questions for authors
  1. What is the single central contribution of the paper, and which experiment isolates it?
  2. How much manual effort does the OCR/HUD calibration and visual-feature specification require per game? How does performance degrade with noisy or missing HUD readings, or with learned state features?
3 comments

We sincerely thank Reviewer KurM for the careful review and constructive feedback, which we address point by point below.

W1 & Q1. The full system differs from the baselines in several respects at once, so the core contribution is unclear. What is the single central contribution, and which experiment isolates it?

The central contribution is skill- and outcome-conditioned selective sharing (SCOCS): a skill's records are shared across places only if that skill yields compatible outcomes under the same observable state. The world index is the infrastructure this mechanism runs on. The existing SCOCS component ablation in Appendix C isolates this contribution using a frozen experience library, with the spatial index, planner, executors, and retrieval budget held fixed while only the sharing rule varies.

To further test whether the shared records predict what will happen at the current place, we have now conducted a new held-out prediction error experiment. The error is defined as 1 − p(observed class), where p is the predicted probability of the observed outcome class; lower is better. SCOCS achieves an error of 0.600, compared with 0.663 for both immediate-outcome OT and simple TV. Together with higher task performance despite a lower non-empty query rate, these results support the value of selecting applicable records rather than simply increasing retrieval coverage. Further prediction results broken down by event type are provided in our response to Reviewer KA9c (Q3).

  • Evidence:
Sharing ruleNon-empty <br />query rateRewardKillsheld-out prediction error
Immediate-outcome OT (γ=0)90.7%10.124.000.663
Simple TV outcome distance93.5%11.134.750.663
Full SCOCS87.4%13.245.750.600
Random same-stratum records (diagnostic only)–––0.783

W2. The paper is dense and terminology-heavy, key definitions are in the appendix, and Figure 1 is hard to follow.

We will move the definitions of state, outcome and the conservative bound into the main text, cut the terminology down to the minimum, redraw Figure 1 as a single information-flow diagram, and add a running example. SCOCS can be stated in one sentence: two places share a skill's records only if, under the same observable state, that skill produces compatible outcome distributions, and missing evidence counts as incompatible. The running example takes one state key, two rooms and one skill, and shows step by step how the outcome histograms are compared and how the upper bound is checked against ε = 0.15 before the records are pooled or kept separate.

W3. Limited evaluation scale and fairer comparisons.

We thank the reviewer for this suggestion. The three ViZDoom scenarios in the main text were selected deliberately, with each one isolating a different ability that experience reuse should support. Deathmatch requires spatial memory and immediate decision-making together; My Way Home is a clean navigation test with no combat; Defend the Center is one of the harder shooting scenarios in ViZDoom. Each cell uses 20 evaluation seeds, and the appendix reports seed-paired differences with bootstrap 95% CIs and Wilcoxon signed-rank tests on Deathmatch, together with paired intervals for the SCOCS component ablation. To broaden coverage, we additionally evaluated all methods on three more ViZDoom scenarios (Health Gathering Supreme, Deadly Corridor, Defend the Line), shown below. In the revision we will run more seeds and add more maps on the two other engines.

Regarding fairness, all VLM baselines and GameWISE-VLM use the same frozen Qwen3.5-9B planner, observation interface, rule-based skill executors, action space, four-step decision interval, and interaction budget. Only the memory mechanism differs: how past experience is stored, organized, and retrieved for the planner.

  • Evidence (three additional ViZDoom scenarios, reward, mean ± 95% CI half-width, n = 20 seeds):
MethodHealth Gathering SupremeDeadly CorridorDefend the Line
No memory275.60 ± 32.63−60.88 ± 53.608.20 ± 2.89
ReAct + Reflexion306.80 ± 30.06−17.23 ± 41.948.80 ± 1.98
SPTM333.60 ± 38.10−57.64 ± 50.129.30 ± 1.47
ExpeL300.05 ± 30.31−12.75 ± 57.1711.55 ± 2.72
RAP299.60 ± 35.53−0.58 ± 68.8010.30 ± 1.82
JARVIS-1356.00 ± 41.760.72 ± 61.7112.50 ± 1.44
GameWISE-VLM438.80 ± 99.30−3.56 ± 53.4314.80 ± 2.62
GameWISE-VLA409.80 ± 68.85106.76 ± 74.028.95 ± 2.08
GameWISE-Hybrid488.75 ± 71.5981.26 ± 45.9814.70 ± 1.75

Q2. How much manual effort do OCR/HUD calibration and visual features require per game, and how does performance degrade with noisy or missing HUD readings or with learned features?

For each game, the calibration and testing of HUD and visual features take approximately one hour of human effort, covering the specification of HUD bounding boxes and the setting and checking of valid value ranges. Each field has a plausible range (health ∈ (0, 200], armor ∈ [0, 200], ammo ∈ [0, 999]); an out-of-range reading is set to unknown rather than accepted.

In practice, OCR failures mostly show up as wrong digit counts (a dropped or spurious digit), which push the value far outside the plausible range; the range check catches these and marks them unknown instead of passing on a wrong state. A single misread digit is rarer, and the coarse binning absorbs part of it: on covered frames, exact health is 94.5% correct, while the health bin used in the state key is 100.0% correct. Under heavier degradation, the cost appears mainly as borrowing less rather than borrowing wrongly: dropping readings makes the system abstain instead of matching on a wrong state. Dropped fields mainly reduce coverage, while noisy-but-plausible readings can pass the gate and mainly reduce matching correctness. Detailed analysis below reveals the graceful degradation properties of our pessimistic formulation.

  • Evidence (offline accuracy on evaluation seeds)
FieldCoverageAccuracy on covered framesOverall accuracy (unknown counted as wrong)
Health bin (used in the state key)94.5%100.0%94.5%
Exact health value94.5%94.5%89.3%
Weapon slot89.2%90.3%80.6%
Armor91.7%91.2%83.6%
  • Evidence (online rerun):
SettingState sourcen (seeds)RewardKillsNon-empty query ratecurrent_self_state_unknown
Engine whitelistEngine variables2014.506.1592.5%0.0%
hud_ocr(SCOCS)Range-limited RGB OCR2013.245.7587.4%8.9%
hud_ocr_noise25OCR + 25% noise2010.204.6066.4%11.3%
hud_ocr_drop25OCR + 25% dropout209.354.0034.8%58.9%

Integrating learned state features into SCOCS requires further research, particularly on how to use them for reliable state matching and outcome comparison.

Reviewer_KurMOpenReview ↗

Updating my score

I thank the authors for the detailed rebuttal and the additional experiments. Naming SCOCS as the central contribution makes the paper’s main takeaway much clearer.

For the final version, I suggest discussing Castro & Precup (AAAI 2010) and lax bisimulation (Taylor et al., NeurIPS 2008). Both are closely related to SCOCS’s per-skill comparison and its conservative/optimistic bounds. I also suggest reporting variance for Tables 1 and 2.

From OpenReview, CC BY 4.0. Authors as signed there; shown unedited.

Related papers

Loading the map…

Discussion (0)

Sign in to comment.