GameWISE: Learning World-Indexed Situated Experience for Adaptive Decision-Making in 3D Games

Visual agents in unfamiliar 3D environments accumulate long interaction streams that are brittle to retrieve or replay as monolithic trajectories. We introduce GameWISE, a world-indexed situated experience reuse framework in which visually re-identifiable places partition long-horizon interaction into locally retrievable and verifiable segments. To determine where skill experience remains behaviorally reusable, Skill-Conditioned Cross-Place Outcome Compatibility Sharing (SCOCS) compares skill-conditioned empirical outcome distributions using optimal transport and recursively evaluates finite-depth successor compatibility under matched observable conditions. Screened successful segments adapt a visual–action controller to take over high-frequency low-level actions, while the VLM uses retrieved spatial and outcome evidence to issue persistent goals and replan on events. Under a common protocol in ViZDoom, GameWISE achieves the strongest system-level results. On held-out Xonotic and Sauerbraten, GameWISE acquires and reuses situated experience online, achieving the best task performance without target-engine policy training.