What Should I Try Next? Decision-Relevant Counterfactual Information Gain for Active World Models

World models enable agents to evaluate actions through imagined futures, yet these predictions are useful only when the underlying environment dynamics are sufficiently identified. Existing exploration methods typically seek novel states, large prediction errors, local model disagreement, or generic information gain. Such objectives may spend costly interactions resolving uncertainty that has little effect on subsequent decisions. We introduce Active Epistemic World Modeling (AEWM), a framework that asks which real interaction should be performed now to make relevant counterfactual futures more reliable. AEWM distinguishes irre- ducible outcome noise from reducible uncertainty about latent mechanisms and targets decision-relevant counterfactual information gain. Its practical Counterfac- tual Disagreement Reduction (CDR) score estimates query-set variance reduction by sampling hypothetical outcomes and reweighting ensemble members without inner-loop retraining. In a matched five-seed study on 60 held-out zero-delay hidden-physics worlds, CDR lowers one-step prediction RMSE at 32 interactions by 19.7% relative to local disagreement. It modestly improves budget-curve area relative to random selection, but matches random at the endpoint and uses roughly 3.2× more scoring time. Historical model ablations expose a delayed-action training-target mismatch, and an all-candidate audit finds weak fidelity of the cheap CDR rankings. The completed study establishes a one-step acquisition result; planner-generated queries and downstream control remain to be tested.