acceptodds
Under review as a conference paper at ICLR 2027

Which Value Head Missed? Outcome-Resolved Proper-Score Excess for World-Model Control

Abstract

A world-model forecast can be uncertain without a consequential outcome, while a confident reward, cost, or termination forecast can resolve with unexpectedly large proper loss. The environment response therefore exposes a learning signal unavailable at forecast time: which value head realized loss above that head's self-expected score. We define outcome-resolved value score (ORVS) as the positive, nominally standardized proper-loss residual and introduce Outcome-Resolved Score Control (ORSC), which propagates immutable collection-time labels through a terminal-masked continuation critic into guarded actor updates. Under a common late-update path, ORVS raises shifted return over uncertainty by points with similar nominal return; a separate ranking analysis finds that its top 10% contains of subsequent harmful continuations, versus for predicted cost and for uncertainty. The control effect recurs with a TD-MPC-style agent ( shifted-return points) and unseen visual distractions ( retention), while SafetyPoint and SafetyCar cost rates reach 2.43 and 3.07 per 1,000 steps.

Then back it, or bet against it.

Related papers

Open the market on this paper to see 7 more related papers.