ErrataWM: Correcting Frozen World Models with a Non-Parametric Memory of Their Own Errors

Latent world models enable strong planning, but once trained they are static: deployed in new environments with shifted dynamics they keep making the same systematic prediction errors, and those errors flow straight into the planner. The usual remedy, gradient-based test-time training, is expensive, rewrites the whole model, and tends to forget the environments it was already good at. We first establish an empirical premise: a frozen world model's errors are not noise but a function of the starting context, so that similar states yield similar mistakes. This holds broadly, across navigation, manipulation, and driving, five world models, and four architectures including regression, autoregressive, and video diffusion models, with context-specific predictability as high as 0.81. Building on this, we propose ErrataWM, a frozen world model paired with a non-parametric memory that turns its recurring errors into scope-gated correction prototypes and applies them at test time with no gradient updates. Because the memory operates in a shared frozen feature space, one model corrects heterogeneous world models across tasks. ErrataWM reduces one-step prediction error by a context-specific margin of 0.12 to 0.15 over a context-blind control, matches gradient-based adaptation without any weight update, and improves downstream success by 9 to 13 points on navigation and manipulation. A single shared memory corrects a navigation and a manipulation world model at once and transfers zero-shot to an unseen driving diffusion model, reaching success competitive with recent test-time world-model methods at a fraction of their adaptation cost.