acceptodds
Under review as a conference paper at ICLR 2027

Learning to Correct Is Not Enough: Reusing Feedback in World Models

Abstract

Accurate short-path correction does not guarantee reliable reuse along longer action paths. We study how a single observation can revise multiple cached futures of a world model without regenerating them. A matched construction adds one known return without changing the unknown response or calibration data, turning horizon-independent minimax decision risk into risk growing with path length until saturation. Legal continuations determine which response uncertainty is repeatedly exposed to the task. For known compact classes of orthogonal response dynamics on a fixed finite directed graph, we characterize when Gaussian-noisy scalar observations on a prescribed short-path bank suffice for reliable decisions along all legal continuations. Learning is possible exactly when, in every strongly connected component and at every positive task precision, only finitely many irreducible types of the joint return action remain task-significant across the candidate class. This permits infinitely many distinct return types when their task influence vanishes. Compatible internal coordinate changes cancel in composition, bounding approximation error per visited component rather than per action; under this criterion, longer paths need not demand finer calibration. We further characterize when all path responses admit a fixed finite exact linear calibration with model-independent coefficients, while showing that such exact linear tables can require exponentially many entries. On a continuous nominal base, even a smooth, uniformly observable class sharing one known response type can preclude uniform all-path learning from noisy short-path data. Controlled studies and simulated planning examine these distinctions and the trade-offs of cached correction. Calibration sufficiency therefore depends not only on what the data reveal, but also on which unresolved response differences future use can make consequential.

Primary area:
learning theory
Keywords:
world models, learning theory, system identification, model-based planning, feedback correction, dynamical systems

Reviews

Among:
Shape:
1234this paper, Top 30%
Figure 1. Mean review rating of the reviewed papers at ICLR 2027. This paper averages 2.67 over 3 reviews, above 70% of the others.
Reviewer_rDMMOpenReview ↗
Rating
3
Confidence
1
Informativeness
2
Presentation
2
Soundness
2
Read review
Paper summary

This paper studies the reliability of reusing feedback-based corrections in cached world-model predictions over long action paths, showing that accurate short-path calibration does not necessarily ensure reliable long-horizon decisions. It theoretically characterizes when worst-case decision regret vanishes under noisy calibration, based on task-significant response types, and distinguishes approximate learning from exact finite calibration. The paper also proposes an observable-endomorphic world model (OEWM) for cached feedback correction and evaluates it on Meta-World, demonstrating improved planning performance over several baselines and substantially lower feedback-time latency than conditioned rerolling.

Critical strengths
  • The paper identifies an important limitation of short-path calibration for long-horizon feedback reuse and provides a principled characterization of when worst-case decision regret vanishes. The distinction between approximate all-path learning and exact finite calibration, together with the analysis of path composition and deployment distributions, offers useful theoretical insights.

  • The paper combines controlled studies with visual planning experiments on Meta-World. Results show that cached feedback correction can improve decision quality over pooling-based and recurrent baselines while substantially reducing feedback-time latency compared with conditioned rerolling. The use of matched comparisons and explicit computational accounting strengthens the practical evaluation.

Critical weaknesses
  • The theoretical guarantees rely on a known compact class of orthogonal response dynamics on a fixed finite graph. Their applicability to neural world models with approximate, potentially misspecified dynamics remains unclear, and the visual experiments do not establish that OEWM satisfies the assumptions required for the all-path guarantees.

  • The visual evaluation focuses primarily on one Meta-World task setting, leaving generalization across tasks, environments, and world-model architectures insufficiently explored. Moreover, the advantage over conditioned rerolling in decision regret is statistically unresolved, the necessity of individual architectural components is not fully established, and the overall computational advantage depends on cache preparation and reuse frequency.

Questions for authors
  • How sensitive are the theoretical conclusions and practical correction performance to violations of the assumed orthogonality and response-class structure? Can the authors provide additional analysis or experiments on model-class misspecification, and clarify how the theoretical characterization can guide the design of practical neural world models?

  • Could the authors evaluate OEWM on additional tasks or environments and conduct targeted ablations to isolate the contributions of its response-transport mechanism and innovation-dependent gate? In addition, could they report end-to-end computational costs, including cache preparation, and characterize the reuse frequency at which cached correction becomes more efficient than conditioned rerolling?

3 comments
AuthorsOpenReview ↗

Response to Reviewer rDMM (1/3): summary and misspecification

Thank you for the careful reading and the concrete questions. We are glad that the distinction between approximate learning and exact calibration came across. Your questions led us to four new experiments, and every point has a direct answer. In brief:

  • Violations (W1, Q1). The lower bounds carry over to any larger class. For a fixed estimate, a violation of size η\eta costs an error controlled by ηH\eta H on plans of length HH, and new experiments with non-orthogonal and out-of-class truth measure the effect (below, Part 3).
  • Ablations (W2, Q2). Two new readers complete the 2×2 design around Table 1. Replacing the path transport by pooling raises regret by .044 with the gate and by .025 without it, and removing only the gate raises it by .016 [.002, .030] (Part 2).
  • Breadth (W2, Q2). In a new comparison on 23 Meta-World tasks, repeated six times, cached correction is not separated from regeneration (+0.26 success points [−0.24, +0.72]) and is 3.0 points above a reader without feedback. The submission also covers two pretrained world models (Part 2).
  • Cost (W2, Q2). Of the 8.5 s of cache preparation in our re-run, 6.7 s is candidate generation and simulator setup, which regeneration also needs. When the plan is chosen with the world model before feedback, as in our setting, cached correction is cheaper from the first feedback event (new, Part 3).
  • Design guidance (Q1). Five rules follow from the theory, each with evidence (Part 2).
  • Regeneration (W2). The claim is a much lower feedback cost, not lower regret (Part 3).

Q1 (W1). Sensitivity to violations of orthogonality and of the assumed class

You are right that the guarantees assume an idealized class. OEWM itself assumes neither orthogonality nor a known class. For the theory:

  • The obstructions are robust. The lower bounds of Proposition 4.3, of the necessity part of Theorem 4.2 and of Proposition F.1 are worst-case bounds over a class. They therefore carry over to every larger class that contains their constructions, including non-orthogonal and misspecified ones. The next two points cover the sufficiency results.
  • A violation costs an error controlled by ηH\eta H. Suppose every true transport SaS_a is within operator distance η\eta of a member TaT_a of the class. Telescoping gives ∥Sp−Tp∥≤(1+η)∣p∣−1\|S_p-T_p\|\le(1+\eta)^{|p|}-1 for every legal path pp. On plans of length HH, every score of a fixed estimate, and hence its regret, therefore moves by at most (1+η)H−1(1+\eta)^H-1, which is about ηH\eta H when ηH\eta H is small.
  • New experiment: non-orthogonal truth. We used the scalar study of Table 3: 64 systems, 64 repeats.
    • Each true transport became Sa=Taexp⁡(ηXa)S_a=T_a\exp(\eta X_a), with XaX_a symmetric, traceless and of unit norm.
    • The estimators are unchanged and still assume rotations.
    • Direction regret (1−cos⁡)/4(1-\cos)/4, as in Table 3, on alternating return queries at H=2048H=2048:
η\etaCorrect relationRelaxed fitExact rotation part of the truthFlagged by return probes
04.3×10⁻⁷.07400/64
10⁻⁴1.7×10⁻⁴.0751.7×10⁻⁴0/64
10⁻³.012.091.0124/64
10⁻².077.178.07754/64
  • The correct relation keeps its advantage over the relaxed fit at every η\eta: −.074 to −.101, with all 95% intervals excluding zero. Even at ηH=2\eta H=2 it is more accurate than the relaxed fit is with no violation at all.
  • Its remaining regret is within 10⁻⁴ of that of the truth's own rotation part. The loss therefore comes from the violation, not from estimation, and it is small whenever ηH\eta H is small.
  • A wrong relation. The same probes detect it: 64/64 systems at 0.01 rad, after which the validated fit falls back to the order-only fit (.065 instead of .268).
  • Practical performance. The Meta-World results are empirical and do not rely on these assumptions, and the paper claims no all-path guarantee for OEWM there (Sections 6 and 7). The pretrained world model's response class is unknown, so we test three related facts.
    • The structure can be learned. In a controlled visual composition task, a per-action transport learned from short data recovers the true transports to within 0.70–1.96% (three seeds; App. H.6). It extrapolates under small violations (Part 3).
    • Short training transfers to long plans. On Assembly, OEWM trained on two to four blocks reaches 77.69% success on twelve-block plans, against 52.08% for capacity-matched pooling and 52.34% without feedback (Section 6, Table 29).
    • The response is largely predictable one step ahead and is branch-specific (new). On the test cells of Table 1, a linear map of the response at one step explains 86–92% of the response at the next, and the median ratio of the response norm between steps is 0.82–1.03. Even an oracle correction shared by all 125 candidates explains only 48% of the squared change that regeneration makes (reply to 8VNC, W1).
AuthorsOpenReview ↗

Response to Reviewer rDMM (2/3): design guidance, ablations and breadth

Q1b. How the theory guides the design of neural world models

  1. Validate on returns, not only on short paths. Fits with no greater training loss differ by five orders of magnitude in long-horizon regret on alternating queries (Fig. 3). Return probes reject the 0.01-rad wrong relation of Table 3 (64/64) and flag a large non-orthogonal violation (54/64 at η=10−2\eta=10^{-2}; Part 1). With pixel-decoded observations (six seeds), a held-out return word predicts which fits fail on returns of plan length 64, and short-path error does not (Part 3).
  2. Spend calibration on returns. At a matched action budget, return probes reduce the long-horizon regret of fits that learn the relation from .0133 to .0010, and the gain persists when the true relations deviate (App. H.4).
  3. Treat the action library as part of the problem, and calibrate rather than restrict it. One known return can change the achievable risk without any new data (Proposition 4.3). In the wrist study, removing reversals cuts selection regret inside the library from .471 to .008 but gives up twelve to forty times as much capability, and calibration with eight words beats restriction at every tested length and noise level (App. H.5).
  4. Compose the correction along the path. In a controlled composition task, a transport learned from one- and two-block data extrapolates to twelve blocks, where pooling, attention and recurrent readers fall to about 11% (App. H.6).
  5. Impose relations only when known, or proposed from data and validated (Table 3; reply to 8VNC, W3).

Q2a (W2). Ablations of the response transport and the innovation gate

A new factorial isolates both components in the protocol of Table 1. Two new readers complete the 2×2 design: one keeps the transport and removes the gate, and one removes both.

We trained them with OEWM's data, seeds and training protocol, at matched active parameters (about 0.97M each). Retraining OEWM's first seed with the same code reproduces the stored checkpoint exactly. On the 576 test cells and six seeds of Table 1:

ReaderPath summaryGateRegret [95% CI]Success
OEWMtransportyes.089 [.077, .101]93.46%
New: no gatetransportno.105 [.092, .118]92.82%
Pooling (Table 1)mean poolingyes.133 [.115, .154]90.10%
New: pooling, no gatemean poolingno.130 [.111, .155]90.54%
No feedback––.339 [.320, .357]72.05%
  • Replacing the transport by pooling raises regret by .044 [.026, .064] with the gate and by .025 [.008, .048] without it.
  • Removing only the gate raises regret by .016 [.002, .030]. Without the transport we detect no gate effect (−.002 [−.014, +.012]).
  • The GRU reader of Table 1 is a second recurrent reader without the gate: it feeds the innovation into the recurrence and has 10% more parameters. Its regret is .020 [.007, .037] above OEWM (Table 1).
  • Earlier ablations show that branch-specific path information is needed. Assigning the innovation to the wrong branch removes nearly all of the gain (Table 21), and an endpoint-only reader loses 8.07 [5.60, 10.68] success points (Table 13).

Q2b (W2). More tasks, environments and world models

A new comparison on 23 tasks shows that cached correction stays close to regeneration beyond Assembly. The submission also evaluates two pretrained world models and a third world model learned from pixels.

  • Tasks: new comparison with regeneration (protocol of App. H.10, Table 30). All 50 Meta-World v3 tasks were screened, and the 23 with at least two points of oracle headroom are evaluated on held-out banks (448 cells per task). Table 30 compares cached correction with the nominal plan only. We added two readers under the same protocol, conditioned regeneration and a reader without feedback, and trained all three six times (three seeds on each of two machines). Success averaged over the 23 tasks and the six runs:
Nominal planNo feedbackCached correctionConditioned regenerationOracle
83.2%84.0%87.1%86.8%96.8%
  • Cached minus regeneration is +0.26 points [−0.24, +0.72]; twelve further trainings give −0.06 [−0.38, +0.27], inside the −2-point margin of App. H.7.
  • Cached correction is 3.04 points [2.19, 3.99] above the reader without feedback.
  • Its gain over the nominal plan, 3.83 points [3.08, 4.59], is consistent with Table 30 (4.14).
  • Pretrained world models (App. H.8, Table 21). On JEPA-WM, cached correction improves success by 13.86 [13.12, 14.63] points over the nominal plan and by 1.70 [1.20, 2.22] over same-seed regeneration from the factual frame. On DINO-WM the latter contrast is +.13 [−.40, .66].
  • A world model learned from pixels (App. H.6). In a controlled composition task whose transport class contains the true generator, the learned transport extrapolates from two blocks to twelve.
AuthorsOpenReview ↗

Response to Reviewer rDMM (3/3): violations in a visual pipeline, comparison with regeneration and end-to-end cost

Q1 (W1), continued: violations in a learned visual pipeline, and the response on 23 tasks (new)

The first three runs use the visual pipeline of App. H.6: an encoder, world model and observer learned from pixels, and one free transport matrix per action learned from one- and two-block data.

  • Non-orthogonal truth inside this class (Sa=Taexp⁡(ηXa)S_a=T_a\exp(\eta X_a), η≤0.1\eta\le0.1). The transport still succeeds at four, eight and twelve blocks in all twelve seeds per η\eta, with relative error 0.6–3.7% to the non-orthogonal truth.
  • Truth outside this class. Each action's rotation angle also depends on the response direction, by up to η\eta, so no fixed matrix per action is exact. Success at twelve blocks stays at 100% for η≤0.01\eta\le0.01 and is 92.7% at η=0.03\eta=0.03, against 10.5% for the GRU reader (eight seeds; the interval is positive in each).
  • Pixel-decoded targets. With every training target also decoded from pixels, the transport succeeds in every cell in all twelve seeds (operator relative error 0.77–2.30%), while GRU and endpoint readers trained on decoded targets reach 12.7% and 10.0% at twelve blocks.
  • The theory's return effect. We ran the matched-return construction of App. H.3 in three dimensions, with every fact and calibration observation decoded from noisy pixels by a learned observer, and the transport fitted without an orthogonality constraint (nine seeds, 576 systems). For the same fit, regret grows with plan length with a known return and stays flat without it; at plan lengths 64 and 2048 the gap is positive in every cell.
  • Validate on returns (rule 1). In six of these seeds, short-path validation error does not predict which fits fail on returns of plan length 64 (rank correlation −0.07 and −0.15), whereas a held-out return word does (+0.49 and +0.50; differences +0.57 [+0.48, +0.66] and +0.64 [+0.56, +0.72]).
  • The measured response on 23 tasks. On the stored banks of the 23 tasks of App. H.10, the median norm ratio between steps is 0.81–1.08 in every task, and a one-step linear map explains 66–89% of the next-step response, more than the best shared prediction in 22 of 23.

Q2c (W2). The comparison with conditioned regeneration

We agree that the regret difference is unresolved, and the paper claims none: the advantage over conditioned regeneration is a much lower feedback cost. Regeneration recomputes every future with the new observation, so it is the reference; in the matched comparison of Table 1, cached correction is statistically indistinguishable from it.

  • In Table 1, the regret difference is −.0025 [−.0166, +.0111]. The interval excludes a disadvantage larger than .0111, about one eighth of OEWM's regret. The p99 feedback latency is about 99 times lower, or 37 times lower against BF16 regeneration.
  • In the independent sixteen-chunk study, OEWM meets a predeclared −2-point success noninferiority margin against regeneration from the factual frame, at about 300-fold lower FP32 p99 latency (App. H.7, Fig. 13).

Q2d (W2). End-to-end cost, including cache preparation, and the break-even reuse frequency

Cache preparation is shared with regeneration. When the plan is chosen with the world model before feedback, cached correction is therefore cheaper from the first feedback event.

We re-timed the cache preparation of Section 6 on the same RTX 5090, in the same process as the feedback-time arms. Their median latencies reproduce the submitted ones within 1% (App. H.9). The re-run takes 8.53 s per root (8.05 s on average in Section 6). Its median components:

  • Candidate generation: 6.23 s. This is the CPU search that constructs our benchmark's 125 candidate plans.
  • Simulator setup: 0.50 s.
  • Every method pays both of these, since regeneration scores the same 125 candidates.
  • World-model rollout of the nominal bank: 1.78 s (FP32) or 0.68 s (BF16). A planner that chooses its plan with the world model computes it before feedback arrives, however the feedback is then handled.

OEWM's own preparation adds 0.09 ms and 128 KB per reader (App. H.9). After it, a feedback event costs 4.43 ms, against 487 ms (FP32) or 183 ms (BF16) for regeneration.

Break-even.

  • In our setting both pipelines compute the nominal bank, so cached correction is cheaper from the first event, by 483 ms (FP32) or 178 ms (BF16) per event.
  • On cost alone, a pipeline that would not otherwise generate futures before feedback recovers the nominal rollout's cost after four feedback events per cache (3.7 with both steps in FP32, 3.8 with both in BF16).

We will add these results to the appendix and state the breadth and cost results in Section 6.

Your questions made the paper stronger. The new results cover the assumptions, task coverage, the components and the cost, and we are glad to address any point that remains. Thank you again for the careful reading.

Reviewer_fRd8OpenReview ↗
Rating
1
Confidence
3
Informativeness
1
Presentation
1
Soundness
1
Read review
Paper summary

The authors claim to be motivated by the following situation. First, a world model simulates possible futures. Then new information implies that those simulations need updating. Can those older, inaccurate simulations be sensibly updated to account for the new information?

They do some math and numerical experiments which are occasionally related to this motivation.

Critical strengths

None

Critical weaknesses

I had an extremely difficult time understanding this paper. It is written in a strange style which is overly technical, does not generally provide useful intuition, and does not meaningfully engage with prior work. I cannot stress enough how poorly organized it is. It reads as entirely or almost entirely LLM-generated.

Part of the problem is that the authors' many specific choices are not well-motivated. They analyze some kind of graph problem, but why that problem as opposed to a different one? Why consider orthogonal transformations? What does any of this have to do with world models as they are typically understood? There is also a major disconnect between the theory and experiments.

There is a huge amount of supplemental content, also probably mostly LLM-generated. This deluge of stuff makes it nearly impossible to understand which results matter and which do not. The authors really need to clarify this.

Finally, it is not clear to me why anyone should care about this work's results or what the practical consequences are.

Questions for authors
  1. Can the authors clarify why they chose their theoretical setting and numerical experiments?
  2. What are the practical consequences of this work, if any?
2 comments
AuthorsOpenReview ↗

Response to Reviewer fRd8

Thank you for the review. Your summary states our question exactly: whether futures that a world model has simulated can be updated when new information arrives. The answer has three parts.

  1. Updating works, and it is fast. A robot imagines 125 candidate futures with DINO-WM on Meta-World and then observes something new, such as actuator backlash. Regenerating them takes about half a second (0.18 s in reduced precision); a learned correction revises them in about 5 ms, at statistically indistinguishable decision regret (Table 1).
  2. Accuracy on short tests does not make a correction safe on long plans. Figure 1 shows why: two response models that differ by 3° per loop score identically after one observation, and after sixty loops they give opposite decisions. No learner can avoid this: with the unknown response and the calibration data unchanged, adding one known return action changes the best achievable worst-case decision risk from independent of plan length to growing with it until it saturates (Proposition 4.3).
  3. The theory says exactly when short tests suffice. Theorem 4.2's criterion is necessary and sufficient within its model class. When it holds, compatible coordinate changes cancel along a plan, and dropping weak response types costs error once per strongly connected part of the graph, not once per action, so short-path calibration can be reliable at every plan length, even with infinitely many response types. When it fails, no learner can be. Theorem 4.4 covers fixed distributions of plans.

These four results (Table 1, Proposition 4.3, Theorems 4.2 and 4.4) carry the paper. The appendix holds their proofs and controls.

Q1. Why this theoretical setting and these experiments?

  • The graph records which actions may follow which, as in a library of motion primitives, and so determines which plans can occur. The effect depends on it: a loop lets a plan repeat the same uncertain motion many times.
  • Orthogonal maps carry the effect of the new observation along a rollout, one map per action (Eq. 2). They neither shrink nor amplify it. Any growth of error must then come from how actions are composed and repeated, not from unstable dynamics (Section 4.1). The method itself needs neither the graph nor orthogonality (Section 3).
  • The experiments link theory and practice at three levels.
    1. Theory. In the matched setting of Proposition 4.3, the regret of one fitted model grows about linearly in the horizon with a return and stays nearly flat without it (median log–log slopes 1.051 and 0.083, App. H.3).
    2. Learning. In a controlled study with a world model learned from pixels, a transport fitted on short data recovers the true one to within 0.70–1.96% (three seeds) and extrapolates from two blocks to twelve (App. H.6).
    3. Practice. Point 1 above, with a pretrained world model on Meta-World Assembly: p99 latency 37 to 99 times lower than regeneration (Table 1).

Q2. What are the practical consequences?

Planners such as TD-MPC and DINO-WM recompute their futures when they replan. Recent work corrects predictions with feedback (Feedback World Model) or caches futures (RIFT). For such reuse, two rules follow.

  1. Validate on returns, not only on short paths. Fits with no greater training loss differ by five orders of magnitude in decision regret on long alternating queries (Fig. 3). In a new test whose observations are decoded from noisy pixels (six seeds), validation on a held-out return word predicts which fits fail on returns of length 64 (rank correlation +0.49 and +0.50), whereas short-path validation error does not (−0.07 and −0.15).
  2. Calibrate on returns. At a matched action budget, adding loop probes lowered long-horizon regret 9 to 13 times across the three tested conditions (App. H.4).

Prior work. Work on compounding model error (e.g., Talvitie, 2014; Janner et al., 2019) studies how one model's one-step errors accumulate in its own rollouts. We study a correction rule that is calibrated on short paths and must hold for every model in a class and every legal continuation; Section 2 contrasts this with switched-system identification, synchronization and gate-set tomography.

We will open the introduction with this explanation and the four results, and add this literature to Section 2.

On the ethics flag. The paper discloses its use of AI assistance in the AI use statement after the main text, as ICLR 2027 policy requires. It lists the tasks for which assistants were used and our own checks of the claims, proofs, code, numbers and citations. For this response we also repeated four computations from the stored code and data, for Tables 1 (one training seed), 3 and 30 and the feedback latencies; all reproduced the stored results, the latencies to within 1%. We take full responsibility for the contents of the paper. We will release the code upon acceptance.

We are glad to clarify any part that is still unclear.

I'm sorry, but I still can't understand much of what this paper is about (and I did try). Everything still appears highly LLM-generated, even these rebuttals. If I can't understand it and it doesn't seem like it involved much human effort, I can't support acceptance.

Reviewer_8VNCOpenReview ↗
Rating
4
Confidence
2
Informativeness
2
Presentation
3
Soundness
3
Read review
Paper summary

This paper proposes to learn a correction model to predict the change in scores for action sequences that have been already generated from a world model without expensive regeneration. It also mathematically formulates the situations when the corrections learns from short sequences can generalize to longer sequences (usually using some kind of deterministic hard constraints) and show that their correction model sits at the pareto-frontier of the metrics (re-generation cost, success rate).

Critical strengths

The paper address an important problem which is to save computational cost of action sequences without regeneration. It proposes and proves a theorem characterizing the situation where correction model trained on short sequences can generalize to long ones. On comparing to several valid baselines, the paper's approach shows the best tradeoff between cost and success rate.

Critical weaknesses
  1. The theorem the paper proves assumes a known response class and fixed finite graph, however the neural experiments do not verify these assumptions or even learn the required structure -- showing this would be important.
  2. The paper does not compare with previous proposed approaches like RIFT and Feedback World Model.
  3. How do the authors propose learning/coming up with the hard constrains they impose on the relationships (which demonstrates the best results in their paper) in general scenarios?
Questions for authors

Please answer my questions in the weakness section.

3 comments
AuthorsOpenReview ↗

Response to Reviewer 8VNC (1/3): summary and W1

Thank you for the careful review and the accurate summary. We are glad that the problem, the theorem and the cost–quality trade-off came across. Each of your three questions led us to new experiments. In brief:

  • W1. In a controlled visual study, the required structure is learned from short data and extrapolates. On Meta-World Assembly, a correction trained on two to four blocks transfers to twelve. New measurements on the data of Table 1 show that the response to feedback is largely predictable one step ahead, approximately keeps its norm and is branch-specific. With observations decoded from noisy pixels, the return amplification that the theory predicts reappears (new).
  • W2. We compared with an FWM-style correction. On the same cached futures, it has regret .144, against .089 for OEWM, also when OEWM is given only the same feedback. RIFT is complementary.
  • W3. OEWM needs no hard relations. Where relations are wanted, a new experiment in the scalar study of Table 3 shows that they can be proposed from data, without a dictionary or the single-action orders, and validated on return probes before use.

W1. Verifying the assumptions and learning the required structure

You are right that this link deserves direct evidence. The assumptions state when an all-path guarantee exists, and OEWM itself does not rely on them. For a pretrained model such as DINO-WM the true response class is not available, so we test the link to neural models at four levels.

  1. The structure is learned in a controlled visual study (App. H.6). A convolutional encoder, world model and observer are learned from pixels. One transport matrix per action is then learned from one- and two-block data, in a structured class that contains the true generator.
    • Across twelve seeds, the transport succeeds in every cell at four, eight and twelve blocks. Attention, pooling, GRU and endpoint readers trained on the same data fall to about 11% at twelve blocks (Table 12).
    • The learned matrices are within 0.70–1.96% of the true transports (three seeds measured).
    • New: when every training target is also decoded from pixels by the observer, the transport still succeeds in every cell at four, eight and twelve blocks, in all twelve seeds, while GRU and endpoint readers trained on decoded targets reach 12.7% and 10.0% at twelve blocks. Its operator relative error is 0.77–2.30% and its singular values 0.96–1.03, although nothing imposes norm preservation.
  2. Short training transfers to long plans on Assembly (Section 6, Table 29). Readers trained on two to four blocks are tested on twelve blocks in 32 new scenes.
    • OEWM reaches 77.69% success, against 52.08% for capacity-matched pooling and 52.34% without feedback. Conditioned regeneration reaches 79.17%.
    • A recurrent reader is in between (62.59%).
  3. In the data of Table 1, the response is largely predictable one step ahead, approximately keeps its norm and is branch-specific (new). The theorem idealizes a response that each action transports without changing its norm, and that depends on the branch. We compared the cached DINO-WM futures of Table 1 with their regenerations (maps fitted on training roots; 576 test cells).
    • Transport. A linear map of the response at one step explains 86–92% of the response at the next step. The best prediction shared by all candidates explains 14–66%. On the 23 tasks of App. H.10 (new), the map explains 66–89% and beats the best shared prediction in 22.
    • Norm. The median ratio of the response norm between consecutive steps is 0.82–1.03, and at step six the norm is 0.85 of step one. In all 23 tasks the median ratio is 0.81–1.08.
    • Branch dependence. Over the six steps, even an oracle correction shared by all 125 candidates, namely the exact mean change in each cell, explains only 48% of the squared change. The other 52% differs across branches.
    • OEWM corrects each branch from its own features and actions. In the short-bank study of Table 21, removing branch specificity while keeping the same feedback loses most of the success gain (broadcast and shuffle).
  4. The theory's return prediction, with pixel-decoded observations (new). We ran the matched-return construction of App. H.3 in three dimensions, with every fact and calibration observation decoded from noisy pixel images by a learned observer trained with the same noise, and the unknown transport fitted without an orthogonality constraint (nine seeds, 576 systems).
    • The same fitted transport has regret that grows with plan length on the library with a known return and stays flat without it. At plan lengths 64 and 2048 the gap is positive in every cell of both studies (three and six seeds).
    • In the six-seed study, short-path validation error does not predict which fits fail on returns of length 64 (rank correlation −0.07 and −0.15), whereas validation on a held-out return word does (+0.49 and +0.50).
AuthorsOpenReview ↗

Response to Reviewer 8VNC (2/3): W2, comparison with FWM and RIFT

W2. Comparison with RIFT and Feedback World Model (FWM)

An FWM-style correction is significantly worse than OEWM on the same cached futures, and RIFT is complementary. We agree that a comparison with FWM is informative, so we ran an FWM-style one. For context, Section 2 of the submission cites both works and states how our setting differs, and under the ICLR 2027 guidelines both are concurrent (FWM: arXiv in May 2026 and NeurIPS 2026; RIFT: arXiv in August 2026).

  • FWM (An et al., NeurIPS 2026) corrects a world model's predictions at inference time, without retraining. It multiplies the observation residual by a fixed gain, chosen by held-out prediction error, and adds the result to the predictions. The correction is shared by all candidates.
    • We implemented this on the cached futures of Table 1, with one gain per step fitted on the training roots. Held-out error selected this form over a single global gain.
    • We then retrained the decision reader on the corrected futures with OEWM's architecture, size and protocol, so feedback reaches the reader only through the shared correction.
    • On the 576 test cells and six seeds of Table 1:
Reader of cached futuresRegret [95% CI]Success
OEWM.089 [.077, .101]93.46%
OEWM given only the FWM-style feedback (new).089 [.076, .102]93.61%
FWM-style correction.144 [.111, .191]88.43%
No feedback.339 [.320, .357]72.05%
  • The paired difference is +.055 [+.023, +.102] in regret and −5.0 [−10.3, −1.2] points in success.
  • The FWM-style correction uses only the visual residual of the latest frame, while OEWM also reads earlier frames and proprioception. Given only that residual, OEWM has regret .089 [.076, .102], .055 [.026, .098] below the FWM-style correction (new). The gap is therefore not explained by the additional feedback.
  • The fixed-gain form explains only 15% of the change that regeneration makes, against 48% for the oracle shared correction of W1.
  • Applying FWM's observer during regeneration instead pays the full regeneration cost. Our conditioned-reroll baseline already gives the same residual to a reader of regenerated futures. In Table 1, OEWM is statistically indistinguishable from it in regret (−.0025 [−.0166, +.0111]), at a p99 latency 99 times lower than FP32 regeneration and 37 times lower than BF16.
  • RIFT (Zhang et al., 2026) makes the future representation cheaper to obtain. Learned anticipation tokens build a world action model's future key/value cache in one backbone pass, without a video rollout, and it reports a 3.1–9.2× speedup over rollout-based models. It builds a fresh cache for each action chunk and does not revise an existing one. Rebuilding the candidates' futures needs at least one pass of the world model over the candidates. In our setup one rollout step takes about 82 ms in FP32 and 30 ms in BF16 (a sixth of the median regeneration time, App. H.9), against a median of 4.4 ms for OEWM, which needs no rollout step. OEWM addresses the complementary step, revising existing futures when feedback arrives, so the two could be combined: a RIFT-style generator could produce the cache that OEWM revises.
AuthorsOpenReview ↗

Response to Reviewer 8VNC (3/3): W3, hard relations

W3. Obtaining hard relations in general scenarios

As you note, hard relations give the best result of Section 5 (Fig. 3). OEWM itself needs none, and the visual results use none. Where relations are wanted, they can be proposed from data and validated on return probes before use. In a new experiment, with neither a dictionary nor the single-action orders, a proposal from data recovers all three relations in 64 of 64 systems and in 128 random systems with other finite orders, and the validation keeps every correct proposal and rejects the 0.01-rad misspecification of Table 3.

  • The visual results use no hard relations. In Section 6, OEWM learns branch-specific corrections from simulator-fork outcomes alone. In Table 1 it is statistically indistinguishable from conditioned regeneration in decision regret, at about 1% of the FP32 p99 latency (3% of BF16). Hard relations appear in Section 5, where they test the theory constructively and show what exact structure buys: a regret of 4.3×10⁻⁷, against .065 without the relation (Fig. 3).
  • Sources of relations.
    1. Known structure of the action library, such as the finite orders of rotation primitives (A² = I and B³ = I in the study of Table 3).
    2. Proposals from data, obtained by fitting a generic family such as finite orders or inverse pairs.
    3. In either case, validation on return probes before use. Returns are the queries that expose relation errors (Lemma 4.1, Proposition 4.3).
  • New experiment. We used the scalar study of Table 3: 64 systems, 64 repeats per probe and H = 2048.
    • After fitting, we measure four return words (ab)^k, k = 4, 8, 16, 32, with a quarter of the repeats.
    • A supplied relation is kept only if a χ² lack-of-fit test at nominal level .01 accepts it; otherwise the order-only fit is used.
Supplied relationKeptRegret, imposedRegret, validated
Correct64/644.3×10⁻⁷4.3×10⁻⁷
Off by 0.01 rad (Table 3)0/64.268.065
  • Without a dictionary or the single-action orders, we fit two unconstrained rotations to all words of length up to three (about as many scalar observations as the bank of Table 3), snap the rotation angles of a, b and ab to the nearest finite orders (at most 12), and refit with these relations. This recovers all three relations in 64 of 64 systems; after validation the regret at H = 2048 is 4.2×10⁻⁷, against .138 for the unconstrained fit.
  • For 128 random systems with single-action orders up to 8 and joint orders up to 12, the same procedure recovers all three relations in every system, and with the 0.01-rad shift no wrong proposal passes the test.
  • Where relations are needed. The theory also says where constraints matter: only for task-relevant return types inside recurrent components.
    • A plan crosses edges between components at most m − 1 times.
    • Coordinate changes at vertices cancel in composition (Section 4.1).
    • Task-invisible directions can be discarded (Theorem 4.2).
    • On 9,716 enumerated graphs up to H = 64 (App. H.2), perturbing crossing edges or vertex coordinates keeps damage below 4×10⁻⁴, while perturbing recurrent edges raises it to 7.8×10⁻².

We will add the FWM comparison, the response measurements, the two pixel-decoded studies and the validation experiment to the appendix (a new section, App. H.11), and expand the discussion of FWM and RIFT in Section 2.

Your three questions made the paper stronger. We are glad to add detail on any point. Our general response restates the paper's question and main results in plain language. Thank you again for the careful review.

Comments on the paper

2 comments
AuthorsOpenReview ↗

Summary of responses and new results

We thank the reviewers. All three asked how the theory connects to the visual experiments; this note answers that and lists the new results.

The question. A planner with a world model has imagined many candidate futures, here 125 DINO-WM rollouts, when a new observation arrives, for example of actuator backlash. Regenerating every future costs about 0.5 s per event (0.18 s in BF16); a learned correction revises the cached futures in about 5 ms. The paper asks when a correction fitted on short action paths stays reliable on longer plans.

Main results.

  1. Same data, different reliability (Proposition 4.3). In a matched construction, adding one known return action, which lets a plan repeat an uncertain loop, with the calibration data unchanged, changes the best achievable worst-case decision risk from independent of plan length to growing with it until it saturates.
  2. When reliable reuse is possible (Theorem 4.2). Take a known compact class of norm-preserving responses on a fixed finite action graph. Noisy short-path calibration gives vanishing regret on all legal plans exactly when each recurrent part of the graph has finitely many task-relevant return types at every precision. Its proof charges approximation error once per strongly connected part of the graph, not once per action.
  3. Deployment matters (Theorem 4.4). Learning can succeed for every fixed distribution of plans even when no all-plan guarantee is possible.
  4. Practice (Section 6, Table 1). On Meta-World Assembly, cached correction is statistically indistinguishable from feedback-conditioned regeneration in decision regret (−.0025 [−.0166, +.0111]). Its p99 feedback latency is 99× lower than FP32 regeneration and 37× lower than BF16.

How theory and experiments connect.

  • The theory's prediction, in a controlled study. With a return, the regret of one fitted estimate grows about linearly with the horizon; without it, regret stays nearly flat (median log–log slopes 1.051 and 0.083; App. H.3). The separation persists when every observation is decoded from noisy pixels by a learned observer (nine seeds, new).
  • What to validate and calibrate. In that study, a held-out return word predicts which fits fail on returns of length 64 (rank correlation +0.49 and +0.50; six seeds, new), and short-path validation error does not (−0.07 and −0.15). Return probes keep the correct relation and reject the 0.01-rad misspecification of Table 3 in 64/64 systems, and at a matched budget they lower long-horizon regret 9 to 13 times in three tested conditions (App. H.4).
  • The required structure can be learned (App. H.6). In a controlled composition task, a per-action transport trained on up to two blocks succeeds at twelve, where attention, pooling and recurrent readers fall to about 11%. With pixel-decoded targets it still does in all twelve seeds, while GRU and endpoint readers trained this way reach 10–13% (new).
  • Short training transfers to long plans on Assembly (Section 6, Table 29). Trained on two to four blocks and tested on twelve, OEWM reaches 77.69% success, against 52.08% for capacity-matched pooling and 52.34% without feedback.
  • Violations of orthogonality and of the class (new). In the study of Table 3, with non-orthogonal truth, the correct relation keeps its advantage in direction regret at every tested size. In the composition task, the learned transport still succeeds at twelve blocks in every seed with non-orthogonal truth and with small violations outside its class; at 0.03 rad, 92.7% on average (GRU: 10.5%).
  • Measured in the data of Table 1 (new). The response is branch-specific: a correction shared by all 125 candidates explains at most 48% of the squared change (reply to 8VNC).

Other new results.

  • FWM-style correction (reply to 8VNC). A fixed-gain shared correction, with a retrained reader, has regret .144 against .089 for OEWM; given only the same feedback, OEWM has .089 (new).
  • Ablations (reply to rDMM). Replacing OEWM's ordered path summary by pooling raises regret by .044 [.026, .064] with the gate and by .025 [.008, .048] without it.
  • More tasks (reply to rDMM). On the 23 tasks of Table 30, with all readers retrained six times, cached correction is not separated from conditioned regeneration (+0.26 points [−0.24, +0.72]) and is 3.0 points [2.2, 4.0] above a reader without feedback.
  • End-to-end cost (reply to rDMM). Of the 8.5 s of re-timed cache preparation per root, 6.7 s (candidate generation and simulator setup) is paid by every method, and 1.8 s (the nominal rollout) by any planner that uses the world model before feedback. Cached correction is then cheaper from the first event, and otherwise from the fourth at matched precision.

Revision. We will add the new results to the appendix, open the introduction with the planning example and the main results, and extend Section 2 with world-model planning, compounding error, FWM and RIFT.

AuthorsOpenReview ↗

Revised PDF: summary of changes

We have uploaded a revised PDF. New and reworded text is in blue.

Unchanged. The title, the statements of Lemma 4.1, Theorems 4.2 and 4.4 and Proposition 4.3, all numbered equations, Table 1 and the figures are as submitted. Every section, theorem, equation, table and figure keeps its number, so the references in our responses apply to the revised PDF as well. The main text is ten pages.

Abstract. The technical claims of the abstract are unchanged. Its opening now says that a correction is calibrated on short action paths and what a return is; this replaces the sentence on legal continuations. Its closing sentences state the main result of Table 1.

New results. Appendix H.11 (Tables 31–39) contains the experiments described in our responses. Each reuses the protocol of a study in the submission. Section 6 states the breadth and cost results.

Introduction and related work. The introduction now opens with a planning example in place of its first paragraph, defines three recurring terms and states the four main results. Section 2 adds world-model planning and compounding error, with three added references, and extends the discussion of FWM and RIFT.

Exposition. Section 3 defines each term where it first appears. Section 4.1 says why the theory uses a finite graph and orthogonal maps. Plain-language text follows Lemma 4.1, Theorem 4.2, Proposition 4.3 and Theorem 4.4; the paragraph after Proposition 4.3 also reads it through the criterion of Theorem 4.2. Sections 5 and 6 write out results that were packed into lists, with the same numbers, and Section 6 says what OEWM's gate is. The discussion adds two practical rules. The captions of Figures 1 and 2 are reworded, and the appendix opens with a short guide.

Shortened. To stay within ten pages, six technical passages in Sections 4.2–4.4 (parts of the proof sketches and two examples) now state the result and point to the appendix. Their full versions were already in Appendices B, C and E and are unchanged. Three sentences of the introduction that restated the second and third main results are merged into the list of main results.

We are glad to clarify any of these changes.

From OpenReview, CC BY 4.0. Authors as signed there; shown unedited.

Related papers

Loading the map…

Discussion (0)

Sign in to comment.