- Rating
- 3
- Confidence
- 1
- Informativeness
- 2
- Presentation
- 2
- Soundness
- 2
Read review
Paper summary
This paper studies the reliability of reusing feedback-based corrections in cached world-model predictions over long action paths, showing that accurate short-path calibration does not necessarily ensure reliable long-horizon decisions. It theoretically characterizes when worst-case decision regret vanishes under noisy calibration, based on task-significant response types, and distinguishes approximate learning from exact finite calibration. The paper also proposes an observable-endomorphic world model (OEWM) for cached feedback correction and evaluates it on Meta-World, demonstrating improved planning performance over several baselines and substantially lower feedback-time latency than conditioned rerolling.
Critical strengths
-
The paper identifies an important limitation of short-path calibration for long-horizon feedback reuse and provides a principled characterization of when worst-case decision regret vanishes. The distinction between approximate all-path learning and exact finite calibration, together with the analysis of path composition and deployment distributions, offers useful theoretical insights.
-
The paper combines controlled studies with visual planning experiments on Meta-World. Results show that cached feedback correction can improve decision quality over pooling-based and recurrent baselines while substantially reducing feedback-time latency compared with conditioned rerolling. The use of matched comparisons and explicit computational accounting strengthens the practical evaluation.
Critical weaknesses
-
The theoretical guarantees rely on a known compact class of orthogonal response dynamics on a fixed finite graph. Their applicability to neural world models with approximate, potentially misspecified dynamics remains unclear, and the visual experiments do not establish that OEWM satisfies the assumptions required for the all-path guarantees.
-
The visual evaluation focuses primarily on one Meta-World task setting, leaving generalization across tasks, environments, and world-model architectures insufficiently explored. Moreover, the advantage over conditioned rerolling in decision regret is statistically unresolved, the necessity of individual architectural components is not fully established, and the overall computational advantage depends on cache preparation and reuse frequency.
Questions for authors
-
How sensitive are the theoretical conclusions and practical correction performance to violations of the assumed orthogonality and response-class structure? Can the authors provide additional analysis or experiments on model-class misspecification, and clarify how the theoretical characterization can guide the design of practical neural world models?
-
Could the authors evaluate OEWM on additional tasks or environments and conduct targeted ablations to isolate the contributions of its response-transport mechanism and innovation-dependent gate? In addition, could they report end-to-end computational costs, including cache preparation, and characterize the reuse frequency at which cached correction becomes more efficient than conditioned rerolling?
3 comments
Response to Reviewer rDMM (2/3): design guidance, ablations and breadth
Q1b. How the theory guides the design of neural world models
- Validate on returns, not only on short paths. Fits with no greater training loss differ by five orders of magnitude in long-horizon regret on alternating queries (Fig. 3). Return probes reject the 0.01-rad wrong relation of Table 3 (64/64) and flag a large non-orthogonal violation (54/64 at ; Part 1). With pixel-decoded observations (six seeds), a held-out return word predicts which fits fail on returns of plan length 64, and short-path error does not (Part 3).
- Spend calibration on returns. At a matched action budget, return probes reduce the long-horizon regret of fits that learn the relation from .0133 to .0010, and the gain persists when the true relations deviate (App. H.4).
- Treat the action library as part of the problem, and calibrate rather than restrict it. One known return can change the achievable risk without any new data (Proposition 4.3). In the wrist study, removing reversals cuts selection regret inside the library from .471 to .008 but gives up twelve to forty times as much capability, and calibration with eight words beats restriction at every tested length and noise level (App. H.5).
- Compose the correction along the path. In a controlled composition task, a transport learned from one- and two-block data extrapolates to twelve blocks, where pooling, attention and recurrent readers fall to about 11% (App. H.6).
- Impose relations only when known, or proposed from data and validated (Table 3; reply to 8VNC, W3).
Q2a (W2). Ablations of the response transport and the innovation gate
A new factorial isolates both components in the protocol of Table 1. Two new readers complete the 2×2 design: one keeps the transport and removes the gate, and one removes both.
We trained them with OEWM's data, seeds and training protocol, at matched active parameters (about 0.97M each). Retraining OEWM's first seed with the same code reproduces the stored checkpoint exactly. On the 576 test cells and six seeds of Table 1:
| Reader | Path summary | Gate | Regret [95% CI] | Success |
|---|---|---|---|---|
| OEWM | transport | yes | .089 [.077, .101] | 93.46% |
| New: no gate | transport | no | .105 [.092, .118] | 92.82% |
| Pooling (Table 1) | mean pooling | yes | .133 [.115, .154] | 90.10% |
| New: pooling, no gate | mean pooling | no | .130 [.111, .155] | 90.54% |
| No feedback | – | – | .339 [.320, .357] | 72.05% |
- Replacing the transport by pooling raises regret by .044 [.026, .064] with the gate and by .025 [.008, .048] without it.
- Removing only the gate raises regret by .016 [.002, .030]. Without the transport we detect no gate effect (−.002 [−.014, +.012]).
- The GRU reader of Table 1 is a second recurrent reader without the gate: it feeds the innovation into the recurrence and has 10% more parameters. Its regret is .020 [.007, .037] above OEWM (Table 1).
- Earlier ablations show that branch-specific path information is needed. Assigning the innovation to the wrong branch removes nearly all of the gain (Table 21), and an endpoint-only reader loses 8.07 [5.60, 10.68] success points (Table 13).
Q2b (W2). More tasks, environments and world models
A new comparison on 23 tasks shows that cached correction stays close to regeneration beyond Assembly. The submission also evaluates two pretrained world models and a third world model learned from pixels.
- Tasks: new comparison with regeneration (protocol of App. H.10, Table 30). All 50 Meta-World v3 tasks were screened, and the 23 with at least two points of oracle headroom are evaluated on held-out banks (448 cells per task). Table 30 compares cached correction with the nominal plan only. We added two readers under the same protocol, conditioned regeneration and a reader without feedback, and trained all three six times (three seeds on each of two machines). Success averaged over the 23 tasks and the six runs:
| Nominal plan | No feedback | Cached correction | Conditioned regeneration | Oracle |
|---|---|---|---|---|
| 83.2% | 84.0% | 87.1% | 86.8% | 96.8% |
- Cached minus regeneration is +0.26 points [−0.24, +0.72]; twelve further trainings give −0.06 [−0.38, +0.27], inside the −2-point margin of App. H.7.
- Cached correction is 3.04 points [2.19, 3.99] above the reader without feedback.
- Its gain over the nominal plan, 3.83 points [3.08, 4.59], is consistent with Table 30 (4.14).
- Pretrained world models (App. H.8, Table 21). On JEPA-WM, cached correction improves success by 13.86 [13.12, 14.63] points over the nominal plan and by 1.70 [1.20, 2.22] over same-seed regeneration from the factual frame. On DINO-WM the latter contrast is +.13 [−.40, .66].
- A world model learned from pixels (App. H.6). In a controlled composition task whose transport class contains the true generator, the learned transport extrapolates from two blocks to twelve.
Response to Reviewer rDMM (3/3): violations in a visual pipeline, comparison with regeneration and end-to-end cost
Q1 (W1), continued: violations in a learned visual pipeline, and the response on 23 tasks (new)
The first three runs use the visual pipeline of App. H.6: an encoder, world model and observer learned from pixels, and one free transport matrix per action learned from one- and two-block data.
- Non-orthogonal truth inside this class (, ). The transport still succeeds at four, eight and twelve blocks in all twelve seeds per , with relative error 0.6–3.7% to the non-orthogonal truth.
- Truth outside this class. Each action's rotation angle also depends on the response direction, by up to , so no fixed matrix per action is exact. Success at twelve blocks stays at 100% for and is 92.7% at , against 10.5% for the GRU reader (eight seeds; the interval is positive in each).
- Pixel-decoded targets. With every training target also decoded from pixels, the transport succeeds in every cell in all twelve seeds (operator relative error 0.77–2.30%), while GRU and endpoint readers trained on decoded targets reach 12.7% and 10.0% at twelve blocks.
- The theory's return effect. We ran the matched-return construction of App. H.3 in three dimensions, with every fact and calibration observation decoded from noisy pixels by a learned observer, and the transport fitted without an orthogonality constraint (nine seeds, 576 systems). For the same fit, regret grows with plan length with a known return and stays flat without it; at plan lengths 64 and 2048 the gap is positive in every cell.
- Validate on returns (rule 1). In six of these seeds, short-path validation error does not predict which fits fail on returns of plan length 64 (rank correlation −0.07 and −0.15), whereas a held-out return word does (+0.49 and +0.50; differences +0.57 [+0.48, +0.66] and +0.64 [+0.56, +0.72]).
- The measured response on 23 tasks. On the stored banks of the 23 tasks of App. H.10, the median norm ratio between steps is 0.81–1.08 in every task, and a one-step linear map explains 66–89% of the next-step response, more than the best shared prediction in 22 of 23.
Q2c (W2). The comparison with conditioned regeneration
We agree that the regret difference is unresolved, and the paper claims none: the advantage over conditioned regeneration is a much lower feedback cost. Regeneration recomputes every future with the new observation, so it is the reference; in the matched comparison of Table 1, cached correction is statistically indistinguishable from it.
- In Table 1, the regret difference is −.0025 [−.0166, +.0111]. The interval excludes a disadvantage larger than .0111, about one eighth of OEWM's regret. The p99 feedback latency is about 99 times lower, or 37 times lower against BF16 regeneration.
- In the independent sixteen-chunk study, OEWM meets a predeclared −2-point success noninferiority margin against regeneration from the factual frame, at about 300-fold lower FP32 p99 latency (App. H.7, Fig. 13).
Q2d (W2). End-to-end cost, including cache preparation, and the break-even reuse frequency
Cache preparation is shared with regeneration. When the plan is chosen with the world model before feedback, cached correction is therefore cheaper from the first feedback event.
We re-timed the cache preparation of Section 6 on the same RTX 5090, in the same process as the feedback-time arms. Their median latencies reproduce the submitted ones within 1% (App. H.9). The re-run takes 8.53 s per root (8.05 s on average in Section 6). Its median components:
- Candidate generation: 6.23 s. This is the CPU search that constructs our benchmark's 125 candidate plans.
- Simulator setup: 0.50 s.
- Every method pays both of these, since regeneration scores the same 125 candidates.
- World-model rollout of the nominal bank: 1.78 s (FP32) or 0.68 s (BF16). A planner that chooses its plan with the world model computes it before feedback arrives, however the feedback is then handled.
OEWM's own preparation adds 0.09 ms and 128 KB per reader (App. H.9). After it, a feedback event costs 4.43 ms, against 487 ms (FP32) or 183 ms (BF16) for regeneration.
Break-even.
- In our setting both pipelines compute the nominal bank, so cached correction is cheaper from the first event, by 483 ms (FP32) or 178 ms (BF16) per event.
- On cost alone, a pipeline that would not otherwise generate futures before feedback recovers the nominal rollout's cost after four feedback events per cache (3.7 with both steps in FP32, 3.8 with both in BF16).
We will add these results to the appendix and state the breadth and cost results in Section 6.
Your questions made the paper stronger. The new results cover the assumptions, task coverage, the components and the cost, and we are glad to address any point that remains. Thank you again for the careful reading.
Response to Reviewer rDMM (1/3): summary and misspecification
Thank you for the careful reading and the concrete questions. We are glad that the distinction between approximate learning and exact calibration came across. Your questions led us to four new experiments, and every point has a direct answer. In brief:
Q1 (W1). Sensitivity to violations of orthogonality and of the assumed class
You are right that the guarantees assume an idealized class. OEWM itself assumes neither orthogonality nor a known class. For the theory: