- Rating
- 2
- Confidence
- 3
- Informativeness
- 1
- Presentation
- 2
- Soundness
- 2
Read review
Paper summary
This paper proposes Successor-measure World Model (SWM) for zero-shot visual control. The central idea is to combine a frozen DINOv2 visual encoder with the Forward-Backward (FB) successor-measure framework: three consecutive DINO CLS features are concatenated and passed through a lightweight LayerNorm + linear projector, after which the projected representation is used to learn policy-conditioned successor measures and a latent-conditioned policy family. At test time, reward-labeled observations are projected through the learned backward representation to obtain a task vector, which directly selects a pretrained policy without downstream policy optimization, planning, or additional environment interaction.
Critical strengths
- The empirical result is consistent and the core design motivation is reasonable The most convincing result is Table 1. SWM outperforms CNN-FB on all 16 downstream tasks, including both relatively static tasks such as Walker Stand and dynamically demanding tasks such as Cheetah Run. The domain-average gains over CNN-FB are reported as 20.2% on Walker, 17.2% on Quadruped, 47.7% on Cheetah, and 30.0% on Jaco. This provides reasonably strong evidence that frozen DINO features can be successfully integrated with FB-style zero-shot control.
- The paper goes beyond reporting a performance gain The robustness experiment and representation probing are useful attempts to understand why the method works rather than simply reporting higher returns. In particular, Figure 3 shows that SWM-CLS does not uniformly improve generic pose or motion decodability over SWM-raw, while the control performance gap between the two representations is enormous. This is a meaningful observation: representation quality for generic state prediction and representation quality for successor-measure control need not be equivalent.
Critical weaknesses
-
Limited methodological novelty. SWM retains the established FB factorization, training objective, latent-conditioned policy, and reward-projection inference. Its concrete addition consists of frozen DINO features, three-frame concatenation, and a LayerNorm–linear projector. The distinction between future occupancy and predictive dynamics therefore largely reflects existing successor-measure learning rather than a new modeling principle. Substantial space is devoted to preliminaries and reformulations of this inherited machinery, while the methodological advance remains limited. The results demonstrate a useful visual implementation, but do not sufficiently establish the conceptual contribution suggested by the world-model framing.
-
Confounded baseline comparisons. The experiments do not adequately separate the benefit of occupancy modeling from that of pretrained visual representations. SWM uses frozen DINO features, whereas Dreamer and TD-MPC2 are adapted to reward-free pretraining and downstream planning with frozen dynamics. Matching interaction budgets and reward labels supports a comparison of these complete systems, but does not isolate the proposed modeling advantage. A predictive baseline using the same pretrained visual features and comparable adaptation is needed to substantiate the motivation for replacing predictive world modeling with successor-measure modeling.
-
Incomplete mechanism analysis. The projector ablation demonstrates its importance, but removing it simultaneously changes feature normalization, dimensionality, conditioning, and the availability of a learned shared transformation. It therefore does not establish that the improvement specifically arises from better occupancy factorization. The state probes show that generic decodability does not explain control performance, but do not identify what does. Normalization-only controls, fixed projections at matched dimensionality, and alternative projector objectives would help distinguish the claimed representation benefit from ordinary feature-conditioning and optimization effects.
-
Limited zero-shot validation. The sixteen tasks demonstrate useful performance within four simulator domains, but the experiments do not establish how reliably the reward-to-policy interface operates under different inference conditions. In particular, sensitivity to the reward-label budget, reward sparsity, and coverage of reward-bearing states is not evaluated. Predicted future occupancy and reward-projection accuracy are also not directly assessed against observed trajectories. Consequently, the evaluation supports improved returns on the selected tasks, while providing limited evidence about the reliability and limitations of the proposed reusable world-model representation.
Questions for authors
See the weaknesses.
3 comments
Follow Up to your Review
Dear Reviewer aiEu,
Thank you again for the time and effort you put into reviewing our work!
We understand that this might be a super busy period for you. We wanted to briefly let you know that we have posted point-by-point responses to your questions and concerns.
As the rebuttal period is ending very shortly, we would be grateful for your feedback on whether the responses address your concerns. We would be happy to clarify anything that remains open.
Thank you!
Best,
The Authors of Submission 47327
A Quick Summary of New Results for W1–W4 & Follow-Up
Dear Reviewer aiEu,
As the discussion period is closing soon, here is a short summary of how our new results (C1–C4) address your concerns:
- W1 – Reframing & broader empirical scope: We will condense the FB preliminaries and foreground our methodological extension alongside new empirical findings across encoders (R3M, CLIP, random ViT in C1) and standard resolution (CNN84 in C3, where SWM still wins with 2.34x fewer FLOPs).
- W2 – The gain comes from successor-measure modeling, not DINO alone: With the same frozen DINOv2 features and adaptor architecture, SWM outperforms TD-MPC2 in all four domains (DINO even hurts TD-MPC2 on locomotion) and beats DINO-WM on Jaco (64.19 vs. 47.60) (C4).
- W3 – FB learning adds gains beyond feature conditioning: At matched dimension with LN, the FB-trained projector improves over a frozen random projection by +69% / +16% / +15% / +220% on Cheetah / Walker / Quadruped / Jaco, while variance-preserving PCA is no better than random (C2).
- W4 – Task specification is label-efficient: With 1/20 of the default reward labels (256 labels), SWM retains 89–97% of its return on locomotion and 72% on Jaco.
We believe these results address the main concerns in your review, and we would be grateful if you would consider updating your assessment.
Best regards, The Authors of Submission 47327
We thank the reviewer for recognizing the consistent control improvements and the value of the robustness and representation analyses. The additional experiments strengthen the design argument, clarify the role of the visual prior and its adaptation, and examine the label budget needed for downstream task specification. Shared results are provided in C1–C4 of our common response.
1. Contribution and world-model framing — W1
SWM contributes a methodological extension of reward-free FB that replaces visual-encoder training from environment data with fixed pretrained visual knowledge. Our controls identify the sources and scope of its gains: the random-ViT control supports the value of visual pretraining, while competitive R3M results show that SWM is not tied to DINO (C1); fixed projection and projector training make distinct contributions (C2); and the same frozen features and adaptor improve FB in every domain but degrade TD-MPC2 on all locomotion domains, suggesting that the benefit is specific to successor-measure learning (C4). This design achieves higher mean returns than CNN84 in all four domains using approximately 43% of its target-domain FLOPs (C3). We will foreground this method extension and its new empirical findings, condense the FB preliminaries, and clearly attribute the inherited factorization and inference machinery.
2. Predictive comparisons with shared visual features — W2
C4 compares TD-MPC2 and SWM with the same frozen DINO encoder and projector architecture, matching the visual prior while contrasting predictive planning with successor-measure control.
With the same frozen DINO features and adaptor, SWM beats CNN-FB in all four domains (C3), whereas TD-MPC2 beats its CNN version only on Jaco and falls below it on all three locomotion domains (C4). DINO-based TD-MPC2 also stays below SWM in all four domains. This contrasting pattern shows that adding the same visual front end does not consistently improve control, suggesting that how these features are adapted and used for control matters. Together with the adaptor ablations in C2, these results support SWM’s design.
The DINO WM CLS baseline (goal image needed, so only in Jaco) provides a complementary shared-feature comparison, where SWM also outperforms predictive planning on Jaco (64.19 vs 47.6).
Together, the comparisons with DINO WM CLS planning and DINO-based TD-MPC2 show that SWM achieves higher mean returns than predictive alternatives using the same pretrained visual features, supporting its successor-measure world modeling design.
3. What the adaptor controls establish — W3
C2 distinguishes the effects of fixed feature transformation and projector training. LN alone helps Walker and Quadruped , not Cheetah or Jaco. A frozen random projection to the same dimension, followed by LN, reaches 59% / 87% / 87% / 31% of full SWM on Cheetah / Walker / Quadruped / Jaco with no learning in the projector. These gains are consistent with the feature-conditioning explanation raised by the reviewer. Training the projection with the FB loss, at the same dimension and with LN, adds a further 69% / 16% / 15% / 220%. We will revise 5.3 to distinguish the gains from fixed feature transformations from the additional gains obtained by training the projector. Fixed transformations recover a large fraction of locomotion performance, while training yields the largest relative improvements on Cheetah and Jaco.
PCA, a fixed projection that preserves visual-feature variance, performs no better than the random projection. Full-SWM improves over both PCA compression and random fixed projections. Together, these results validate the complete adaptor and establish FB-driven projector learning as a source of control gains beyond fixed feature transformations.
4. Reward-label budget and zero-shot task specification — W4
With all networks frozen and the inference rule unchanged, SWM retains 89–97% of the default mean return on locomotion and 72% on Jaco using 256 reward labels—one twentieth of the 5,120-label default.
The table reports mean normalized returns over four tasks per domain, with N = 5,120 as the 100% reference [95% CI].
These results support substantially reduced-label task specification for the evaluated models, with greater low-budget degradation and uncertainty on sparse-reward Jaco. Increasing the budget beyond the default yields no consistent mean-return improvement across domains.