- Rating
- 3
- Confidence
- 2
- Informativeness
- 2
- Presentation
- 3
- Soundness
- 2
Read review
Paper summary
The paper proposed a simple RSI principle for world models. The authors show that self-training worsens a trend toward reduced diversity in the generated samples. This effect comes from mode-seeking sampling in unconditional generators and from model–policy coupling in conditional dynamics models. To reduce the resulting parameter shift, they apply negative extrapolation, which pushes the world model to produce synthetic experience that are diverse and more useful for training downstream policies.
Critical strengths
-
Exporing RSI for world model sounds like an interesting idea, the author provide algorithms for both unconditional generative replay and for conditional latent dynamics.
-
The writing is easy to follow and the ideas are explained very clearly.
Critical weaknesses
-
In S^2ER, the author used tFID as a selection metric. However, it is widely known that FID can be easily hacked [1] [2]. First and second order moment can not guarantee robust representation alignment for the two distribution compared. Besides, compute FID is costly. It is a bit confued to me why S2ER increases total GPU-hours by only 1.7% over SYNTHER. I suggest the author to provide some reasoning for why they used tFID as selection metric, and why this aligns with the downstream control performance.
-
The authors argued that the method is data efficient, but did not compare with some other data efficient world models like [3] and [4]. Any reason for this?
[1] Jayasumana, Sadeep, et al. "Rethinking fid: Towards a better evaluation metric for image generation." 2024 IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR). IEEE, 2024.
[2] Yang, Jiawei, et al. "Representation Fr'echet Loss for Visual Generation." arXiv preprint arXiv:2604.28190 (2026).
[3] Sonigra, Romil V., and P. R. Kumar. "Dreaming Smoothly and Sample Efficiently with Gradient Penalized Latent Dynamics." arXiv preprint arXiv:2605.23089 (2026).
[4] Aghabozorgi, Mehran, et al. "WIMLE: Uncertainty-aware world models with IMLE for sample-efficient continuous control." arXiv preprint arXiv:2602.14351 (2026).
Questions for authors
As authors mentioned, 'Recursive self-training further amplifies the concentration'. Will use methods like TRPO together with the proposed algorithms help with this issue? I wonder if this can be a replacement for degradation threshold. The authors can either provide some theoretical or experical analysis.
Reviewer-Author Discussion Phase is Ending
Dear Reviewers,
Thank you for your service in the review process. Now that we are approaching the deadline of the author-reviewer discussion period. Please take time to read authors' responses and let them know if it addresses your concerns. Thank you.
Best,
Area Chair