- Rating
- 3(+1)
- Confidence
- 2
- Informativeness
- 2
- Presentation
- 2
- Soundness
- 3
Read review
Paper summary
This paper introduces Latent Successor Models (LSM) for offline visual reinforcement learning. The main successor model adapts a flow matching based method predicts future latent states with a given horizon and current state. The successor model is trained in a boostrapped method whose target network is the EMA version of the online network. The training target is only the encoded future state, without reward or value. This enables critic model to learn truncated -return using latent states directly sampled from the successor model at selected horizons, without generating every intermediate state. The experiments show that the proposed method can predict future latent and values in a long horizon with a much smaller error compare to autoregressive methods, which results in a higher success rate than other baselines.
Critical strengths
The presentation of the paper is generally clear. The core mechanism using bootstrapped flow matching as a successor model is novel and the proposed method achieves better performance than other baselines over multiple benchmarks.
Critical weaknesses
The main concern is about the missing ablations, for example, "Bootstrapped velocity at all noise levels" (Eq. 8) vs clean-future-state bootstrap (Eq. 7, LUHM-style) is the claimed main novelty, but the proposed method is different from LUHM in many aspects including the reward estimator and actor desgin. The authors are encouraged to conduct more detailed ablations to help understand the effect of each module.
Questions for authors
No
2 comments
I appreciate the authors new ablations and I will update the recommendation to weak acceptance.
We thank the reviewer for the detailed feedback and suggestions. The reviewer raised the question about the contributions of the three different components between LSM and LUHM, which we have addressed by running additional experiments to evaluate three variants of LSM, each replacing only its training objective, return estimator, or policy with the corresponding component from LUHM. We conduct experiments on visual-cube-double-task3 and visual-scene-task3 using the same training setup and evaluation protocols as in Sec. 5.3. Below, we report success rates as the mean and standard deviation across 4 random seeds.
These results suggest that policy extraction and the bootstrapped velocity objective contribute to LSM’s performance, while the choice of return estimator has little effect on these two tasks. This observation is compatible with the discussion in Chung et al. (2026) [1], where the λ-return is a special case of the ν-return. Are there additional ablation experiments that the reviewer would like to see?
[1] Chung, H., Lee, J., & Oh, S. (2026). Offline reinforcement learning with universal horizon models. arXiv preprint arXiv:2605.15603.