- Rating
- 4
- Confidence
- 3
- Informativeness
- 2
- Presentation
- 2
- Soundness
- 3
Read review
Paper summary
The authors "introduce Gaussian Latent Dynamics-based Successor Features (GLDSF), which combines latent dynamics prediction with regularization toward an isotropic Gaussian to learn control-relevant representations with well-conditioned geometry for SF evaluation.". That is, they build successor features in the ZSRL framework on a representation based on latent dynamics in such a way that the underlying geometry is well captured by an isotropic Gaussian, which has several favorable mathematical properties. They also improve the policy retrieval mechanism over the standard SF retrieval mechanism by searching for the most appropriate policy without further policy rollout. They backup the approach with some theory and show that it outperforms other frameworks on state-based and pixel-based ExoRL benchmarks.
Critical strengths
- Clearly distinguishing the reward coefficient w and the behavior code z that are often conflated is a useful conceptual contribution
- the authors provide compelling evidence that regularizing the latent successor features representation towards an isotropic Gaussian has a positive effect on performance in the pixel-based case.
- the model-based test-time adjustment mechanism is also a clever contribution
- the experimental methodology is clean
Critical weaknesses
- The authors do not have a reproducibility statement and they do not release the code, but they provide it with the paper
- In Section 6.2, it seems to me that the relationship to using skills is misleading: "More generally, pretrained behaviors can serve as a basis for constructing downstream policies through skill composition"... In this work, a task, hence a reward function, induces a single choice of z, the framework does not use different z over time to perform a single episode, otherwise there is something I misunderstood.
Questions for authors
-
The fact that isotropic Gaussian regularization provides no advantage in the state-based case is unexpected after reading the methods. The authors probably slightly oversold this contribution before getting these experimental results. Helping the reader better understand why this regularization makes a difference in the pixel-based case and not in the state-based case would make the paper much stronger. Can you think of any toy environments where its effect can be better scrutinized?
-
Why not use a figure similar to Fig. 2 to replace Table 3? A figure would be more readable...
Minor points, typos:
- it seems that is not used, naming policies would be more standard than
- line 118: Test-time adjustment then search -> searches
- 119 : a USF -> an USF (?)
- Eq. (3) and line 130: aren't and the same thing?
- 141 code , -> code,
- 360: It improves 9 of the 12 state-based dataset–domain entries and 7 of the 12 pixel-based entries. -> wasn't it 4 domains? This raises a cherry-picking alert!
- 410: sequences , -> sequences,
- 441: Hierarchical approach (Chandrasekar & Machado, 2026) -> A hiera...
2 comments
Fine
OK, the paper looks even stronger with the new additions, I'm still in favor of accepting it
Weakness 1 (Reproducibility) We added the reproducibility statement.
Weakness2 (Skill composition) Thank you for noting this ambiguity. While w^ is fixed per episode, GLDSF-TTA re-searches the behavior code at every step (revised Section 3.2), so the selected policy can change (closed-loop control). We revised Section 6.2 to distinguish this state-dependent switching from explicit skill composition: GLDSF-TTA selects codes in the code space at each state, emerging temporal composition but without explicitly constructing composite options or learning separate skill-level dynamics.
Question1 (Modality-dependent regularization) Thank you for this insightful suggestion. We clarify that Theorem 4.1 bounds worst-case SF error amplification (under fixed feature energy and Bellman residuals) rather than guaranteeing improved average control performance in all settings.
To better explain the modality-dependent effect, we added a controlled 2D PointMass goal-reaching study in Appendix E. It has deterministic, fully actuated dynamics and six goal locations, with rewards decaying exponentially with squared goal distance. Both modalities use identical dynamics, rewards, and pretraining trajectories; only observations differ: clean 2-dimensional physical coordinates versus 24×24 rendered pixels. The feature dimension d is fixed at d=16 throughout. We report here: (1) condition number (max/min eigenvalue ratio of the feature second-moment matrix); (2) effective rank (entropy-based feature dimensionality); (3) position R² (average linear-probe R² for physical coordinates); and (4) control return (discounted downstream reward of the frozen policy). For the specific information please see newly added Appendix E. All values in tables are mean ± standard deviation over six seeds and rounded to one decima.
Table 4: Representative PointMass results.
The key difference is the feature geometry. NoReg retains linearly decodable positions in both modalities, but its pixel representation is much more ill-conditioned (~107× higher condition number than states; effective rank 1.9/16). Gaussian regularization alleviates this severe spectral concentration (condition number 40.1; effective rank 12.1), accompanying a substantial return increase. In contrast, NoReg state observations already have a comparatively reasonable geometry, leaving less room for additional regularization to translate into control gains. Indeed, the state-based return increases only modestly.
Table 5: Gaussian regularization coefficient sweep on pixel observations.
The coefficient sweep reveals a trade-off between geometry and task information. Weak-to-moderate regularization improves conditioning, effective rank, and return, whereas excessive regularization (α₃=10) pushes condition number toward 1 and effective rank toward 16 while suppressing task information—dropping position-probe score to 0.2 and return to 28.5 (below NoReg's 45.0). Thus, effective control requires balancing geometric regularity with preserving task-relevant features, as favorable geometry alone is insufficient. More results are presented in Appendix E.
We have revised Sections 4.1 and 5.3 to clarify the scope of the theoretical result and to better understand geometry importance, respectively.
Question 2 We followed the suggestion and replaced Table 3 with Figure 2; average returns are reported in Section 5.3.
Minor: We addressed all typos, including "improves 9 of the 12 state-based…". We appreciate you pointing this out! Our original comparison between GLDSF and GLDSF-TTA included insignifaicant entries due to an oversight. To prevent overstatement, we have removed the per-entry win counts and focus on aggregate performance and task variation.