- Rating
- 2
- Confidence
- 3
- Informativeness
- 1
- Presentation
- 3
- Soundness
- 2
Read review
Paper summary
This paper analyzes how nested training of latent action model affects their information in the representations and how downstream policy generalizes under distribution shift. They compare flat baseline with multi-prefix and retained-prefix setting where every or sampled prefix latent actions per sample contribute to training. From cross-task transfer analysis from LIBERO, LIBERO-Plus and MimicGen simulation environments, they present action recoverability, generalization performance and action supervision diagnostic showing that nested training enables generalizable latent actions.
Critical strengths
- Interesting research question. Latent action models store sufficient information to predict the next state from the current state, expecting action-relative information being stored in the latent action. This paper asks how the information is stored and organized when nested training is applied on the latent action and explains whether that organization relates to generalization under distribution shift.
Critical weaknesses
-
Limited Contribution. Nested or ordered training is already known to help in general representation [1, 2], visual tokenization [3] and (latent) action tokenization [4, 5]. They also show improvements downstream performance, data efficiency and generalization. Compared to prior methods, they show semantic hierarchy of low-to-higher level details in FlexTok and coarse-to-fine trajectories in OAT even without explicit bias except nested training as the methods elaborate. Even if nest training can also be applied to latent action (which is also shown in FlexLAM), this work's analysis are neither sufficient nor have a interesting latent action specific property explored as a standalone contribution.
-
Limited Analysis. Linear probing and R² analysis only comes from linear decodability, not the deeper structure or geometry of the latent representations or how policy uses them. Also, MoLA seems to not be a reasonable baseline to compare on this work, unless this work is built on base architectures of MoLA. The results also show disagreement between action R² and success rates (Table 2 vs 3), inconsistent behavior (Table 9), which are insufficiently discussed. Also, fundamental behaviors of latent action and nested training are missing, for example, link to latent action bottleneck or huge gap between two nested variants.
[1] Matryoshka Representation Learning, NeurIPS 2022
[2] Learning Ordered Representations with Nested Dropout, ICML 2014
[3] FlexTok: Resampling Images into 1D Token Sequences of Flexible Length, ICML 2025
[4] OAT: Ordered Action Tokenization, RSS 2026
[5] FlexLAM: Resolving the Bottleneck Trade-off in Latent Action Learning, Arxiv 2606.19408
Questions for authors
-
Semantics of Latent Action. Do your existing results show what information latent action tokens carry, like how high-to-low-level visual content in FlexTok or coarse-to-fine trajectories in OAT? Beyond existing analysis, what does this work show that is not expected from prior works?
-
Experimental Configurations. Can you clarify the downstream policy baseline, design and configurations how latent action is used?