- Rating
- 3(+1)
- Confidence
- 1(−1)
- Informativeness
- 2(+1)
- Presentation
- 2
- Soundness
- 2
Read review
Paper summary
LAPIS learns a policy from videos with few action labels. It selects high-value examples, trains a flow policy to generate latent actions, and decodes them into robot commands. All experiments are in simulation.
Critical strengths
s1. The paper studies whether improvements in latent actions translate into better execution.
s2. The ablations help identify which components improve performance.
Critical weaknesses
w1. There are no real-robot experiments, despite the paper’s focus on whether latent actions can actually be executed.
w2. The theory assumes an exact decoder and actions restricted to the dataset. The actual method does not satisfy these assumptions.
w3. Selecting training examples does not guarantee that the policy’s generated latents are executable. Codebook projection does not guarantee this either.
w4. The method mainly combines existing filtering, flow-policy, and CFG techniques. Its novelty beyond this combination is limited.
Questions for authors
q1. Could you test the method on a real robot to verify its claims about execution?
q2. How often do policy-generated latents fail to produce the intended outcome after decoding?
q3. Does stronger CFG increase these execution errors?
q4. Could you compare LAPIS with a flow-based GCIQL policy using the same latent model, decoder, and training budget? This would help isolate the benefit of your filtering rule.
5 comments
Response to Reviewer he27 (2/3): execution failures, guidance strength, codebook projection
w3 / q2 / q3. Thank you for these questions, which go to the core of the paper. We agree that selecting training examples and codebook projection do not by themselves guarantee execution; Appendix D.8 says the same. We therefore measured it.
Measurement. A latent encodes an intended transition, so its execution fails when the executed transition encodes to a different latent. The submitted paper measures this round-trip error as in Fig. 3, averaged over decision steps on antmaze-medium. To report how often execution fails, we turn it into a failure rate on cube-single and scene (4 seeds; 5 goals × 20 episodes per task). At every decision point we save the simulator state, execute the 3-step chunk decoded from the latent chosen by the policy, re-encode the start observation and the observation 3 steps later, and record . We then restore the saved state and continue the episode unchanged. The reference is the distribution of dataset latents, decoded and executed the same way from dataset states. A latent fails if its exceeds the 95th percentile of this reference, so dataset latents fail 5% of the time by construction; the threshold is set per task and seed, since each seed has its own LAM. We compare LAPIS (, the reported setting), the same LAPIS checkpoints without codebook snapping, a max-Q method (CRL-style; Eysenbach et al., 2022), and AWR-weighted flow (Peng et al., 2019). The tables are Tables 19–20 in the new App. E.9 of the revision.
Execution failure rate (%), 4 seeds:
| setting | cube-single | scene |
|---|---|---|
| dataset latents (reference) | 5.0 | 5.0 |
| LAPIS () | 5.8 | 12.6 |
| AWR-weighted flow | 7.1 | 15.4 |
| max-Q (CRL-style) | 6.7 | 24.0 |
- q2. LAPIS policy latents fail 5.8% of the time on cube-single, close to the dataset reference, and 12.6% on scene, the lowest among the compared methods; max-Q fails about twice as often there (24.0%).
- q3. Fig. 3 already reports this for guidance strength: for LAPIS from the BC end () to the aggressive end () on antmaze-medium, at . Up to , the value we use, it stays at the level (6.1 at , 1 and 3; BC 6.0). Much stronger guidance does increase it (6.2 at , 6.5 at ), and success drops at (success panel of Fig. 3).
- w3. The low failure rate is not produced by codebook projection either. We evaluated the same checkpoints with and without snapping (LAPIS , 4 seeds):
| cube-single | scene | |
|---|---|---|
| failure rate (%), with / without snap | 5.8 / 5.8 | 12.6 / 12.4 |
| success difference (without − with, points) | −2.3 ± 4.0 | +1.5 ± 6.5 |
Without snapping, the failure rate is unchanged and the success difference is within seed variation (paired by seed). Both executability and performance come from the latents the policy generates.
Response to Reviewer he27 (3/3): novelty and comparison with flow-based GCIQL
w4. The method mainly combines existing filtering, flow-policy and CFG techniques. Thank you for this comment. We agree that the individual components exist: the gate belongs to the family of advantage-filtered regression (CRR; Wang et al., 2020), and flow actors and guidance are existing techniques. The revision adds the missing citations for flow matching and classifier-free guidance (Sec. 6). Our contribution has three parts.
- Why these components are needed in a learned latent action space. Which latents can be realized depends on the state. BC only imitates observed latents, so this causes no problem. Offline-RL improvement, however, moves away from the observed distribution and selects latents that are unrealizable at the current state. This difficulty does not exist for physical actions.
- An analysis that identifies the condition. We show that latent improvement carries over exactly to execution when the policy stays within observed state–latent transitions (Theorems 1 and 2; Proposition 2 for learned decoders). The gate, which selects only observed transitions, and the flow actor, which represents their distribution, follow from this condition.
- Controlled experiments showing what is decisive: changing the extraction rule under the same critic, and changing the critic under the same rule (table below).
With the same extraction rule, replacing the critic lowers performance substantially (first two rows below), so a simple combination of the components does not work.
q4. Comparison with a flow-based GCIQL policy using the same latent model, decoder and training budget. Thank you for the suggestion. We ran this comparison: same LAM, decoder, 5% labels, training budget and evaluation protocol; mean ± std over 4 seeds.
| method | antmaze-medium | cube-single | cube-double | scene |
|---|---|---|---|---|
| LAPIS (TD-InfoNCE critic + gate + flow) | 49±3 | 62±5 | 31±6 | 29±4 |
| IQL critic + gate + flow (LAPIS actor) | 32±4 | 42±14 | 12±4 | 20±3 |
| IQL critic + AWR-weighted flow (Flow-GCIQL) | 18±4 | 55±8 | 13±4 | 19±4 |
| IQL critic + DDPG+BC flow | 38±7 | 6±4 | 5±1 | 9±1 |
The critics are TD-InfoNCE (Zheng et al., 2024) and IQL (Kostrikov et al., 2022); the flow-actor extractions are AWR (Peng et al., 2019) and the one-step DDPG+BC of FQL (Park et al., 2025).
- LAPIS outperforms the flow-based GCIQL the reviewer suggested (IQL critic + AWR-weighted flow) on all four tasks.
- With the same flow actor and the same gate, replacing only the critic with IQL lowers performance on all four tasks.
- The other flow-based GCIQL variant (DDPG+BC flow) is also below LAPIS on all four tasks.
- The full grid, varying both the critic and the extraction rule, is Table 4(b) in Sec. 7.4 of the revision.
References
- Bruce et al. (2024). Genie: Generative interactive environments. ICML.
- Chen et al. (2024). IGOR: Image-GOal representations are the atomic control units for foundation models in embodied AI. arXiv:2411.00785.
- Eysenbach, Zhang, Levine, Salakhutdinov (2022). Contrastive learning as goal-conditioned reinforcement learning. NeurIPS.
- Kostrikov, Nair, Levine (2022). Offline reinforcement learning with implicit Q-learning. ICLR.
- Park, Li, Levine (2025). Flow Q-learning. ICML.
- Peng, Kumar, Zhang, Levine (2019). Advantage-weighted regression: Simple and scalable off-policy reinforcement learning. arXiv:1910.00177.
- Schmidt, Jiang (2024). Learning to act without actions. ICLR.
- Wang et al. (2020). Critic regularized regression. NeurIPS.
- Ye et al. (2025). Latent action pretraining from videos. ICLR.
- Zheng, Salakhutdinov, Eysenbach (2024). Contrastive difference predictive coding. ICLR.
Response to Reviewer he27 (1/3): real robots and the theory's assumptions
We thank the reviewer for focusing on the central question of the paper: whether latent actions selected by the policy are actually executed. Changes in the revised PDF are shown in blue.
w1 / q1. Real-robot experiments. We agree that real-robot validation is valuable; Appendix G.2 (Limitations) of the submission states that real-robot deployment is outside the scope of the current evaluation. Our claims are comparative: judging them requires comparing several critics and extraction rules on the same LAM and decoder, with multiple baselines under the same data and evaluation protocol, across tasks and seeds. Such controlled, repeated comparisons are still hard to run on real robots, so we focused on the standard benchmarks of offline goal-conditioned RL (OGBench visual tasks, D4RL Kitchen), and within this setting we verified the execution claim directly (q2/q3 below). Moving this improvement rule onto real robots, building on the imitation results of real-robot latent-action work, is the natural next step.
w2. The theory assumes an exact decoder and actions restricted to the dataset. Thank you for raising this point. Theorems 1 and 2 are indeed stated for the batch-exact decoder and batch-constrained policies, but neither assumption is specific to LAPIS. Existing latent-action pipelines (Schmidt & Jiang, 2024; Ye et al., 2025; Chen et al., 2024; Bruce et al., 2024) train latent policies by behavior cloning and decode latents into actions with a small labeled set; without stating a theory, they rely on the same two conditions to reproduce the data's behavior.
What is specific to offline RL is that policy improvement shifts the policy away from qB(z∣s), and methods that maximize the critic can push latents outside the data, where the decoder receives unseen inputs and execution can break. Theorems 1 and 2 identify when improvement still carries over to execution, and this condition is the design principle of LAPIS: it scores only observed transitions with the critic and trains the policy on the selected ones, which keeps the decoder in the same regime as in BC. We measured how far it departs (q2/q3 below).