- Rating
- 3
- Confidence
- 2
- Informativeness
- 3
- Presentation
- 3
- Soundness
- 3
Read review
Paper summary
This paper studies whether latent actions learned from unlabeled trajectories capture command effects that can be reused across states. It shows that fitting observed trajectories and satisfying algebraic or cycle-consistency constraints can still yield incorrect action operators. For finite, fully observed systems with reversible commands, distinct command effects, and a connected transition graph, the authors prove that repeated-command trajectories combined with trials that re-execute a command after an intervening command identify the true operators up to a global relabeling, under full-support sampling. They also derive a bound on operator disagreement using local reconstruction error and independently sampled target reuse error. In controlled experiments, training on re-execution trials recovers the true operators across all restarts, while repeated-command training can produce consistent but incorrect solutions. Experiments with frozen LAPO models across 15 Procgen environments report 33.0% accuracy for transferred codes versus 54.4% for codes inferred at the same targets, illustrating a gap between local reconstruction and cross-state action reuse.
Critical strengths
- The paper clearly distinguishes reconstruction, consistency, and action reuse, with concrete counterexamples showing why consistency alone does not guarantee correct action semantics.
- The identification theorem and error bounds provide a principled connection between data collection, reuse evaluation, and operator recovery.
- The experiments include useful controls, such as matched transition budgets, same-target comparisons, and bootstrap uncertainty estimates.
Critical weaknesses
- The connection between the theory and neural-model experiments remains incomplete. Procgen reuse errors measure the performance of the current encoder–decoder pipeline, without distinguishing unstable code selection, failures in cross-state decoding, and the absence of an effective fixed action dictionary. Their implications for operator identification therefore remain limited.
- The controlled experiments cover a restricted identification setting. All training experiments use regular systems with , where native reuse already yields independent uniform targets and the data provide nearly complete coverage. These results do not sufficiently validate the complementary roles of the two protocols in general sparse systems or recovery under limited coverage.
- The cycle diagnosis does not sufficiently separate generation errors from action-correspondence failures. The cycle re-encodes model-generated images, so low return accuracy may also reflect generation quality or encoder sensitivity to generated frames. The existing controls do not clearly distinguish these effects from failures of cross-state action reuse.
Questions for authors
- Could the authors compare source-inferred codes with fixed per-command codes selected on separate data and evaluated at independent targets? This would help distinguish failures in code selection from limitations in the decoder’s cross-state use of the codes.
- Could the authors compare persistence-only, native-reuse-only, and their combination in sparse connected systems with , using matched transition budgets and varying sampling coverage? This would clarify the complementary roles of the protocols and recovery under limited data.
- Could the authors add controls that re-encode generated frames at the same state and replace generated target successors with real ones? These would help separate generation errors and encoder sensitivity to generated frames from cross-state reuse failures.
Thank you for the clarifications and for addressing my questions. After considering your response, I’m keeping my original score.