acceptodds
Under review as a conference paper at ICLR 2027

Consistency Is Not Identification: What Reuse Reveals About Latent Actions

Abstract

Latent-action models learn action representations from unlabeled video by encoding observed transitions into codes. Many methods shape these representations using algebraic constraints or cycle-consistency checks that re-encode a model's own predictions. However, we prove that a model can fit the observed trajectories and satisfy these constraints yet predict the wrong effect when an action is reused in a new state. In this sense, consistency is not identification. We then show which data resolve this ambiguity. For reversible actions on a connected finite state graph, repeated-command trajectories and trials that re-execute a command after an intervening one together uniquely determine the set of true action operators. To evaluate learned actions in this setting, we further show that, for deterministic models with as many codes as commands, the distance between learned and true operators is bounded by local reconstruction error plus reuse error at independently sampled targets. In controlled experiments, training on re-execution trials learns the true state-transition operators on every restart, whereas training only on repeated-command data can yield consistent but incorrect operators. In LAPO models across Procgen environments, codes inferred at the target select the correct outcome in of eligible pairs, compared with for codes transferred from another state on the same pairs. These results show that the gap between reconstructing observed transitions and reusing actions in new states also appears in latent-action models trained on video.

Primary area:
unsupervised, self-supervised, semi-supervised, and supervised representation learning
Keywords:
latent actions, latent action models, identifiability, world models, learning from video, cycle consistency.

Reviews

Among:
Shape:
1234this paper, Top 30%
Figure 1. Mean review rating of the reviewed papers at ICLR 2027. This paper averages 2.67 over 3 reviews, above 70% of the others.
Reviewer_TeV3OpenReview ↗
Rating
3
Confidence
2
Informativeness
3
Presentation
3
Soundness
3
Read review
Paper summary

This paper studies whether latent actions learned from unlabeled trajectories capture command effects that can be reused across states. It shows that fitting observed trajectories and satisfying algebraic or cycle-consistency constraints can still yield incorrect action operators. For finite, fully observed systems with reversible commands, distinct command effects, and a connected transition graph, the authors prove that repeated-command trajectories combined with trials that re-execute a command after an intervening command identify the true operators up to a global relabeling, under full-support sampling. They also derive a bound on operator disagreement using local reconstruction error and independently sampled target reuse error. In controlled experiments, training on re-execution trials recovers the true operators across all restarts, while repeated-command training can produce consistent but incorrect solutions. Experiments with frozen LAPO models across 15 Procgen environments report 33.0% accuracy for transferred codes versus 54.4% for codes inferred at the same targets, illustrating a gap between local reconstruction and cross-state action reuse.

Critical strengths
  1. The paper clearly distinguishes reconstruction, consistency, and action reuse, with concrete counterexamples showing why consistency alone does not guarantee correct action semantics.
  2. The identification theorem and error bounds provide a principled connection between data collection, reuse evaluation, and operator recovery.
  3. The experiments include useful controls, such as matched transition budgets, same-target comparisons, and bootstrap uncertainty estimates.
Critical weaknesses
  1. The connection between the theory and neural-model experiments remains incomplete. Procgen reuse errors measure the performance of the current encoder–decoder pipeline, without distinguishing unstable code selection, failures in cross-state decoding, and the absence of an effective fixed action dictionary. Their implications for operator identification therefore remain limited.
  2. The controlled experiments cover a restricted identification setting. All training experiments use regular systems with m=N=8m=N=8, where native reuse already yields independent uniform targets and the data provide nearly complete coverage. These results do not sufficiently validate the complementary roles of the two protocols in general sparse systems or recovery under limited coverage.
  3. The cycle diagnosis does not sufficiently separate generation errors from action-correspondence failures. The cycle re-encodes model-generated images, so low return accuracy may also reflect generation quality or encoder sensitivity to generated frames. The existing controls do not clearly distinguish these effects from failures of cross-state action reuse.
Questions for authors
  1. Could the authors compare source-inferred codes with fixed per-command codes selected on separate data and evaluated at independent targets? This would help distinguish failures in code selection from limitations in the decoder’s cross-state use of the codes.
  2. Could the authors compare persistence-only, native-reuse-only, and their combination in sparse connected systems with m<Nm<N, using matched transition budgets and varying sampling coverage? This would clarify the complementary roles of the protocols and recovery under limited data.
  3. Could the authors add controls that re-encode generated frames at the same state and replace generated target successors with real ones? These would help separate generation errors and encoder sensitivity to generated frames from cross-state reuse failures.
Reviewer_K6NPOpenReview ↗
Rating
3(+1)
Confidence
3
Informativeness
2
Presentation
2(+1)
Soundness
3
Read review
Paper summary

The paper gives sufficient conditions for identifiability of actions (up to global permutation) given action-less state trajectories in an RL setting. The authors assume deterministic transitions and population distributions. Their main result (Theorem 3.3) shows that, under some additional assumptions, when native reuse laws agree and persistence run laws agree the commands are identified (upto permutation).

Critical strengths
  • The motivation is clear and the general structure and flow of the paper is good.
  • I like the figures and the visualizations. They help to understand the concepts of the paper.
  • I think the results are interesting in general
Critical weaknesses
  • The theoretical results seem to be a little limited. The Propositions seem to be largely simple consequences of known results (see questions section) and Theorem 3.3 seems to apply in a limited setting (pointwise-distinguishable permutations with "connected underlying undirected graph").
  • My biggest issues is that some of the phrases are used inconsistently which makes the paper hard to read. For example, 1. it seems that the terms "command", "action" and "operator" are used for the same thing, 2. it is unclear if the initial state is fixed or sampled uniformly in section "Data-collection protocols." 3. What is the definition of “Reversible commands”? Does it simply mean bijective? If so then use bijective. If you mean "there exists another action in the operator family which is the inverse" then say so. Is “Pointwise distinguishability” simply infectivity within the family? Please try to use established mathematical phrases if possible, define any non-standard terminology clearly and mathematically precisely and use all phrases consistently. Do not use two or more phrases to mean the same thing.
Questions for authors

Can you clarify the novelty of all the Propositions and the Theorems. I liked that you used Propositions for less novel results but you need to clarify in what way the results are novel (or already established). Some of them (e.g. Proposition 3.1 and Proposition 4.1 but maybe more) seems to be simple applications for well known results in group theory and probability theory. In this case this needs to be made clear.

For the rebuttal please be very precise with what you want to add for the final version of the paper. For example: "Proposition 3.1 is a simple application of lemma xy in book of group theory zx. We now added the following sentence after Proposition 3.1 "[sentence explaining novelty]. Proposition 3.2: ..."

Reviewer_xy6fOpenReview ↗
Rating
2
Confidence
3
Informativeness
2
Presentation
2
Soundness
2
Read review
Paper summary

This paper examines when and why latent-action models that satisfy internal consistency including algebraic constraints, cycle consistency, and exact reconstruction of observed transitions may nevertheless fail to identify the true action operators required for correct cross-state reuse. The authors present counterexamples demonstrating that target-blind consistency is insufficient and investigate identification conditions under two unlabeled data protocols. Experiments on controlled finite systems and frozen LAPO models in Procgen support their theoretical findings.

Critical strengths
  1. The paper establishes a negative result: exact reconstruction and internal algebraic or cycle consistency do not, by themselves, guarantee identification of action operators that can be correctly reused across states.
  2. The paper provides a useful evaluation perspective: latent actions should be assessed through reuse against observed environment outcomes, alongside reconstruction and internal consistency checks. This directly tests whether learned codes retain their action meaning across states.
  3. The authors provide an anonymized repository containing the experimental pipeline, supporting transparency and facilitating reproduction of the reported results.
Critical weaknesses
  1. Although the paper offers useful recommendations for evaluation, it provides limited actionable guidance on improving action identification during LAM training. Empirically validated choices of training objectives, architectures, or data-collection protocols would strengthen its contribution.
  2. The video experiments are limited to frozen checkpoints of the original LAPO model. Evaluating subsequent methods that explicitly seek to improve latent action learning, such as https://arxiv.org/abs/2502.00379 and https://arxiv.org/abs/2505.04999, would help determine whether the observed reconstruction–reuse gap persists across stronger models. The current evidence supports this diagnosis for LAPO, while leaving its broader empirical generality unresolved.
  3. The theoretical guarantees are derived in a finite, fully observed, deterministic setting that abstracts away key challenges in learning actions from video. It remains unclear how the identification results extend to partial observability, stochastic dynamics, and learned visual representations, limiting their direct applicability to practical LAMs.
  4. Some parts of the paper, particularly the abstract and contribution summary, are difficult to follow. Dense sentences combine multiple assumptions and results, while the relationships between commands, latent codes, and action operators are not immediately clear. More intuitive explanations before the formal claims would improve readability.
Questions for authors
  1. Could you describe a practically implementable data-collection and training protocol that improves action identification without observing ground-truth action labels? In particular, how could environment rollouts provide repeated executions of the same command at independently sampled target states, and how would departures from independence affect your guarantees?
  2. Could you extend the reuse evaluation to more recent latent action models, such as LAOM and CLAM, to test whether the reconstruction–reuse gap persists in methods designed to improve latent action quality?
1 comment

Thank you for the clarifications and for addressing my questions. After considering your response, I’m keeping my original score.

From OpenReview, CC BY 4.0. Authors as signed there; shown unedited.

Related papers

Loading the map…

Discussion (0)

Sign in to comment.