acceptodds
Under review as a conference paper at ICLR 2027

Identifiability of Latent State Dynamics under Adaptive Multi-Environment Selection

Abstract

Latent state models underpin many machine learning systems, but in practice data is rarely collected i.i.d. and is instead generated by an adaptive mechanism that depends on the latent state itself. This introduces a form of hidden confounding that standard causal representation learning (CRL) methods do not address. We propose the Latent Adaptive Selection Model (LASM), a structural causal framework for modeling latent dynamics under adaptive data collection. Unlike prior CRL settings with exogenous interventions, LASM captures endogenous selection, where policies both depend on and distort the latent state. Our main contributions are: (i) a geometric characterization of non-identifiable directions induced by adaptive policies via the Policy Selection Gradient Matrix; (ii) a sharp identifiability result showing that, under a logged-policy anchor and a policy diversity condition, the latent transition matrix is exactly recoverable across environments; (iii) a tight lower bound on the number of environments required for identifiability; and (iv) the Multi-environment Adaptive Invariant Latent Estimator algorithm, which enforces latent frame anchoring via a logged-policy consistency objective and promotes diversity through a log-determinant regularizer. Experiments on synthetic data verify the predicted identifiability threshold, while real-world experiments on ASSISTments demonstrate that recovered skill-transition structures are highly reproducible across schools () only under multi-environment pooling, and collapse in single-environment settings.

Primary area:
causal reasoning
Keywords:
Causal Learning, Latent Dynamics, Adaptive Environment

Reviews

Among:
Shape:
1234this paper, Bottom 10%
Figure 1. Mean review rating of the reviewed papers at ICLR 2027. This paper averages 1.67 over 3 reviews, above 10% of the others.
Reviewer_RoXvOpenReview ↗
Rating
2
Confidence
2
Informativeness
2
Presentation
3
Soundness
1
Read review
Paper summary

The paper proposes identifying latent linear-Gaussian dynamics by combining known state-dependent policies across environments. The policy-spanning argument provides an interesting way to fix latent coordinates. The question is relevant, and the synthetic experiments suggest benefits from combining diverse policies.

Critical strengths

Using known policies to anchor latent coordinates is an interesting idea. The policy-spanning argument is interpretable and addresses a relevant distinction between predicting observations and recovering latent dynamics.

Critical weaknesses

The main concern is the justification of Theorem 1. Appendix A.2, lines 652-659, moves from collective emission injectivity to a shared affine latent transformation and pointwise policy preservation. I could not establish these steps under the stated noisy, action-dependent emission model. This is a concern about the proof provided, rather than a demonstrated counterexample to the theorem.

Algorithm 1 also assumes PCA recovers Zhat=MZ, which does not generally follow from nonlinear emission injectivity. The ASSISTments validation additionally needs clarification about policy coordinates and school splits.

Questions for authors
  1. Theorem 1 needs a clearer model-equivalence argument. Appendix A.2, lines 652-659, moves from collective emission injectivity to a shared affine latent transformation and pointwise equality of policy probabilities. I could not establish these implications from the stated assumptions, particularly with noisy, action-dependent emissions. Could the authors supply the missing argument or clarify the required assumptions? Once a shared policy-preserving transformation is established, the spanning argument appears valid. This concern is about the proof provided; I have not established that the theorem is false.

  2. The estimator's scope is unclear. Algorithm 1 assumes PCA recovers Zhat=MZ, although the formulation permits nonlinear injective emissions. Injectivity alone does not ensure this: for standard Gaussian Z and g(z)=(z,z^2), population one-component PCA retains z^2-1 and loses the sign of Z. Is the solver intended for a narrower linear-emission setting? Please specify the conditions supporting its recovery claim and explain how its transition regression relates to Equations 7-9.

  3. The real-data interpretation needs clarification. ASSISTments policies are estimated rather than supplied as known latent-coordinate functions. How are their coordinates anchored consistently across schools? Please also specify the school subsets used for Table 3; the description of disjoint leave-one-out subsets is ambiguous. The reported agreement is useful evidence of stability, but does not by itself establish identification.

Reviewer_t8u3OpenReview ↗
Rating
2
Confidence
2
Informativeness
2
Presentation
2
Soundness
1
Read review
Paper summary

This paper studies the identifiability of latent state dynamics under adaptive data collection, where the action selection mechanism depends on the underlying latent state. The authors introduce the Latent Adaptive Selection Model (LASM), which assumes shared linear-Gaussian latent dynamics across multiple environments with different adaptive selection policies.

The main theoretical contributions: (1) characterizing non-identifiable directions (2) establishing an identifiability result for the latent transition matrix under certain conditions; (3) deriving a lower bound on the number of environments required for identifiability. The paper also proposes the Multi-environment Adaptive Invariant Latent Estimator (MAILE), which uses logged-policy information to anchor the latent representation.

Critical strengths
  1. The paper investigates latent dynamics identification under adaptive data collection, where actions depend on unobserved latent states. This is a relevant problem in applications such as recommender systems, education, and healthcare.

  2. The Policy Selection Gradient Matrix provides an intuitive geometric perspective on which latent directions can be distinguished by an adaptive policy.

Critical weaknesses
  1. The main identifiability results rely on a number of very strong assumptions whose necessity and practical validity are insufficiently justified. Particularly, assumptions 3 and 4, without adequately discussing their necessity or the extent to which they can be relaxed.

Assumption 3 requires the stacked emission map to be injective, with a full-rank Jacobian on the support of the latent variables. This is a strong requirement for a partially observed latent system. In many practical applications, the observations may not uniquely determine the underlying latent states, even when information from multiple actions is considered. Assumption 4 requires the adaptive policies to be known log-linear functions of the latent states, with fully observed policy parameters. This is also a highly restrictive assumption. Although policies may be logged in practical systems, this does not necessarily imply that their functional dependence on an unobserved latent state is known in a fixed coordinate system.

These assumptions are especially consequential because the paper's central contribution is an exact identifiability guarantee. It is therefore important to understand which assumptions are genuinely necessary for identifiability and which are merely sufficient conditions adopted for mathematical convenience.

  1. Insufficient justification of the main theoretical argument.

In Appendix A.2, the proof of Theorem 1 states that, by Assumption 3, the injectivity of the emission model implies that two observationally equivalent latent representations are related by an affine transformation. However, injectivity alone does not generally imply that two latent representations must be related by an affine transformation, especially when the emission functions are allowed to be unknown and nonlinear. An additional argument or structural restriction seems necessary to justify this step. Please correct me if I am mistaken.

Questions for authors

Necessity of Assumptions 3 and 4: To what extent are these assumptions necessary for the main identifiability results?

Justification of Theorem 1: In Appendix A.2, why does injectivity of the emission function imply that observationally equivalent latent representations are related by an affine transformation?

Practical availability of the known policies: How can Assumption 4 be satisfied when the policies depend on latent states that are not directly observed? In the real-data experiments, the policies are estimated rather than known. How sensitive is the method to estimation errors or misspecification of these policies?

Reviewer_cySNOpenReview ↗
Rating
1
Confidence
2
Informativeness
1
Presentation
3
Soundness
2
Read review
Paper summary

The paper studies the identifiability of the transition function in a Latent Adaptive Selection Model (LASM), which is essentially a POMDP without rewards, restricted to linear-Gaussian transitions Zt+1=BZt+CAt+ϵtZ_{t+1} = BZ_t + C_{A_t} + \epsilon_t, injective emission functions, and known log-linear policies (i.e. softmax policies with logits that are linear in the latent state). Identifying the latent transition dynamics is thus reduced to recovering the matrix BB. It is assumed that we have data from a finite number of environments with the same transition and emission functions, but different policies.

In this setting, the paper defines the Policy Selection Gradient Matrix (PSGM) Φe\Phi^e for any environment ee with its own policy, the kernel of which contains the latent directions to which the policy is insensitive. The main result shows that BB is exactly identifiable whenever the intersection of ker⁡Φe\ker \Phi^e across all environments is zero, which the paper calls the policy diversity condition. This holds for any number of environments, including a single one whose own kernel is zero. Since a softmax over kk actions responds to at most k−1k − 1 latent directions, the condition requires at least ⌈d/(k−1)⌉\lceil d / (k - 1) \rceil environments.

Experiments on synthetic and semi-synthetic data show the transition matrix becoming recoverable exactly at the predicted identifiability threshold, and on student logs from ASSISTments 2009, with schools as environments, the estimated transitions are consistent across disjoint groups of schools only when environments are pooled.

Critical strengths

The paper is easy to read and conveys the "identifiability arises from policy diversity across adaptive environments" intuition well.

Critical weaknesses
  1. In the proof of Thm 1, the authors state that "By Assumption 3, the emission model is injective, so the two latent representations are related by an affine change of coordinates" -- this does not follow unless we assume the emission function is linear, which is not stated anywhere. Given the rest of the restrictive assumptions on the linearity of transitions and policies, the technical contribution is limited

  2. The paper overstates its novelty. Reizinger et al. show that policy diversity yields identifiability in their 2025 paper Skill Learning via Policy Diversity Yields Identifiable Representations for Reinforcement Learning, using the same term, and Schur studies nearly the same setup of distinct policies over shared dynamics in the 2026 paper Identifying Latent Actions and Dynamics from Offline Data via Demonstrator Diversity. Neither is cited here.

Questions for authors
  1. Did you intend for the emission function to be linear, or is there something I'm genuinely missing?
  2. How does this work differ from the works of Schur and Reizinger et al?

From OpenReview, CC BY 4.0. Authors as signed there; shown unedited.

Related papers

Loading the map…

Discussion (0)

Sign in to comment.