acceptodds
Under review as a conference paper at ICLR 2027

Beyond Reconstruction: Dynamical Admissibility in Physics-Based Temporal Modeling

12 views

Abstract

Learning representations from observations supports physics-based modeling of evolving systems. Accurate reconstruction and small mechanistic residuals on sampled trajectories do not, by themselves, establish whether a learned representation supports closed latent evolution. We study the dynamical admissibility of time-varying representations, connecting temporal causal representation learning with dynamical systems. Using semiconjugacy and exact lumpability, we distinguish latent closure from compatibility with a prescribed mechanistic flow. Invertible reparameterizations preserve closure through conjugate dynamics, whereas retaining the prescribed flow requires them to commute with that flow. In approximate settings, we separate representation-induced nonclosure from transition-model error, and derive finite-horizon bounds relating nonclosure to reconstruction that identify when accurate reconstruction controls nonclosure and when unresolved future-state information imposes a tradeoff between future reconstruction accuracy and latent closure. Experiments on controlled synthetic systems and real-world cerebral perfusion data illustrate these relationships and the complementary roles of reconstruction accuracy, latent-transition consistency, and mechanistic consistency.

Primary area:
applications to physical sciences (physics, chemistry, biology, etc.)
Keywords:
Temporal Causal Representation Learning, Latent Dynamical Systems, Physics-Informed Learning

Reviews

Among:
Shape:
1234this paper, Bottom 49%
Figure 1. Mean review rating of the reviewed papers at ICLR 2027. This paper averages 2.25 over 4 reviews, above 49% of the others.
Reviewer_bevSOpenReview ↗
Rating
2(+1)
Confidence
3
Informativeness
2
Presentation
3
Soundness
2
Read review
Paper summary

The paper addresses the question of when a learned time-varying representation z=P(x)z = P(x) constitutes a valid dynamical state, in the sense that its present value suffices to determine its future evolution. In the exact setting, closure is formalized through semiconjugacy, meaning that any invertible reparameterization preserves closure under the conjugated flow, but preserving a prescribed mechanistic flow requires that the reparameterization commute with it. For the approximate setting, the authors decompose the latent prediction defect into an irreducible term attributable to the representation itself (the conditional variance of the future latent) and a residual transition-model error. From this they derive two finite-horizon guarantees. First, the current reconstruction error upper-bounds the degree of nonclosure, and second, when the representation fails to determine the future state, the future reconstruction error and the nonclosure cannot simultaneously vanish. The theory is validated on a 16-dimensional synthetic coupled system and on cerebral perfusion MRI, where the proposed PI-TR is compared against a PINN, Mech-ID, and an adapted CITRIS applied without interventions.

Critical strengths
  1. The question the paper poses is well chosen and clearly articulated. Distinguishing between a representation that merely reconstructs its inputs well and one that actually supports closed autonomous evolution addresses a genuine problem in scientific ML and physics-informed latent modeling, where these two properties are often conflated. The authors' decision to ground this distinction formally in semiconjugacy and lumpability theory, rather than relying on heuristic regularizers, is interesting.
  2. Theorems 2.4 and Remark 2.5 draw a conceptually clean and valuable distinction, i.e. preserving closure is always achievable through conjugation, whereas preserving a specific mechanistic flow additionally requires the reparameterization to commute with it. This separation offers a useful lens for anyone interpreting identified CRL latents as mechanistic states. That said, the underlying mathematics is fairly elementary, so the contribution here is more clarifying than deep.
  3. The biomedical testbed constitutes a genuine strength of the paper. Testing on 4D perfusion MRI with a physiologically motivated hidden state is a considerably more demanding evaluation than the rendered benchmarks that dominate temporal CRL work. The authors are also forthright about the method's limitations.
Critical weaknesses
  1. The paper's central diagnostic (Eq. 3.2) and bounds (Thm 3.3) are never actually used in practice. The main claim is that the framework separates representation-induced nonclosure from transition-model error, yet every experiment reports only the fitted RτR_\tau, the quantities EτE_\tau, Δτ\Delta_\tau, LτL_\tau and LDL_D are never estimated. This has three consequences. First, the trend observed in the synthetic experiments cannot be attributed to nonclosure rather than merely to a transition model that is harder to fit. Second, the explanation offered via Eq. (4.1) is only conditional, so it falls short of a demonstrated mechanism. Third, nothing in the paper establishes whether the derived bounds are tight or vacuous in practice, leaving the empirical relevance of the theory unclear.
  2. The theory does not match the paper's main real-world system. Def. 2.2 and Def. 3.1 are formulated for an autonomous flow with fixed parameters θ\theta, whereas the perfusion system departs from this setting. It is driven by an external input u(t)u(t) (Eq. D.2), θ\theta varies from patient to patient while a single transition GG is shared across them and the transition explicitly receives normalized time relative to the observed concentration peak making it non-autonomous. Moreover, the prescribed mechanistic flow acts on the head outputs rather than on ZZ itself. So, an invertible h:Z→Zh: Z \rightarrow Z does not actually correspond to the PI-TR architecture. As a result, the paper's central theoretical guarantees are not directly applicable to its own flagship experiment.
  3. The CRL framing promises more than the paper delivers. The abstract and Sec. 1 position the work as connecting temporal CRL with dynamical systems theory, yet this connection is not substantiated in the manuscript. No identifiability result from the CRL literature is used or proved, and the intersection HCRL∩HdynH_{CRL} \cap H_{dyn} (Remark 2.5) is never characterized for any concrete case, leaving the purported link between the two frameworks purely formal. Moreover, the CITRIS baseline omits intervention labels, intervention-specific latent partitions, and the intervention-target classifier, stripping away precisely the components that define CRL, which makes it essentially a temporal VAE. The framing therefore overstates the contribution relative to what is actually established.
  4. The perfusion evaluation has no held-out test set, and model selection and reporting are both performed on the same 7 patients. Checkpoint selection, the start-time and horizon choices, and the reported metrics are all derived from the same 7 validation patients. Furthermore, the 20 seeds share the data split and the pretrained initialization , so the ±\pm values quantify seed variability only and exclude variability across patients. This matters because most of the reported differences are far smaller than one standard deviation: RMSE 0.025±0.00030.025 \pm 0.0003 vs. 0.026±0.00190.026 \pm 0.0019, reconstruction 0.2370.237 versus 0.2380.238 and defects at Peak and Late roughly 0.24±0.0140.24 \pm 0.014 for both methods. Reading method rankings into differences of this size is not statistically justified.
  5. The theoretical novelty is limited relative to the classical and RL literature. Prop. 2.3 is acknowledged as classical, Thm 2.4 follows from two lines of algebra applied to conjugacy, and Eq. 3.2 is the standard conditional-mean decomposition, with (3.5) reducing to a triangle inequality. Closely related notions also appear in model-irrelevance abstractions and bisimulation in RL (Li, Walsh & Littman, 2006; Zhang et al., ICLR 2021), self-predictive representations (Schwarzer et al., 2021; Ni et al., ICLR 2024), which study exactly when the next latent is predictable from the current one, and the Mori–Zwanzig formalism, in which unresolved dynamics manifest as memory and noise terms. The paper should position itself explicitly against these lines of work and state precisely what is new which, as far as I can tell, is mainly (3.4) and its use as a selection criterion in physics-informed modeling.
  6. Table 2 favors PI-TR by construction and the result on the clinically relevant hidden state is set aside. PI-TR is trained with an ODE trajectory-fitting loss built on the same Euler integrator (with clipping) that also defines the evaluation metric whereas PINN is trained only with finite-difference residuals. So, the comparison is not between methods on equal footing. Moreover, PINN actually performs better on s2s_2, the unobserved quantity of clinical interest, for τ=2−7\tau=2-7. PI-TR's aggregate advantage comes almost entirely from s1s_1, the very component it is directly trained to fit. Remark 4.1 argues this away, but I believe the issue deserves direct discussion rather than a caveat.

Li et al., Towards a Unified Theory of State Abstraction for MDPs; Zhang et al., Learning Invariant Representations for Reinforcement Learning without Reconstruction; Schwarzer et al., Data-Efficient Reinforcement Learning with Self-Predictive Representations; Ni et al., Bridging State and History Representations: Understanding Self-Predictive RL;

Questions for authors
  1. Can you report an estimate of EτE_\tau (for example, a high-capacity regressor fitted after training to predict Z=Z^= from ZZ with PP frozen) next to RτR_\tau in the synthetic sweeps, including m = 16? Does the κ\kappa-trend persist in EτE_\tau, or is it mostly transition error?
  2. Can you estimate Δτ\Delta_\tau, LτL_\tau and LDL_D on the synthetic system and show both sides of (3.4) and (4.1)?
  3. What do patient-level cross-validation and a full start-time × horizon sweep show? Do any CITRIS vs PI-TR differences in Table 1 survive a paired test across patients?
  4. How do you reconcile the autonomous, fixed θ\theta assumptions of Sec. 2–3 with the forced, patient-specific, time-conditioned perfusion model? Which theorem, if any, applies to Sec. 4.2?
6 comments

Thank you for identifying the ambiguity in the original experiments. We clarify that, a larger fitted defect could mean that the representation loses information, or simply that its transition is harder to fit. We now measure these effects separately in Section 4.1 and evaluate both sides of the reconstruction bounds in Appendix F.2. Please also refer to our Global Response for an overview of the revisions. Below, we address your comments and questions point by point.

W1/Q1. Does the coupling trend persist after improving the transition?

Yes under compression. The full-dimensional result is different. For each of the 330 trained models, we froze the encoder PP, decoder DD, and original transition GG. On fixed pairs P(x)→P(xτ+)P(x)\to P(x_\tau^+), we trained (i) a residual correction G+HψG+H_\psi, updating only HψH_\psi, and (ii) an independent MLP from scratch. Validation selected the checkpoint and predictor, with the original GG also eligible. We evaluated both predictors on the same held-out pairs.

To see what this comparison isolates, write Tτ(G′)=E∥E[Zτ+∣Z]−G′(Z)∥2\mathcal T_\tau(G')=\mathbb E\|\mathbb E[Z_\tau^+\mid Z]-G'(Z)\|^2 for transition-model error, suppressing the fixed P,μP,\mu. By Eq. (3.2),

\mathcal R_\tau(G^{\rm opt})=\mathcal E_\tau+\mathcal T_\tau(G^{\rm opt}).$$ The same $\mathcal E_\tau$ appears in both. Thus $\mathcal R_\tau(G)-\mathcal R_\tau(G^{\rm opt})=\mathcal T_\tau(G)-\mathcal T_\tau(G^{\rm opt})$: the reduction isolates removable transition error. To estimate what remains, we also estimated $\mathcal E_\tau$ independently of either predictor. For $m<16$, we sampled pairs of initial states with the same encoding and evolved both. Half the expected squared distance between their future encodings equals the irreducible nonclosure. Each entry below shows $\kappa=0\to1$ **at fixed $m$** (five-seed means; normalized errors $\times 10^3$). **Table I. Original and refitted defects at $\kappa=0$ and $\kappa=1$ across six latent dimensions.** | $m$ | Original defect | Refitted defect | Direct nonclosure estimate | |---:|---:|---:|---:| | 2 | 3.27 → 95.07 | 2.09 → 94.63 | 2.07 → 95.08 | | 4 | 2.00 → 95.22 | 1.69 → 94.94 | 1.64 → 94.69 | | 6 | 1.68 → 93.05 | 1.47 → 92.63 | 1.42 → 91.78 | | 8 | 1.49 → 94.29 | 1.19 → 93.47 | 1.09 → 90.62 | | 12 | 2.02 → 36.32 | 1.34 → 35.20 | 1.08 → 32.62 | | 16 | 1.29 → 2.17 | 0.42 → 0.63 | Inverse check below | At $m=4$, refitting removes only 0.29% of the original defect at $\kappa=1$, and the independently estimated nonclosure tracks nearly the entire increase. At $m=16$, refitting removes 71% at $\kappa=1$. Here we also tested a predictor that uses only $z$ to recover a candidate initial state, evolve it, and encode its future. Its normalized test risk was below $5.3\times10^{-14}$ for every one of the 55 models. This is an achievable prediction risk, whose population value upper-bounds $\mathcal E_\tau$. Therefore, the evidence points to information loss under compression and transition approximation error at full dimension. `Figure 2` shows all 11 coupling values. `Appendices F.1–F.2` give the refit settings, conditional sampler, and larger-sample checks.

W1/Q2. Are the reconstruction bounds non-vacuous?

Yes, both original bounds can be informative simultaneously. Their usefulness differs across learned representations. We tested the upper bound in Eq. (3.4) and the lower bound in Eq. (4.2) (submitted Eq. (4.1)). Using the same conditional pairs, we estimate Δτ2\Delta_\tau^2 as half the mean squared distance between paired future observations. We bound LDL_D and LτL_\tau using the frozen weights and known synthetic flow. The lower bound becomes positive precisely when future-state uncertainty exceeds future reconstruction RMSE.

Table II. Estimated latent nonclosure and its lower and upper bounds at m=8m=8 for κ=0\kappa=0 and κ=1\kappa=1.

Fixed m=8m=8Δ^τ\hat\Delta_\tauFuture recon. RMSELDL_D boundLτL_\tau boundLower boundE^τ\hat{\mathcal E}_\tauScalar upper bound
κ=0\kappa=00.0990.1290.9558.2100.00110.424
κ=1\kappa=10.4840.1240.85539.570.02240.090610.06

The last three columns use the same normalized squared-error scale. This directly tests the explanation in Section 4.1: reconstruction stays near 0.13, but uncertainty rises from 0.099 to 0.484. The lower bound therefore goes from zero to about 25% of the measured nonclosure. Its estimated value is positive in 157 of 275 compressed models.

For the upper bound, the benchmark is the error of always predicting the mean future encoding, ignoring the input. The scalar bound falls below that benchmark in only 3 of 330 learned models; accounting for the direction of reconstruction errors raises this to 92 of 330. There is also a concrete case where both original scalar bounds are non-vacuous: for the controlled representation P(x)=bP(x)=b at κ=0.6\kappa=0.6, the lower bound, estimated nonclosure, upper bound, and constant-predictor risk are respectively 0.0269<0.0848<0.8044<1.06580.0269<0.0848<0.8044<1.0658 (raw squared units). The lower bound rules out zero nonclosure, and the upper bound improves on ignoring the input. Appendix F.2 and Table 8 give the calculations and distinguish this result from tightness across the learned models.

W4/Q3. What survives patient-level evaluation and the full start-time × horizon sweep?

The tracer RMSE advantage over CITRIS survives both tests. The original seven validation patients were used for checkpoint selection. In revision,

  1. We evaluated all 20 original checkpoint seeds on 32 separate ISLES 2017 test patients.
  2. We also retrained both methods in five folds on 43 distinct development patients, repeating pretraining within each training split and keeping checkpoint-selection patients separate from evaluation patients.
  3. The full sweep includes every valid start and horizon: 83,660 patient–start–horizon combinations on the test set and 102,005 in cross-validation. We average within patients so that longer scans do not receive greater weight.

Table III. Patient-averaged tracer RMSE and latent defect for CITRIS and PI-TR in held-out testing and five-fold cross-validation.

Patient-averaged outcome (lower is better)EvaluationCITRISPI-TR
Eight-frame tracer RMSE32 held-out patients0.034010.02394
Eight-frame tracer RMSEFive-fold CV0.041660.02898
Full-grid tracer RMSE32 held-out patients0.051340.02777
Full-grid tracer RMSE, excluding the six submitted settings32 held-out patients0.051200.02771
Full-grid latent defect (×103\times10^3)32 held-out patients0.09630.1460
Full-grid latent defect (×103\times10^3)Five-fold CV0.10460.2078

For the paired Table 1 comparison, PI-TR has lower eight-frame tracer RMSE in 27 of 32 test patients. PI-TR is also lower in 38 of 43 out-of-fold patients. For latent defect, CITRIS was better at both Early horizons. CITRIS also had lower endpoint reconstruction error at all four Peak/Late settings after adjustment. Therefore, the tracer advantage extends beyond the selected settings, while Table 1 does not establish a general PI-TR advantage in latent defect.

Appendix F.3 separates the submitted settings (Table 9), full sweep (Table 10), and cross-validation (Table 11).

W2/Q4. Which theoretical statement applies to perfusion?

The input curve u(t)u(t) is not part of our fitting process: For each patient, we calculate the entire input curve u(t)u(t) from the scan before fitting during preprocessing, and hold the patient parameters fixed during each rollout (Appendix E.2).

The learned predictor uses both the current latent state and time relative to the scan peak, so the error decomposition in Eq. (3.2) applies when we condition on both. Theorem 3.3 is tested on the autonomous synthetic system in Section 4.1. In perfusion (Section 4.2), the ODE governs the physiological outputs, and we evaluate how closely those outputs follow it.

W3. What does Theorem 2.4 add to a concrete CRL identification result?

New Corollary 2.6 in Section 2.2 gives an explicit case. On a fixed closed quotient, under the polynomial observation and autoencoder assumptions of Ahuja et al. (2023), exact reconstruction identifies coordinates up to an invertible affine map hh.

  • Every such hh preserves closure if the flow is transformed with it.
  • But if the mechanistic flow GtG_t is prescribed in the reported coordinates, the admissible maps are exactly those satisfying h∘Gt=Gt∘hh\circ G_t=G_t\circ h for every tt.

This identifies which part of the affine ambiguity remains after the mechanism is fixed. Appendix C states the inherited assumptions; Appendix D.2 proves the corollary. The perfusion CITRIS adaptation has no intervention labels and serves as a temporal variational baseline, not a test of interventional identification.

W5. What is distinct from state abstraction and self-prediction?

We agree that semiconjugacy and the conditional-mean decomposition are established. State abstraction and self-predictive RL likewise ask what a representation must retain to predict its future. Our finite-horizon question is more specific: for a given encoder and decoder, what does observation reconstruction imply about latent nonclosure? Theorem 3.3 distinguishes two cases: accurate current reconstruction upper-bounds nonclosure, while accurate future reconstruction can force nonclosure to be positive when states sharing the same encoding have different futures. The Q2 measurements above show when that distinction matters numerically.

W6. Does Table 2 establish better hidden-state estimation?

No. PI-TR is trained against the same clipped Euler rollout used for evaluation, while PINN is trained on finite-difference residuals. We checked whether the observed ordering depended on the selected windows: across all 511 eligible start–horizon cells on the 32 test patients, PI-TR had lower mean joint and observed-state (s1s_1) discrepancies, while PINN had lower mean hidden-state (s2s_2) discrepancy in every cell. The joint advantage therefore persists across windows and is driven by s1s_1.

Reviewer_WDsCOpenReview ↗
Rating
3
Confidence
2
Informativeness
2
Presentation
3
Soundness
2
Read review
Paper summary

This paper studies the dynamical admissibility of learned time-varying representations of physics-based models, which refers to whether a learned latent representation can solely determine its own future evolution (exact latent closure), rather than merely encoding its corresponding observation. Building on the classical notions of semiconjugacy and exact lumpability, the authors formulate when the exact latent closure of time-varying representations holds. Then, to quantify and analyze how far learned latents deviate from exact latent closure, they firstly decompose the latent prediction error into (1) representation-induced nonclosure and (2) transition-model error. Here, (1) representation-induced nonclosure means the error induced by information lost by the learned encoder which is needed to determine the future latent evolution and (2) transition-model error means the error induced by the imperfect learned transition model. Then, they provide theoretical results showing that accurate reconstruction can bound the representation-induced nonclosure but there is a tradeoff between future reconstruction error and nonclosure error when the current latent misses information needed to determine its future evolution.

To empirically analyze these theoretical results, the authors firstly train a temporal VAE on synthetic coupled ODE datasets and show that as the dynamic coupling strength increases, reconstruction error remains almost unchanged while fitted semiconjugacy defect increases, especially under dimensional compression, which indicates that accurate reconstruction cannot guarantee consistent latent predictions solely, so latent-transition consistency should be assessed separately from the reconstruction error. Additionally, the authors propose a physics-informed temporal representation model (PI-TR) which extends a temporal VAE with mechanistic consistency based on prescribed cerebral perfusion hemodynamics and conduct experiments on real cerebral perfusion MRI data with several baselines. The results show that, with these mechanistic constraints, PI-TR achieves the lowest trajectory estimation error and lower joint and relative mechanistic discrepancies than PINN while latent transition discrepancy is higher than CITRIS at early starts but slightly lower at peak and late starts. These results therefore indicate reconstruction accuracy, latent-transition consistency and mechanistic consistency capture different aspects of learned time-varying representations and thus should be evaluated separately.

Critical strengths
  1. Well-structured and clearly written paper : This paper is overall well-structured and clearly written with proper illustrations and background for mathematical notions. I think the logical flow of the paper is easy to follow, starting from the exact (idealized) setting built on classical notions of semiconjugacy and lumpability to formulate when the exact latent closure of learned representations holds. It then naturally extends to the approximate (realistic) setting to provide a way to quantify how far learned representations deviate from exact latent closure. Finally, theoretical results are provided with synthetic and real-world experiments. This logical progression makes the framework easy for readers to understand

  2. A theoretical framework for analyzing the dynamical admissibility of learned time-varying representations : The authors provide a theoretical framework to analyze whether a learned time-varying latent can determine its own future evolution solely beyond how well it can reconstruct observations. They characterize the exact latent closure in the context of learned representations and discuss its importance, and then mathematically formulate the conditions for exact latent closure. Also they provide an error decomposition, theoretical bounds to analyze the practical (approximate settings) deviation from the exact closure in learned representations and propose evaluating learned representations in three different aspects (reconstruction error, latent-transition consistency, mechanistic consistency) to assess their dynamical admissibility beyond reconstruction alone. Therefore, I think these formulation and findings provide a useful framework to evaluate the dynamical fitness of time-varying representations.

Critical weaknesses

Limited empirical validation of the theoretical results : Although the authors theoretically decompose the semiconjugacy defect into (1) representation-induced nonclosure and (2) transition-model error, it seems that all the experiments in the paper evaluate the overall fitted defect only, without separately estimating these components. Also, the unresolved future-state variation in the theoretical bounds and the Lipschitz constants are not estimated. So, in my opinion, it remains unclear that the empirical results truly validate the these theoretical results. Thus, it would be helpful to separately estimate representation-induced nonclosure (isolated from transition-model error) and empirically analyze the theoretical bounds, rather than simply illustrating changes of the overall latent-transition discrepancy. I believe addressing this would strengthen the connection between the theoretical contributions and the empirical results, and therefore reinforce the paper's main contributions.

Questions for authors
  1. Further justification of theoretical results : As discussed in the weakness section, could the authors separately estimate representation-induced nonclosure and empirically examine the theoretical bounds? I think it would be very helpful to support theoretical results directly and if this is addressed, I would be willing to positively reassess my evaluation.

  2. Clarification of contributions : I think the framework provided in the paper is overall plausible and interesting, however, it seems that several components of the framework are based on well-established concepts, rather than newly proposed. For example, the characterization of exact latent closure follows the classical notions of semiconjugacy and lumpability, as the authors acknowledge. Also, in my opinion, Theorem 2.4 and error decomposition formula seem to be relatively straightforward results. Of course, brining these concepts together in the context of learned time-varying representations is valuable, however, I think it would be more helpful to distinguish which theoretical results are genuinely new. Additionally, I think the positioning of PI-TR is somewhat unclear. PI-TR like the main proposed model, but it is not emphasized and briefly introduced in the main text. So, could the authors clarify whether PI-TR is intended as a methodological contribution, or simply as an illustrative model?

  3. Broader empirical validation including other datasets : I think the proposed framework does not seem to be restricted to cerebral perfusion domain and could be applied to other physics-based temporal dynamics models. Since the current real-world evaluation is only limited to a single dataset, could the authors evaluate the framework on additional datasets including broader baselines? I think further evaluation on additional datasets such as dynamical systems with high-dimensional observations and known latent states would be a interesting way to further enhance the paper's contributions.

4 comments

Thank you for your comments and questions. Please see our Global Response for an overview of the revisions. Below, we address your comments and questions point by point.

Q1. Can the two error components be estimated separately, and are the bounds informative?

Yes. In Section 4.1, we estimate nonclosure by asking a direct question: when two initial states have the same encoding today, how different are their encodings after they evolve? For each compressed encoder, we sample pairs of states with the same current encoding and evolve both to τ=0.5\tau=0.5. Half the average squared distance between their future encodings estimates irreducible nonclosure Eτ\mathcal E_\tau. This calculation uses the simulator and the frozen encoder alone.

We then evaluate the original and refitted predictors on the same pairs. By Eq. (3.2), subtracting estimated nonclosure from each predictor’s total squared error gives its estimated transition-model error. Table I shows all three quantities at fixed latent dimension m=4m=4 (five-seed means, normalized errors ×103\times 10^3).

Table I. Decomposition of original and refitted prediction errors into nonclosure and transition-model error at m=4m=4.

Coupling κ\kappaOriginal defectRefitted defectNonclosureOriginal transition errorRefitted transition error
01.9741.6701.6440.3300.025
0.520.14519.95119.9240.2210.027
194.96294.71294.6890.2730.023
  • As coupling increases, estimated nonclosure rises from 1.644 to 94.689, while the original transition error stays below 0.34 at all three settings. This directly attributes nearly the entire increase at m=4m=4 to information lost by the encoder.
  • At m=16m=16, the result changes: refitting removes 71% of the original defect at κ=1\kappa=1. A separate predictor that uses zz to recover a candidate initial state and then evolves it achieves normalized test risk below 5.3×10−145.3\times10^{-14} across all 55 full-dimensional models.

Figure 2 shows the full coupling sweep. Appendices F.1–F.2 give the refitting procedure, conditional sampler, and larger-sample checks.

For Theorem 3.3, we use the same pairs to estimate uncertainty in the future observations, Δτ\Delta_\tau, and bound the Lipschitz constants from the frozen network weights and known flow. At m=8m=8 and κ=1\kappa=1, future-state uncertainty is 0.484, while future reconstruction RMSE is only 0.124. With a decoder Lipschitz bound of 0.855, this gap gives a normalized lower bound of 0.0224, about 25% of the measured nonclosure (0.0906). The encoder–flow Lipschitz bound is 39.57, giving a loose scalar upper bound of 10.06 at this setting. Across the sweep, the estimated lower bound is positive for 157 of 275 compressed models. The scalar upper bound falls below the error of always predicting the mean future encoding in 3 of 330 models. Accounting for the directions in which reconstruction errors occur raises that count to 92 (Appendix F.2, Table 8).

Q2. Which contributions go beyond the established results?

We agree that exact lumpability, semiconjugacy, and the conditional-mean decomposition are established. The new Corollary 2.6 in Section 2.2 shows what Theorem 2.4 adds in a concrete identification setting. Under the assumptions of Ahuja et al. (2023), exact reconstruction on a fixed closed quotient identifies its coordinates up to an invertible affine transformation. We characterize exactly which of those transformations preserve the prescribed mechanistic equations: those that commute with the flow.

In the approximate setting, Theorem 3.3 connects errors that can be measured in different spaces. Accurate reconstruction of the current observation gives an upper bound on latent nonclosure. If states with the same current encoding have substantially different future observations, accurate reconstruction of those futures gives a lower bound on nonclosure. The experiment above measures how large that lower bound is for a learned encoder.

Q3. How far does the empirical validation extend?

We extended the evaluation within ISLES 2017 by testing the original models on 32 patients held out from model selection and repeating training in five patient-level folds on 43 development patients. Across all 83,660 valid start–horizon pairs on the held-out patients, mean tracer RMSE was 0.02777 for PI-TR, 0.03065 for Mech-ID, 0.05134 for CITRIS, and 0.26152 for PINN. CITRIS had a lower full-grid latent defect than PI-TR. Appendix F.3 reports results for the original settings (Table 9), the full sweep (Table 10), and cross-validation (Table 11).

AuthorsOpenReview ↗

Gentle Reminder

Dear Reviewer,

As the rebuttal period comes to an end, we hope to make the best use of the remaining time for further discussion. We have addressed all comments and concerns raised in your review with utmost care, and hope that our rebuttal and revisions have met your expectations.

We look forward to further discussion. Thank you again for your time and effort!

Thank you for the detailed response and the additional experiments. Most of my concerns are addressed, so I will keep my score.

Thank you for reviewing our response and additional experiments. We are glad that the revisions have addressed most of your concerns. Your suggestions helped us clarify the theoretical contribution and strengthen the empirical evaluation.

Thank you again for your time and constructive feedback!

Reviewer_FpauOpenReview ↗
Rating
2
Confidence
3
Informativeness
2
Presentation
2
Soundness
2
Read review
Paper summary

This paper investigates the dynamical admissibility of time-varying latent representations. The authors use semiconjugacy and exact lumpability to characterize latent dynamical closure, analyze the effects of reparameterization on prescribed dynamics, and derive theoretical bounds relating reconstruction error to irreducible nonclosure. The proposed framework is evaluated on synthetic dynamical systems and real-world cerebral perfusion data.

Critical strengths

The synthetic experimental design is interesting and novel. By controlling the dynamical coupling strength and the relative reconstruction importance of different state variables, the authors clearly illustrate the discrepancy between reconstruction fidelity and latent-transition consistency. These controlled experiments provide useful intuition about the limitations of reconstruction-based representation learning.

Critical weaknesses
  1. Limited theoretical novelty. I find the main theoretical results rather straightforward and do not think they reveal sufficiently deep or new insights. The first inequality (3.4) in Theorem 3.3 is essentially a standard error bound under a Lipschitz continuity assumption. Both the conclusion and its proof are fairly intuitive.

Similarly, the second inequality (3.5) mainly provides a quantitative formulation of a straightforward observation: if the future state can be accurately reconstructed from its latent representation, and the future latent state can be accurately predicted from the current latent state, then the current latent state should also be sufficient to predict the full future state. The contrapositive of this observation already captures the qualitative meaning of the inequality.

Overall, these results seem to formalize relatively intuitive observations using standard mathematical tools, rather than providing substantial new theoretical insights.

  1. Unclear connection to Causal Representation Learning. The paper repeatedly emphasizes its connection to CRL, but this connection is not sufficiently clarified. The main theoretical results appear to rely primarily on general properties of dynamical systems, without substantially using causal assumptions or identifiability conditions specific to CRL.

In particular, it is unclear how Theorem 2.4 goes beyond classical results on reparameterization in dynamical systems, or what essential role CRL plays in the proposed theoretical framework. This also makes the motivation and theoretical exposition somewhat difficult to follow.

Minor Concerns

  1. The experiments mainly demonstrate qualitative trends consistent with the theoretical analysis, but do not directly estimate the key theoretical quantities (\mathcal E_\tau) and (\Delta_\tau). Consequently, they do not sufficiently distinguish representation-induced nonclosure from transition-model error, nor do they evaluate the tightness of the proposed bounds.

  2. The ICLR 2026 paper Information Shapes Koopman Representation has already investigated the tradeoff between information preservation, compression, and expressiveness in dynamical representations from an information bottleneck perspective. A more detailed discussion of its relationship to the present work would help clarify the novel contributions of this paper.

Questions for authors
  1. Compared with classical results on lumpability and conjugacy, as well as existing work on dynamical representation learning, what fundamentally new theoretical insights do the proposed results provide?

  2. Could the authors directly estimate (\mathcal E_\tau) and (\Delta_\tau) in the synthetic experiments, thereby validating representation-induced nonclosure rather than only measuring the fitted transition defect?

From my experience with similar architectures, even when the encoder, decoder, and a Neural ODE can fit training trajectories very well, the learned latent dynamics may still suffer from substantial overfitting.

I believe that understanding the interplay between data availability, model capacity, and prior constraints is a particularly important direction. A more systematic theoretical and experimental investigation of these factors could provide more actionable insights into the practical challenges of dynamical representation learning.

3 comments

Thank you for highlighting the synthetic design. Your observation about Eq. (3.5) captures the qualitative argument. We agree that the proof uses standard tools. What the inequality adds is a quantitative question that we can now answer experimentally: given the measured reconstruction error, how much prediction error must remain even with the best possible latent transition? Please see our Global Response for an overview of the revisions. Below, we address your comments and questions point by point.

W1/Q1. What does the bound add to the qualitative argument?

Classical lumpability asks whether all states with the same encoding have the same encoded future. Theorem 3.3 also applies when their futures differ. In particular, if future-state uncertainty Δτ\Delta_\tau exceeds future reconstruction RMSE by d>0d>0, every predictor based on the current encoding incurs latent mean squared error of at least (d/LD)2(d/L_D)^2, where LDL_D bounds decoder sensitivity. This gives a measurable threshold and an error floor for a learned representation.

The distinction matters in the coupling sweep below.

Minor concern 1/Q2. Can we measure nonclosure and assess tightness directly?

Yes. In Section 4.1, we freeze each learned encoder and sample pairs of initial states with the same encoding. Under independent conditional sampling, half the expected squared distance between their future encodings equals Eτ\mathcal E_\tau. The same calculation on future observations gives Δτ2\Delta_\tau^2. Separately, we refit the transition on frozen encodings and evaluate it on held-out pairs. This gives two independent ways to examine the coupling trend.

At fixed m=8m=8 and τ=0.5\tau=0.5, the five-seed means are:

Table I. Estimated nonclosure, refitted defect, and reconstruction bounds across coupling strengths at m=8m=8 and τ=0.5\tau=0.5 (five-seed means).

Coupling κ\kappaΔ^τ\hat\Delta_\tauFuture recon. RMSELower boundE^τ\hat{\mathcal E}_\tauRefitted defectScalar upper bound
00.0990.12900.00110.00120.424
0.50.2530.1330.00250.02080.02154.14
10.4840.1240.02240.09060.093510.06

The last four columns share the same normalized squared-error scale. Future reconstruction RMSE stays near 0.13 as coupling increases, but states with the same encoding develop increasingly different futures. At κ=1\kappa=1, the lower bound accounts for about 25% of the measured nonclosure, and the direct estimate accounts for nearly all of the refitted defect. The scalar upper bound is loose here. Across the sweep, the estimated lower bound is positive for 157 of 275 compressed models. The scalar upper bound improves on always predicting the mean future encoding in 3 of 330 models, rising to 92 of 330 when the bound accounts for the direction of reconstruction errors.

Figure 2 shows all six latent dimensions. Appendix F.2 gives the sampler, larger-sample checks, and bound calculations.

W2/Q1. Where does CRL enter Theorem 2.4?

The coordinate-change argument in Theorem 2.4 is classical. New Corollary 2.6 in Section 2.2 applies it to a concrete identification result. On a fixed closed quotient, the assumptions of Ahuja et al. (2023) identify latent coordinates up to an invertible affine map hh. All such maps preserve closure under the transformed flow. If the mechanistic flow GtG_t is prescribed in the reported coordinates, the admissible maps are exactly those satisfying h∘Gt=Gt∘hh\circ G_t=G_t\circ h for every tt. For example, swapping two coordinates with distinct decay rates preserves predictability, but assigns each variable the wrong rate if the original equations are kept fixed.

The CRL result supplies the affine ambiguity. The dynamical condition determines which part of that ambiguity the prescribed mechanism permits.

(Minor) Data, capacity, and overfitting.

Your concern about fitting training trajectories motivated a separate check with up to 262,144 training pairs, wider predictors, validation-only selection, and a fresh test set. At κ=1\kappa=1, the expanded refit reduced held-out defect by only 0.31% at m=4m=4, compared with 87.79% at m=16m=16 (five seeds). Additional predictor capacity therefore makes a substantial difference when the encoding retains the state. Under compression, the independently measured nonclosure explains why additional capacity yields little improvement. Section 4.1 also varies the temporal penalty at fixed dimension and coupling to examine the effect of the training constraint.

(Minor) Relation to Information Shapes Koopman Representation.

Cheng et al. (ICLR 2026) study how to learn finite-dimensional Koopman coordinates that support simple linear evolution while retaining diverse modes. Our question concerns a given encoder and allows nonlinear latent evolution: when two physical states receive the same encoding, how different can their future encodings be? The distinction is visible even for a constant encoder. It has zero nonclosure because every future encoding is identical, yet it discards the entire physical state. Reconstruction and future-state uncertainty expose that loss. This connects the expressiveness concern in their work directly to why we evaluate reconstruction and closure together.

AuthorsOpenReview ↗

Gentle Reminder

Dear Reviewer,

As the rebuttal period comes to an end, we hope to make the best use of the remaining time for further discussion. We have addressed all comments and concerns raised in your review with utmost care, and hope that our rebuttal and revisions have met your expectations.

We look forward to further discussion. Thank you again for your time and effort!

Reviewer_ZgwVOpenReview ↗
Rating
2
Confidence
2
Informativeness
2
Presentation
2
Soundness
2
Read review
Paper summary

The paper focuses on time varying representations for physics based models with dynamically evolving and admissible latent states. The work aims to address the gap in prior formulations based on trajectory, time invariant parameters. It uses semiconjugacy and lumpability to model these properties and distinguishes representation based nonclosure and transition model error.

Critical strengths

Problems are well formulated and presented cleanly. Distinguishing preservations of closure (with conjugate dynamics) and a prescribed flow, showing the divergence of reconstruction and latent transitional accuracy is useful and well motivated. The synthetic and real world perfusion evaluation are complementary and generally well formulated.

Critical weaknesses
  1. As the central theoretical contribution of the paper, which is to separate representation induced nonclosure and error from model evolution, equation 3.2 clearly separates these targets. But the synthetic and cerebral perfusion experiments are based on the fitted semiconjugacy defect R_τ that contains nonclosure (E_τ) and the transition error. Could the authors separately evaluate/discuss the components?
  2. The real world experiments need some caution in interpretation. The perfusion evaluation seems to have a 37 and 7 training and validation split for model selection and analysis, no separate test set is described. The authors also state “selected scenarios from exploratory start horizon sweep”, hence the robustness outside of the selected settings are unclear.
  3. Regarding mechanistic consistency (Table 2), PITR includes ODE trajectory fitting term which is absent from PINN baseline; the effect of this additional objective to the mechanistic results is unclear.
Questions for authors
  1. Could the authors discuss more extensive sweep and evaluations on cases unused for the model selection in the real world experiment (or perhaps with patient level CV)?
  2. In table 1, given very similar reconstruction errors for CITRIS and PITR; CITRIS has lower latent transition error at Early, and PITR at Peak and Late. Could authors discuss how consistent the differences are across the seeds (and patients) and perhaps provide CI for the delta between models?
5 comments

Thank you for your questions. The six settings in Table 1 came from an exploratory sweep on the seven validation patients used for checkpoint selection. We therefore repeated the comparison on 32 held-out patients across all valid start times and horizons. PI-TR retained lower tracer RMSE than CITRIS on average across the full grid. For latent defect, CITRIS had a consistent advantage at Early, while the smaller Peak and Late differences varied across patients and seeds. Please see our Global Response for an overview of the revisions. Below, we address your comments and questions point by point.

W2/Q1. Do the results extend beyond the selected settings and validation patients?

We evaluated the original 20 checkpoints per method on 32 held-out ISLES 2017 patients. Each scan contributed every valid start time and forecast horizon, giving 83,660 patient–start–horizon combinations. We averaged within each patient first so that longer scans did not receive greater weight. We also repeated training in five folds on 43 distinct development patients, with separate patients for training, checkpoint selection, and evaluation in each fold.

Table I. Paired comparisons of CITRIS and PI-TR on cerebral perfusion.

Patient-averaged outcome (lower is better)CITRISPI-TR
Tracer RMSE, full grid on 32 test patients0.051340.02777
Tracer RMSE, same grid excluding the six submitted settings0.051200.02771
Eight-frame tracer RMSE, five-fold cross-validation0.041660.02898
Latent defect × 1,000, full grid on 32 test patients0.09630.1460

Removing all six submitted settings barely changes the tracer comparison. For latent defect, averaging all horizons at each named start reproduces the original phase pattern on the test patients: CITRIS has lower mean defect at Early, and PI-TR has slightly lower means at Peak and Late. Across all starts and horizons, CITRIS has lower defect. Appendix F.3 reports the original settings (Table 9), full sweep (Table 10), and cross-validation (Table 11).

Q2. How consistent are the Table 1 differences across seeds and patients?

Below, the difference is PI-TR minus CITRIS on the 32 test patients. The intervals are paired 95% patient-bootstrap intervals after averaging the 20 seeds within each patient. The seed column compares methods after averaging patients within each matched seed. SH and LH denote 9 and 11 frames.

Table II. Paired differences in latent defect (PI-TR minus CITRIS) at the six original forecast settings.

SettingLatent defect difference × 1,000 [95% CI]Seeds with lower PI-TR defectPatients with lower PI-TR defect
Early-SH+0.2450 [+0.2061, +0.2861]0/200/32
Early-LH+0.2158 [+0.1847, +0.2474]0/200/32
Peak-SH+0.0043 [−0.0172, +0.0277]12/2018/32
Peak-LH−0.0129 [−0.0362, +0.0118]12/2021/32
Late-SH0.0000 [−0.0219, +0.0232]12/2020/32
Late-LH−0.0181 [−0.0411, +0.0063]12/2020/32

The Early advantage for CITRIS holds for every seed and patient. At Peak and Late, all four intervals include zero, so the small mean differences do not establish a reliable winner. Endpoint reconstruction differences were also small: the two Early intervals included zero, while all four Peak/Late comparisons favored CITRIS after correction for the 13 Table 1 comparisons. The clearest PI-TR advantage was eight-frame tracer RMSE: it was lower in all 20 matched seeds and 27 of 32 test patients (paired difference −0.01008, 95% CI [−0.01344, −0.00625]).

W1. Can the two components of the fitted defect be separated?

In Section 4.1, we froze each synthetic encoder and refit only the transition. We also estimated nonclosure independently by evolving pairs of states with the same current encoding and measuring how far apart their future encodings became. At fixed m=4m=4, increasing κ\kappa from 0 to 1 raised estimated nonclosure from 0.00164 to 0.09469, closely tracking the refitted defect (0.00169 to 0.09494). At m=16,κ=1m=16,\kappa=1, refitting alone removed 71% of the original defect. Figure 2 and Appendices F.1–F.2 show the full sweep and estimation procedure.

For perfusion, we can measure how much error a better transition removes. In an exploratory refit on three matched seeds with encoders and decoders frozen, full-grid test defect (× 1,000) fell from 0.083 to 0.042 for CITRIS and from 0.154 to 0.065 for PI-TR. These reductions measure removable transition error.

W3. How does the ODE training objective affect Table 2?

PI-TR is trained to match trajectories from the same ODE integrator used in Table 2, while PINN is trained on finite-difference residuals. On the 32 test patients, PI-TR had lower mean discrepancy for the observed tracer state and the joint trajectory at all 511 eligible start–horizon settings. PINN had lower discrepancy for the hidden state at every setting. The joint advantage is therefore driven by the observed tracer. It supports better agreement with the prescribed rollout under PI-TR’s training objective.

AuthorsOpenReview ↗

Gentle Reminder

Dear Reviewer,

As the rebuttal period comes to an end, we hope to make the best use of the remaining time for further discussion. We have addressed all comments and concerns raised in your review with utmost care, and hope that our rebuttal and revisions have met your expectations.

We look forward to further discussion. Thank you again for your time and effort!

Comments on the paper

2 comments
AuthorsOpenReview ↗

Global Response

Dear Reviewers,

Thank you for the time and care you put into reviewing our paper. Several of you raised the same question about the original experiments: 1) when the fitted transition defect increases, 2) how much of that increase comes from information lost by the representation, and 3) how much comes from the transition predictor?

In response, we designed new experiments to measure both components, added a concrete application of the theory to an identification result, and expanded the perfusion evaluation to held-out patients.

We have marked the revisions in blue in the updated manuscript, and we appreciate the reviewers’ suggestions, particularly the request to measure the quantities appearing in the theory directly. We summarize our major changes as below.

Major Revisions

  • New Corollary 2.6 in Section 2.2. To address concerns about the connection to CRL, we give a concrete example of what Theorem 2.4 adds to a CRL identification result. On a fixed closed quotient, the assumptions of Ahuja et al. (2023) identify latent coordinates up to an invertible affine map. Every such map preserves closure under the corresponding change of dynamics. Exactly those that commute with the prescribed flow also preserve the original mechanistic equations. [bevS, Fpau, WDsC]

  • Updated Figure 2 and new Appendices F.1–F.2. To determine whether the coupling trend reflects information lost under compression or underfitting of the transition predictor, we measure both sources of error. Across 330 synthetic models, we refit the transition with the encoder frozen and independently estimate nonclosure under compression. At fixed m=4m=4, increasing coupling from 0 to 1 raises estimated nonclosure from 0.00164 to 0.09469, closely tracking the refitted defect (0.00169 to 0.09494). At m=16m=16, refitting removes 71% of the original defect at the strongest coupling. [bevS, Fpau, WDsC, ZgwV]

  • New bound evaluations in Appendix F.2 and Table 8. To assess when the bounds in Theorem 3.3 are informative, we evaluate them on the learned representations. We estimate future-state uncertainty and compute Lipschitz bounds from the frozen networks and known synthetic flow. The estimated lower bound is positive for 157 of 275 compressed models. The scalar upper bound improves on always predicting the mean future encoding in 3 of 330 models. Accounting for the direction of reconstruction errors raises that count to 92. We also exhibit a controlled case in which both original scalar bounds are informative simultaneously. [bevS, Fpau, WDsC]

  • New patient-level evaluation in Appendix F.3, Tables 9–11. To test whether the perfusion findings extend to new patients and depend on the forecast window, we expanded the evaluation across patients, start times, and horizons. We evaluated the original checkpoints on 32 held-out ISLES 2017 patients, repeated training in five patient-level folds, and swept every valid start time and horizon. On the held-out full grid, PI-TR achieved the lowest tracer RMSE among the four methods (0.02777), while CITRIS had the lower latent defect. For the original eight-frame tracer outcome, PI-TR improved on CITRIS in 27 of 32 test patients and 38 of 43 out-of-fold patients. [bevS, ZgwV, WDsC]

Additional Clarifications

  • On what is identified. Appendices C and D.2 state the identification assumptions and prove the new corollary. Section 2.2 explains separately what follows from coordinate identification and what follows from prescribing a mechanistic flow.
  • On the reliability of the nonclosure estimates. Appendix F.1 documents predictor selection and sensitivity to training-set size and network capacity. Appendix F.2 gives the conditional sampling procedure and its larger-sample checks.
  • On sensitivity to the forecast window. Appendix F.3 distinguishes the six settings reported in the original Table 1 from the full start-time and horizon sweep, and reports which conclusions persist across patients.

We appreciate the reviewers’ suggestions, particularly the request to measure the quantities appearing in the theory directly. We address the individual questions in the corresponding review threads.

Dear Reviewers, the authors’ responses are now available. If you haven’t already, please review them and share your comments before the discussion period closes. Thank you for your time and effort.

From OpenReview, CC BY 4.0. Authors as signed there; shown unedited.

Related papers

Loading the map…

Discussion (0)

Sign in to comment.