acceptodds
Under review as a conference paper at ICLR 2027

Structure and Limits of Composable Coarse-to-Fine Generative Models

Abstract

We study coarse-to-fine generative models that combine deterministic coarse evolution with stochastic reconstruction. We characterize the temporal laws of Markov rollouts with transition kernels that compose exactly at all positive times. Under Lipschitz regularity of the restriction, quadratic-Wasserstein Lipschitz regularity of the decoder, and controlled coarse-flow regularity, every positive-time joint law is a mixture of independent emissions along a deterministic flow of persistent classes. This representation does not require exact consistency between reconstructed samples and their prescribed coarse states. It implies that mean-square continuity forces the visited class laws to be point masses. Within the same exactly composable setting, we bound the path-law distance to independent reconstruction under approximate fiber consistency, with dependence on decoder sensitivity and fiber error. Explicit constructions show that weaker decoder regularity permits stochastic class evolution and that increasing decoder sensitivity can preserve temporal dependence as the fiber error vanishes. Controlled temporal experiments separate identical marginals from path statistics, and a small learned reconstruction study shows why improved temporal correlation alone is insufficient.

Primary area:
generative models
Keywords:
coarse-to-fine generative models, Markov semigroups, temporal consistency, stochastic reconstruction, scientific machine learning

Reviews

Among:
Shape:
1234this paper, Top 17%
Figure 1. Mean review rating of the reviewed papers at ICLR 2027. This paper averages 2.75 over 4 reviews, above 83% of the others.
Reviewer_DR7COpenReview ↗
Rating
1
Confidence
2
Informativeness
1
Presentation
1
Soundness
1
Read review
Paper summary

This work studies coarse-to-fine generative models that repeatedly restrict a fine-scale state to a coarse one, evolve it deterministically at the coarse level, and resample the fine scale. The main claimed result is that, if the time steps compose exactly, the generated states at different times are independent given the initial "class", so all temporal dependence is determined by this class.

Critical strengths

The main results are stated with explicit assumptions and full proofs. However, due to the poor presentation, I could not reliably assess the significance and novelty of these results.

Critical weaknesses

The most critical problem of the paper is that it is badly structured and hard to follow.

  • The paper does not clearly state the motivation, contributions and importance of the work. Even the Abstract, Introduction and Conclusion do not provide this information; they rather list theoretical facts. These points are important to clearly understand the impact of the work.

  • The problem is not well motivated. The four works cited in the Introduction (Mardani et al., 2025; Fan et al., 2025; Wang et al., 2022; Li et al., 2021) do not consider the problem studied in the paper. Since the paper does not address an already established problem, it should clearly explain why this problem is important and how it differs from other approaches.

  • Sections 2-5 are technically complex and hard to read. They should be simplified and supplemented with useful commentary.

  • Section 6 does not clearly explain why these experiments are needed. Moreover, it uses but does not explain the term "fixed coarse context", which conveys the essence of Figure 2, the only experimental figure in the main text. Figure 2 also uses Var(Jn)\mathrm{Var}(J_n), which is not introduced in the main text.

Due to its very poor presentation, the paper is almost unreadable and hard to judge. I believe it needs significant revision.

Questions for authors

Please address the weaknesses above.

2 comments
AuthorsOpenReview ↗

Response to Reviewer DR7C

Thank you for your review.

  1. We agree that Sections 2–5 are highly technical. We were aware of this when preparing the submission and tried to make the paper as accessible as possible. We keep the main results in the main text so that the logical development stays complete, and we move the proofs to the appendices. In addition, (i) the Introduction states the central result in words before the technical sections, (ii) terms are explained in words where they first appear, and (iii) Table 1 summarizes the operators and probability laws, and Figure 1 illustrates Theorem 1. The results rely on measure-theoretic tools, so some passages remain dense. If any passage is still hard to follow, we would be happy to provide more explanation.

  2. Thank you for pointing out that the motivation was not clearly stated. Three of the four works cited in the Introduction use the structure we study, a deterministic coarse evolution with fine detail sampled from the coarse state at each time, so the question we ask arises in practice. Mardani et al. (2025) also leave the temporal coherence of the generated fine scales to future work. That said, we agree that the paper should make these points clearer. In the revision, the first paragraph of the Introduction strengthens the motivation and states the research question, and the Introduction ends with a list of contributions, as you suggested.

  3. At the start of Section 6, we added an explanation of why the experiments are needed. We also now define "fixed coarse context" and Var(Jn)\mathrm{Var}(J_n) in the main text. If any other terms need clarification, please let us know.

Reviewer_DR7COpenReview ↗

Follow-up on the authors’ response

Thank you for clarifying the introduction and stating the contributions explicitly. The research question is now easier to understand, and its motivation is clearer. I also appreciate the clearer explanation of Figure 2.

However, the abstract and Sections 2–5 remain unchanged, and I still find them difficult to follow. This makes it hard for me to assess the significance of the paper’s contributions. I recognize that my own familiarity with the subject may play a role, and I ask the area chair to take this uncertainty into account when weighing my review.

Overall, presentation remains my main concern. Given that the paper’s contribution is primarily theoretical, I consider the difficulty of interpreting its main results a substantial weakness. For this reason, I am keeping my original rating.

Reviewer_BjUzOpenReview ↗
Rating
3
Confidence
2
Informativeness
3
Presentation
3
Soundness
2
Read review
Paper summary

This paper studies coarse-to-fine generative models that combine deterministic coarse evolution with stochastic fine-scale reconstruction. It proves that, under exact Chapman--Kolmogorov composition at all positive times and suitable regularity assumptions, every positive-time joint law is a mixture of conditionally independent reconstructions along a deterministic evolution of persistent classes. The paper further bounds the path-law distance to independent reconstruction under approximate fiber consistency, identifies the product of decoder sensitivity and fiber-recovery error as the key quantity, and uses controlled and PDE-reconstruction experiments to show why matching one-time marginals or coarse recovery alone does not ensure temporal consistency.

Critical strengths

1.The paper is clearly written and states its setting, assumptions, and scope of its conclusions carefully. 2.It addresses an important but often overlooked issue in coarse-to-fine generation: temporal joint distributions, in addition to one-time marginals and coarse constraints. 3.The structural classification, the quantitative approximate-fiber result, and the boundary constructions form a coherent theoretical contribution.

Critical weaknesses

1.The main conclusions rely on strong idealized assumptions: exact Chapman--Kolmogorov composition for every initial state and all positive times, deterministic coarse evolution, a time-invariant decoder, and global Lipschitz-type controls. These assumptions are generally difficult to verify or enforce in neural generative models. More importantly, the paper itself shows that small local composition discrepancies can accumulate under repeated reconstruction, but provides no quantitative result connecting approximate composition to path-law error. Hence, the practical reach of the theory remains unclear.

2.The practical interpretation of Theorem 5 depends on the product of decoder sensitivity and fiber-recovery error tending to zero, not on small fiber error alone. Proposition 9 shows precisely that temporal dependence can persist as fiber error vanishes when decoder sensitivity grows. While the experiments report coarse/fiber recovery, they do not provide an operational estimate or diagnostic for decoder sensitivity, or for this product. It is therefore difficult to tell when a learned model is actually in the regime close to independent reconstruction.

3.The experiments are primarily mechanistic illustrations and do not directly test the theoretical assumptions in a target learned model. The controlled advection and rotation experiments validate deliberately constructed phenomena. The learned Burgers experiment is driven by a supplied reference coarse trajectory, and its history-based model falls outside the memoryless architecture analyzed by the theory; the paper explicitly notes that it does not test exact composition. The larger PDE experiments mainly compare coarse constraints, endpoint error, and stability. A learned model that freely rolls out according to Q_{t} = G(\phi_{t}(Ru)) while measuring composition defects and path statistics is still missing.

Questions for authors

1.For practical neural reconstruction models, how do the authors recommend diagnosing or controlling departures from exact composition? Can the authors provide a bound, or at least a reproducible diagnostic protocol, relating time step size, local Chapman--Kolmogorov defects, coarse-flow amplification, and decoder sensitivity?

2.The relevant quantity in Theorem 5 is LGϵL_{G}\epsilon. For the learned PDE models, can the authors report or estimate a quantity related to LGL_{G} (e.g., local input-to-output/Wasserstein sensitivity) and assess whether it, together with fiber error, predicts temporal correlations or path-statistic errors?

3.Can the authors add a more direct learned example of the analyzed architecture: a model that freely rolls out without fine-scale history beyond the coarse trajectory, while reporting composition defects, fiber error, one-time scores, and conditional temporal correlation/increments/time-average uncertainty? This would more directly substantiate the proposed evaluation guidance.

2 comments
AuthorsOpenReview ↗

Response to Reviewer BjUz

We thank the reviewer for the careful reading and for the concrete suggestions on how to test the theory in a learned model. We read each weakness and the question with the same number as addressing the same issue, so we reply to them in pairs. We start with W3/Q3, since the experiment we added in response to W3/Q3 also underlies our answers to the other two pairs.

  1. (W3, Q3) We agree that this example is valuable and have added it as Appendix G.5. Learned decoders now roll out freely as Uk+1∼G(ϕh(RUk))U_{k+1}\sim G(\phi_h(RU_k)), with a coarse solver for ϕ\phi and no fine-scale history, and we report all the quantities the reviewer listed (Tables 5 and 6, Figure 6). Two decoders are memoryless, as in Eq. (1): an exact-fiber decoder, and an unconstrained decoder whose coarse recovery error is tuned by a partial projection. A memory decoder is included for comparison. The results agree with the theory. The exact-fiber decoder and the full projection cannot be distinguished from independent reconstruction, as expected under exact fiber consistency. All memoryless decoders reach lag-one correlation 0.63–0.67, against 0.998 for the true trajectories, whereas the memory decoder reaches 0.993. Since lag-one correlation measures how similar consecutive generated fields are, this shows that without fine-scale history the generated fine details lose most of the temporal coherence of the true trajectories.

  2. (W1, Q1) In short, exact composition can be checked numerically but is hard to prove. On the assumptions, we agree that they are strong, but they are not artificial. The applications cited in our introduction evolve the coarse state deterministically and generate fine detail with a fixed decoder (e.g., Fan et al., 2025; Li et al., 2021; Mardani et al., 2025), which is the structure we analyze. The strongest assumption is exact composition, and it can be tested numerically on a trained model. Because the decoder is random, we draw many samples of Qs+t(u)Q_{s+t}(u) and of QsQt(u)Q_sQ_t(u) from the same states and check whether their distributions differ by more than sampling noise. This can be costly, but it is feasible. For example, the newly added Appendix G.5 applies this test to learned decoders and detects composition defects, that is, cases where composition fails, when the coarse recovery error is large. Such a test only covers finitely many states and step sizes, so it can reveal a violation but cannot certify exact composition everywhere. As the reviewer notes, proving exact composition for a neural model remains hard. On approximate composition, we do not have a bound linking it to path-law error. We agree that this is an interesting but difficult problem, and Section 7 lists it as open.

  3. (W2, Q2) Appendix G.5 now estimates LGL_G for the learned decoders. Their laws are Gaussian, so W2W_2 between two of them has a closed form. At each of 256 coarse states visited by the rollout, we compute how fast W2(G(c),G(c′))W_2(G(c),G(c')) grows as c′c' moves away from cc, and take the 95th percentile of these values as LGL_G. This describes how sensitive the decoder is at almost all visited states, without letting one unusual state set the value, and it differs from the maximum by less than 15%. For the exact-fiber decoder and all five projection settings of the unconstrained decoder, LGL_G stays between 2.7 and 3.0. The coarse recovery error ε\varepsilon is 0.098, 0.046, 0.0088, and 0.00088 for the four partial projections, and at single-precision rounding level for the exact-fiber decoder and the full projection. As LGεL_G\varepsilon decreases, the distance between the free rollout and independent reconstruction decreases with it (Figure 6, left). Thus LGεL_G\varepsilon predicts how far the path law departs from independent reconstruction. It does not predict temporal correlation or errors relative to the true trajectories: across all values of LGεL_G\varepsilon, the lag-one correlation stays between 0.63 and 0.66, against 0.998 for the true trajectories.

Thank you for your response. I have no further questions and will keep my score unchanged.

Reviewer_atoWOpenReview ↗
Rating
4
Confidence
2
Informativeness
2
Presentation
3
Soundness
3
Read review
Paper summary

The paper discusses the identifiability problem where given known markovian dynamics but access only to coarse observations is it feasible to reconstruct the original true dynamics under regularity conditions. It is primarily a theoretical work and showcases simulations as empirical proofs.

Critical strengths
  1. The resolution problem is quite central to several applications and having bounds of the sort presented in this work are a welcome step.
  2. The conditions are well reasoned and the central approach to the proof is consistent with prior literature on this topic.
Critical weaknesses
  1. In general with coarse reconstructions a standard approach is to use iterated discretization in such a scenario it is important to know if the markov semigroup transfers across scales and if the bounds hold even then as a starting point the classic physical setting of a RG flow would be interesting to explore in this context.

  2. While not directly relevant , a few experiments on a PINN or a NN based PDE solver would be very welcome since such learned proxies are often used in such scenarios where little fine information is present.

Questions for authors

I do not have any significant questions from the authors.

2 comments
AuthorsOpenReview ↗

Response to Reviewer atoW

We thank the reviewer for the positive assessment and the suggestions.

  1. We read this point as asking whether our results still apply when the evolved coarse state is refined through several intermediate resolutions, C=X0→X1→⋯→XK=XC = X_0 \to X_1 \to \cdots \to X_K = X, with a stochastic decoder GkG_k at each level. The answer is yes. The composed kernel G=GK∘⋯∘G1G = G_K \circ \cdots \circ G_1 is again a probability kernel from CC to P2(X)\mathcal{P}_2(X), so the transition keeps the form Qt(u)=G(ϕt(Ru))Q_t(u) = G(\phi_t(Ru)) of Eq. (1). If each level is W2W_2-Lipschitz with constant LkL_k, gluing optimal couplings level by level shows that GG is W2W_2-Lipschitz with LG≤LK⋯L1L_G \le L_K \cdots L_1. Since RR and ϕt\phi_t are unchanged, Theorem 1 applies whenever the rollout composes exactly as in Eq. (2). The connection to RG flow is an interesting direction for future work.

  2. We agree that learned proxies such as PINNs or neural PDE solvers are often used for the coarse evolution when little fine-scale data is available. Our experiments already use learned neural decoders, while the coarse evolution is a numerical solver. A learned coarse solver is a deterministic map and fits Eq. (1), but it is usually trained at a fixed step, so the rollout is defined only at multiples of that step and falls short of Eq. (2) at all positive times. This places it in the approximate-composition setting that we identify as open in Section 7. Such experiments would be valuable, and we leave a careful study to future work, as it is beyond what we can complete within the discussion period.

Thank you for your response! I have no further questions at the moment and am inclined to keep my rating !

Reviewer_rWjDOpenReview ↗
Rating
3
Confidence
2
Informativeness
2
Presentation
2
Soundness
3
Read review
Paper summary

The paper studies a specific class of generative models that simulate temporal dynamics at a coarse scale and use a stochastic decoder to reconstruct the finer details. The main finding is that, under certain assumptions, the generated states at different time steps are independent and only controlled by a fixed initial label. This limits the models' ability to capture realistic temporal dynamics, as the generated finer-scale information cannot influence future predictions. The authors also run controlled experiments to showcase how these limitations translate to model performance in practice.

Critical strengths
  • The paper provides a rigorous theoretical analysis of why the studied coarse-to-fine generative models cannot properly capture temporal dependencies at the finer scale. Theorem 1 is the central result supporting this claim. It shows that, under specific assumptions about the fine-to-coarse and coarse-to-fine (reconstruction) operators, the generated fine-scale states are conditionally independent given an initial class. This means that the fine-scale information is "hallucinated" from the initial class and does not influence future predictions. Overall, this is an interesting theoretical result that could have implications across several domains where generative models are used to simulate temporal dynamics, from video generation to weather modeling.

  • The authors support their theoretical findings with extensive experiments on PDE dynamics. In a controlled experiment, they show that independently reconstructing the fine-scale state produces the correct distributions at each time step, but fails to capture temporal correlations and the overall evolution of the trajectory. In further experiments, the authors use trained decoders to demonstrate that independent reconstructions produce significantly worse temporal consistency than a model that also utilizes previously generated fine-scale states. These results help illustrate the practical consequences of the theoretical findings.

Critical weaknesses
  • The paper studies generative models that simulate dynamics at the coarse scale and use a memoryless decoder to generate the states at the fine scale. However, it is unclear how representative this architecture is of models used for these simulation tasks. Even when not explicitly conditioning on previous fine-scale states, the models often use additional conditioning information to guide both coarse state predictions and fine reconstructions. The paper would benefit from a discussion of how the studied architecture relates to practical modeling and, more importantly, why researchers choose this two-stage approach. Is modeling dynamics only at the coarse scale necessary due to computational constraints, or are there other motivations for this design, and what is the trade-off to be made?

  • One of the central findings of the paper is that memory-less reconstruction cannot properly capture fine-scale temporal dynamics. The authors compare independently reconstructing at each time step with a memory-based model in the Burgers experiment, where the memory-based model is shown to significantly improve temporal correlations, consistent with the theoretical findings. However, in the larger experiments on (Allen–Cahn, Burgers, Navier–Stokes), the authors only evaluate reconstruction models without memory. These experiments show the reconstruction accuracy and numerical stability, rather than whether generated trajectories capture the correct temporal dynamics. Extending these experiments with a model that explicitly incorporates fine-scale memory and improves the temporal dynamics could help further ground the findings in empirical results.

Questions for authors
  • One of the results in the paper shows that increasing the sensitivity of the reconstruction (decoder) can preserve temporal dependencies at the finer scale. However, how practical is such a result, and would this ultimately make the model more sensitive to small errors or noise from the deterministic dynamics model?
2 comments
AuthorsOpenReview ↗

Response to Reviewer rWjD

We thank the reviewer for the careful summary and the constructive questions.

  1. The two-stage structure is common. The dynamical applications cited in our introduction, turbulence super-resolution (Fan et al., 2025), cosmological super-resolution (Li et al., 2021) and atmospheric downscaling (Mardani et al., 2025), all evolve or receive the coarse state first and then sample fine detail at each time. This wide use is what motivates our study.

The main reason for modeling dynamics only at the coarse scale is cost: fine-scale simulation is expensive, while the coarse model is cheap. A second reason is that the two stages can be built separately, so an existing coarse solver can be kept while the decoder is trained offline, as in Fan et al. (2025).

Additional conditioning that is fixed in time or depends only on the current coarse state changes only ϕt\phi_t or GG, so our results still apply. Models that also condition on previously generated fine states or on hidden states are not the main object of our study, but we include such a memory decoder as a comparison in the one-dimensional Burgers study (Section 6). Our results explain why such models are needed. Under Eq. (1), exact composition and the stated regularity, generated fine detail carries no information forward beyond the initial class (Theorem 1). Keeping fine-scale detail over time therefore requires a model outside this setting, for example one with memory. The trade-off is speed against fine-scale temporal structure.

We agree that this discussion helps readers, and we added sentences to the first paragraph of the introduction that explain why this structure is studied and what question it raises.

  1. Thank you for raising this point. We agree that this comparison would make the empirical support more convincing. We looked into running it, but training a history-conditioned diffusion model on the two-dimensional benchmarks takes a long time, and our estimate showed that we could not obtain reliable results across PDEs and seeds within the discussion period. Rather than report preliminary results that we cannot fully validate, we leave this comparison to future work, and we have added a sentence to the limitations in Section 7 making this explicit.

  2. We appreciate this question, which points to an interesting open problem. The result is a theoretical observation rather than a practical recommendation. We found it in our analysis and include it because it clarifies what the architecture can represent, but how to use it in practice is not yet clear. The only example we study in detail is the rotation construction in Section 5. It shows that if a small part of the fine-scale information is left in the coarse state, a decoder with growing sensitivity can read it back out, so temporal dependence can persist as the coarse recovery error vanishes. The reviewer is right that this is fragile. The information sits in a small coarse offset, and the decoder must be steep to recover it, so errors or noise in the coarse dynamics of the same size as that offset can destroy it. Whether such constructions extend beyond this example, and how much information they can carry, remain open questions, as noted in Section 5. Our current results stop at this constructive example.

Thank you for your responses.

Your additional clarifications have addressed some of my concerns, and given that my initial recommendation was acceptance, I am inclined to leave it as is.

Comments on the paper

1 comment
AuthorsOpenReview ↗

General Response

We thank all reviewers for their time and comments. We have uploaded a revised manuscript, with all changes highlighted in blue, together with updated code. The main changes are:

  1. The Introduction now states the motivation and research question more clearly and ends with a list of contributions.
  2. Appendix G.5 adds a free-rollout experiment with learned decoders. It estimates decoder sensitivity and coarse recovery error and compares memoryless and memory-conditioned decoders. The code package includes this experiment.

Other changes are described in our individual replies.

From OpenReview, CC BY 4.0. Authors as signed there; shown unedited.

Related papers

Loading the map…

Discussion (0)

Sign in to comment.