acceptodds
Under review as a conference paper at ICLR 2027

Disintegrated Stochastic Interpolants

Abstract

Diffusion and flow-based models are effective across a wide range of generative tasks, from images and text to physical systems governed by partial differential equations. In scientific applications, known equality constraints can confine the target distribution to a lower-dimensional set, making it singular with respect to the ambient Lebesgue measure. An ambient model must learn both how to drive samples onto this set and how to recover the nontrivial distribution along it. We introduce disintegrated stochastic interpolants, which prescribe the collapse onto the constraint set and learn the remaining distributional transport. Our construction decomposes the state space into level sets of a known constraint map and separates transport between these sets from transport within them. The constraint values follow an analytically prescribed evolution towards zero, while the model learns only transport within the moving level sets. We establish conditions under which these components recover the target distribution and derive corresponding training objectives and sampling algorithms for flow and diffusion models. Beyond singular targets, the proposed framework can also exploit conservation laws and symmetries. For symmetric distributions, we recover the quotient-space diffusion loss xu2026quotient as a special case of our training objective. Across geometric and physics-based problems, our disintegrated models learn target distributions more accurately at matched training budgets while satisfying the prescribed constraints substantially more closely than ambient alternatives.

Primary area:
generative models
Keywords:
Stochastic Interpolants, Diffusion Models, Flow Matching, Generative Modeling, Geometric Deep Learning

Reviews

Among:
Shape:
1234this paper, Top 15%
Figure 1. Mean review rating of the reviewed papers at ICLR 2027. This paper averages 3.00 over 4 reviews, above 85% of the others.
Reviewer_RkFoOpenReview ↗
Rating
3
Confidence
3
Informativeness
3
Presentation
2
Soundness
3
Read review
Paper summary

The paper generalizes the theory of stochastic interpolants to targets whose structure is partly known through a structure map c whose level sets (fibres) partition the state space, for example for systems with constraints or invariant under symmetrries. Transport is disintegrated into a prescribed motion between fibres and a learned motion within fibres (a vertical velocity). The theory is fully developped giving the decomposition of the continuity equation and regression objectives. Experiments cover constrained partial differential equation (PDE) trajectory generation, QM9 molecular geometry with prescribed moments, a synthetic Stiefel target, and a planar chain with symmetry and bond constraints.

Critical strengths
  1. A very detailed and techically solid formulation of the problem in terms of measure-theoretic disintegration. This is the right mathematical framework, at the correct level of generality and I found this part of the paper extremely well written and very informative.

  2. The implicit construction (Equations 22 to 24 and Appendix C) is practical, with matrix-free projections and structure-specific direct solves. This makes the abstract measure-theoretic framework widely applicable. The theoretical foundation provided by the papers are overall very strong.

Critical weaknesses
  1. The abstract claims structure is satisfied "substantially better than alternatives", but in Table 2 the conservation error of CFFM equals that of the disintegrated model. Likewise the result in Table 3 seems to be summarized in a rosy manner. From my understanding the results from the paper are roughly comparable with state of the art models.
Questions for authors
  1. Can you justify better your claim that your "substantially better than alternatives"?

  2. Can you provide a critical analysis of the assumptions in the theory parts. For example the assumption of uniquness in Theorem 2.6. Can one verify the uniqueness by durect arguments in specific setting (say Stielfel or circle case)?

  3. The quotient-space diffusion equivalence is proved for planar rotations only. The main application of quotient-space diffusion is the three-dimensional rotation group acting on molecular coordinates, where no smooth global section exists and the orbits are curved. Does the equivalence in the abstract and Section 5 extend to that case, and what is the structure map?

2 comments

We thank the reviewer for their excellent questions, and we are very grateful for a thorough engagement with our work and for highlighting its strengths. We answer the questions below:

Can you justify better your claim that your "substantially better than alternatives"?

In the PDE experiments, DSI achieves better performance than a model with four times as many parameters and is also better than ambient Flow Matching in the QM9 experiments and competitive with the specialized baseline.

But we agree that "substantially better than alternatives" is an overstatement. Our main contribution is a theoretical framework that covers many possible methods, and building better algorithms based on this theory is a future direction. The current results show early evidence that it is possible to develop algorithms based on this theory that will be substantially better than alternatives.

Therefore, we will weaken the claims by changing the following sentence:

“ Across geometric and physics-based problems, our disintegrated models learn target distributions more accurately at matched training budgets and satisfy prescribed structure substantially better than alternatives.”

to

“Across geometric and physics-based problems, our experiments show that disintegrated models can improve distributional accuracy while satisfying prescribed structure.”

Can you provide a critical analysis of the assumptions in the theory parts. For example the assumption of uniquness in Theorem 2.6. Can one verify the uniqueness by durect arguments in specific setting (say Stielfel or circle case)?

Yes, we can. Focusing on the uniqueness assumption in Theorem 2.6, Theorem 1.1.10 in [1] provides a sufficient criterion: for an Itô SDE with locally Lipschitz drift and diffusion coefficients, solutions with the same initial distribution have the same law. For the base schedule Bt(y)=(1−t)yBt(y)=(1-t)y, our sampling equation (20) has the Itô form

d\widehat X_t =b_\theta(t,\widehat X_t)\,dt+\sigma(t,\widehat X_t)\,dW_t, \qquad b_\theta(t,x) =-\frac{H_xc(x)}{1-t} +\widehat u_t(x)+\varepsilon_t\widehat s_t(x) +\varepsilon_t\mathcal H(x),\\ \sigma(t,x)&=\sqrt{2\varepsilon_t}\,P_x, \end{aligned}$$ where $\mathcal H=\sum_jD(Pe_j)[Pe_j]$ is the curvature correction. We can apply Hsu’s theorem to the augmented state $(t,X_t)$. It is sufficient that the coefficients are locally Lipschitz jointly in $(t,x)$. Since sums and products of locally Lipschitz functions are locally Lipschitz, sufficient conditions on every $[0,T], T<1$, are: - **Constraint:** $c \in C^3$, with full-rank Jacobian on an open set U, and the scheduled fibres through each regular initial state remain in a compact subset of U. Then $H,P$ and $\mathcal H\in C^1$, making both the curvature term and the prescribed drift locally Lipschitz. - **Network outputs:** The velocity and score/noise networks are locally Lipschitz jointly in $(t,x)$. Projection by $P_x$ preserves this property. - **Noise schedule:** The sampling amplitude $\sqrt{2\varepsilon_t}$ is locally Lipschitz in time. For the **circle**, $c(x)=\|x\|-1$ is smooth with nonzero gradient on $U=\mathbb R^2\setminus\{0\}$. The prescribed radius satisfies $$r_t=(1-t)r_0+t \in[\min(r_0,1),\max(r_0,1)].$$ Thus, for $r_0>0$ the condition on the constraint function is satisfied. For **Stiefel**, $c(X)=X^\top X-I$ is smooth with a full-rank Jacobian whenever X has full column rank. Starting from $G_0 \succ 0$, the prescribed evolution $$G_t=X_t^\top X_t=(1-t)G_0+tI$$ keeps matrices bounded and away from rank deficiency, verifying the constraint condition. These, together with the stated network and schedule assumptions, establish the assumptions for Theorem 2.6. > The quotient-space diffusion equivalence is proved for planar rotations only Thank you for raising this point! It's actually an important but subtle technical point which we would like to clarify. Yes, the equivalence extends to SO(3) actions but with a slight change to what is presented in the main text. The problematic part in the main text is that in our formulation the vertical subspaces always come from the kernel of the structure map $c$. The regularity properties of the structure map make Dc integrable, whereas the directions orthogonal to rotation orbits need not be. Allowing projectors that do not arise from a structure map preserves all the theorems related to the prescribed/learned decomposition and extends the transport, training and sampling constructions, recovering QSD under its assumptions without requiring a global section. However, presenting this more general formulation adds abstraction and complexity unnecessary for our focus on constraints, so we have rewritten Appendix E.1 to explain this generalization, and will qualify the equivalence claims in the abstract and Section 5 accordingly. 1. Hsu, E. P. (2002). Stochastic Analysis on Manifolds. Graduate Studies in Mathematics, Vol. 38. American Mathematical Society

I thank the authors for their detailed answers. This is only reinforce my opinion that is a strong theoretical work, and I keep my grade.

Reviewer_feSrOpenReview ↗
Rating
3
Confidence
1
Informativeness
3
Presentation
2
Soundness
2
Read review
Paper summary

[DISCLAIMER] The topic of this paper is outside my area of expertise and I may not be the most suitable reviewer to provide a thorough and fair assessment. I have already notified the Area Chair about this. Therefore, my comments are more high-level and based on my limited knowledge of this field and my understanding of the paper. Thank you for your understanding.


This paper introduces a theoretical framework for generative modeling approaches, such as flow and diffusion models, in settings where part of the structure of the target distribution is known in advance, such as conservation laws, equality constraints, or symmetries. The main idea of the paper is to encode the known data structure or constraints using a map whose level sets partition the ambient state space into two components: (i) a distribution over structural values and (ii) conditional distributions within the corresponding level sets (or fibers as referred to in the paper).

With this formulation, the transport formulation in generative models is divided into two components: (i) transport between fibers whose evolution is formulated analytically and does not require any learning or training and (ii) transport within each fiber (level set) which is learned using a neural network similar to flow-matching or diffusion models. The main advantage of this decomposition is that the generative model does not need to learn the entire transport from data. Instead, part of the transport can be handled by leveraging prior knowledge of the target distribution's structure/constrain, potentially reducing the computational budget required to train generative models such as flow-based and diffusion-based models. Across various experiments, including PDE trajectory generation, molecular reconstruction and a synthetic benchmark, the proposed method significantly improves satisfaction of the target constraints compared to standard ambient flow matching.

Critical strengths
  • The strongest aspect of the paper is the decomposition of generative transport into analytically formulated dynamics (between-fibre) and learned dynamics (within-fibre). This has two main advantages: (i) it provides a systematic way to address how known constraints in the target distribution can be incorporated into generative modeling and (ii) depending on the task and application this approach offloads part of the computational burden from the neural network used in generative models such as flow-matching and diffusion models, potentially speeding up the training process.

  • Also, the results on the continuity equation (Theorem 2.3), velocity regression (Theorem 2.4), and fibre scores connect geometric construction to practical generative modeling objectives (flow and diffusion) and provide a common framework for understanding several classes of generative models that might otherwise appear to be separate constructions.

  • The experiments show significant improvements in structural fidelity over standard ambient baselines. Specifically, the proposed method reduces constraint violation by several order of magnitude on the Stiefel, PDE, and QM9 experiments compared to baselines. Overall, I believe these results support the main motivation of the paper that known structure can reduce the computation burden placed on the generative models.

Critical weaknesses
  • Although the paper evaluates the proposed framework on PDE trajectory generation and molecular generation in Sec4, the evaluation remains relatively limited in scale. Specifically, the paper does not show the approach on large-scale generative tasks, such as image synthesis, where flow- and diffusion-based models are widely used. This leaves an important question about the scalability of the proposed theory and it remains unclear how the method behaves computationally in higher-dimensional settings.

  • The disintegrated formulation reduces what the neural network must learn but my understanding is that it introduces additional geometric computations and repeated operations such as retraction mappings during training and sampling which might make training slower in some cases, as also reported in Tables 5 and 8 for two different tasks. So, despite the claimed benefits, the proposed method may have limitations in terms of computational efficiency. If this understanding is correct, I would recommend that authors highlight this limitation in the paper as it would provide practitioners with better insights into when the proposed method is beneficial and how it can be used in future research.

Questions for authors
  • Regarding the high-dimensional tasks, as I mentioned in the weaknesses, can authors provide evidence or experimental results on a higher-dimensional task? Even if the results are negative, I believe it would be valuable to include quantitative and qualitative results in the paper, as they could provide readers with useful insights into the limitations of the current approach and potential future directions for this work. Also, one interesting point to discuss in the context of high-dimensional tasks is whether the target coordinate is always difficult to define explicitly, making an implicit formulation necessary or whether there are ways to define an explicit target coordinate to simplify the learning process.
2 comments

We thank the reviewer for their excellent questions and thorough engagement with our work. We answer the questions below:

If this understanding is correct, I would recommend that authors highlight this limitation in the paper as it would provide practitioners with better insights into when the proposed method is beneficial and how it can be used in future research.

We agree that this is a limitation that should be acknowledged. Please see our response to Reviewers SiYu and XQC7 for a precise quantification of this computational overhead.

We also acknowledge this in the paper in lines 479-480: “The main limitations are that the implicit method introduces computational overhead through projections and retractions, which can offset the benefits of a simpler learning problem.” However, to highlight this limitation more prominently, we will restructure the conclusion to have a separate paragraph on limitations.

Regarding the high-dimensional tasks, as I mentioned in the weaknesses, can authors provide evidence or experimental results on a higher-dimensional task?

Thank you for the suggestion. We added CIFAR-10 image inpainting, using 32x32 RGB images and a central 16x16 missing region found in [1] and [2]. We evaluate all 10,000 test images at 100 NFE, comparing explicit DSI with an adapted I²SB OT-ODE baseline and a frozen pretrained DDNM model. DSI and I²SB each receive 20,000 training updates.

MethodFID ↓Classification accuracy ↑PSNR ↑SSIM ↑
DDNM (pretrained)6.360376.75%21.30080.7422
DSI5.836279.51%21.54720.7566
I²SB (OT-ODE adaptation)5.258682.10%22.17850.7783

DSI improves over the frozen DDNM baseline and approaches I²SB's performance, although I²SB remains better on these quality metrics. However, all of these methods are really similar to each other in terms of the actual computations being done.

We would like to make two comments about these results

  1. This experiment uses an explicit version and the constraints are linear, so it is perhaps a bit less interesting than the experiments in the paper because it doesn’t test the nonlinear geometry and implicit formulation developed in the paper. Extending the implicit approach to image problems with nonlinear constraints is a promising direction for future work.

  2. The close connections to DDNM and I²SB illustrate DSI's role as a general framework: similar decompositions arise as a natural heuristic in task-specific methods, while DSI provides a unified treatment and explicit conditions for distributional correctness. We also discuss AssembleFlow in the paper as another example of the explicit formulation.

Also, one interesting point to discuss in the context of high-dimensional tasks is whether the target coordinate is always difficult to define explicitly, making an implicit formulation necessary

Mathematicians have studied when such coordinates exist. For fibre bundles, this is the question of whether a global trivialisation exists; see Neeb (2024, Proposition 2.1.1(b)). These results tell us about existence, but finding coordinates that are numerically stable and computationally efficient is a separate problem. The key point for our work is that global coordinates are not always available, and even when they exist, they may be difficult to construct. And our understanding is that for most nonlinear constraints such coordinates don't exist.

Liu, Guan-Horng, et al. "I 2^ 2 SB: Image-to-Image Schr" odinger Bridge." arXiv preprint arXiv:2302.05872 (2023).

Wang, Yinhuai, Jiwen Yu, and Jian Zhang. "Zero-shot image restoration using denoising diffusion null-space model." arXiv preprint arXiv:2212.00490 (2022).

Karl-Hermann Neeb, Differential Topology of Fiber Bundles (2024), Proposition 2.1.1(b).

I thank the authors for the clarifications and for providing additional results on the image-classification task. Despite the existing computational overhead and the fact that the method has not been well evaluated on high-dimensional tasks, such as image generation problems, as acknowledged by the authors, I believe the paper proposes a strong mathematical foundation/framework that can be further extended by researchers, either by developing better algorithms based on this theory and/or by extending the implicit approach to image problems with nonlinear constraints (both also have been mentioned by the authors).

Therefore, I maintain my original score (3 - weak acceptance).

Reviewer_XQC7OpenReview ↗
Rating
3(+1)
Confidence
3
Informativeness
2
Presentation
2
Soundness
2
Read review
Paper summary

The paper introduces disintegrated stochastic interpolants, a framework for generative modeling that uses known structural information. A structure map c\mathbf{c} partitions the data space into fibres, each containing points with the same structural value c(x)\mathbf{c}(x). The target distribution is described by the distribution of these structural values, which is assumed known, and the conditional distributions within the fibres. The generative velocity is divided accordingly: a prescribed horizontal component moves samples between fibres according to a known base schedule, while a learned vertical component moves samples within each fibre. The measure-theoretic formulation accommodates singular targets and provides a common treatment of equality constraints and symmetries. It also recovers quotient-space diffusion under the stated geometric assumptions. The implicit method is evaluated on a Stiefel manifold benchmark, trajectory generation for partial differential equations, and moment-constrained molecular reconstruction on QM9.

Critical strengths
  1. The theoretical development is careful and accommodates singular target distributions without requiring ambient Lebesgue densities.
  2. The structure-map formulation provides a common treatment of constraints and symmetries, with an explicit connection to quotient-space diffusion under the stated assumptions.
  3. In continuous time, the prescribed structural evolution is preserved even when the learned tangent fields are imperfect (Theorem 2.6).
  4. The implicit construction works directly with constraint functions and supports sources outside the target manifold, including ambient Gaussians, without requiring global fibre coordinates.
  5. On both trajectory-generation tasks, the method reports lower errors in the generated ensemble means than conditional functional flow matching at matched network capacity. These errors are also lower than those of a larger baseline with approximately four times as many parameters.
Critical weaknesses
  1. The practical diffusion method uses an approximate score target. The projected Gaussian target after nonlinear retraction generally does not satisfy Eq. (17), as required by Theorem 2.5. The fibrewise score-matching alternative in Appendix B.6 is not evaluated. Diffusion is tested only on a small Stiefel experiment, with 150 training updates, 128 evaluation samples, and no intrinsic manifold baseline, leaving the approximation’s impact unclear.

  2. Some comparisons do not isolate the proposed decomposition. On QM9, the proposed method uses a near-manifold source, while ambient flow matching uses a Gaussian. The chain comparison between the two DI variants changes both the structure map and endpoint alignment. Matched-source and matched-alignment comparisons would clarify which changes drive the improvements.

  3. Applicability depends on accurately known and compatible structure. The theory assumes a known structure map and target base law, a full-rank constraint Jacobian along the paths, and compatibility between the interpolant and prescribed structural dynamics. Its robustness to approximate or redundant constraints, or to structural dynamics that must be learned or fine-tuned, remains unclear.

  4. The quality improvements are mixed and incompletely measured. On the PDE tasks, constraint errors are already comparable to conditional functional flow matching, and a terminal-projection baseline is missing on Stiefel and QM9. The trajectory metrics do not assess joint structure or local equation residuals. The proposed method improves mean-field error but has higher standard-deviation error than the larger conditional baseline. On QM9, slightly higher diversity accompanies lower validity and worse stability than Stiefel flow matching. Its advantages over strong baselines are therefore metric-dependent.

  5. The overall computational benefit is not established. Gains per update or parameter do not account for geometric overhead. Chain sampling is about 7.6 times slower than ambient flow matching, and the cost of constructing cached training paths is not reported. Accuracy comparisons at matched total training times would clarify the practical benefit.

Questions for authors
  1. Can you compare the projected-noise target with the fibrewise implicit score-matching objective in Appendix B.6 at several noise amplitudes?
  2. Do the improvements persist when using the same source distribution on QM9 and the same endpoint alignment between the two DI variants in the chain experiment?
  3. What remains guaranteed if constraints are approximate or structural dynamics are learned or fine-tuned? How sensitive are results to small errors in the assumed structure?
  4. Can you add a terminal-projection baseline on Stiefel and QM9, and assess trajectory correlations and local equation residuals for the PDE tasks?
  5. How does accuracy compare at matched total training times, including corrected-path construction and caching, and what is the corresponding sampling cost?
5 comments

We thank the reviewer for their comments, and we respond to their questions below:

Can you compare the projected-noise target with the fibrewise implicit score-matching objective in Appendix B.6 at several noise amplitudes?

Yes. We trained St(10,4) diffusion models using projected-noise regression or fibrewise implicit score matching (Appendix B.6), keeping the velocity objective unchanged. We varied γ(t)=asin(πt)\gamma(t)=a sin(\pi t), so larger a increases intermediate-path noise while noise remains zero at both endpoints. Each amplitude uses separately trained models with matched architectures, paths, samplers and tuning budgets.

After 20,000 updates, validation-selected models achieved the following unsquared SW₂ on 2,048 test samples, using 128 sampling steps:

Noise amplitude aProjected-noise SW₂ ↓Implicit score-matching SW₂ ↓
0.100.0186760.017947
0.350.0189330.019232
0.700.0180890.018323

Both objectives achieve comparable sample quality.

Do the improvements persist when using the same source distribution on QM9 and the same endpoint alignment between the two DSI variants in the chain experiment?

Yes, they do!

QM9. Our original setup followed Stiefel-FM closely, but we agree that comparison with Gaussian-source ambient FM is more informative. With the same Gaussian source:

MethodCE ↓Valid (%) ↑Stable ↓Diverse (Å) ↑NFE
Ambient FM9.53 ± 0.7449.05 ± 1.86−0.317 ± 0.0241.4519 ± 0.0183200
Revised DSI8.22517×10⁻⁶53.55−0.4897701.49042200
DSI — paper(5.19 ± 0.38)×10⁻⁵56.82 ± 1.42−0.521 ± 0.0311.4882 ± 0.0147200

We will repeat this experiment over three seeds and replace the corresponding table results.

Chains. We repeated the comparison between DSI (bonds), which constrains bond lengths, and DSI (full), which additionally preserves orientation. This removes the coupling difference acknowledged in Appendix E.2.

20,000 updates; 5,000 test samples; 200 NFE; one training seed.

Metric ↓DSI (bonds)DSI (full)
Shape MMD² ×10⁴0.168−0.333
Mode-weight TV0.00880.0088
Orientation KS0.007030.00875
99th-percentile bond error2.22×10⁻¹⁶2.22×10⁻¹⁶

Both retain strong shape accuracy and machine-precision bond constraints.

What remains guaranteed if constraints are approximate or structural dynamics are learned or fine-tuned? How sensitive are results to small errors in the assumed structure?

What remains guaranteed. Theorem 2.6 guarantees the prescribed structural evolution in continuous time even when the learned tangent fields are imperfect. If the constraint is misspecified, this guarantees satisfaction of the assumed constraint, not recovery of the true data distribution. In this case, we need to modify the constraint to incorporate the imperfect information.

Sensitivity experiment. We added a controlled experiment in which observations increasingly violate an assumed constraint. Clean samples X follows a three-component von Mises mixture on the unit circle. We corrupt them as Yσ=X+σZ,Z∼N(0,I2),Y_\sigma=X+\sigma Z,\qquad Z\sim\mathcal N(0,I_2), while retaining the constraint c(x)=∥x∥2−1=0c(x)=\|x\|^2-1=0.

Both methods start from the same standard Gaussian source. DI-FM projects the noisy endpoints onto the circle, learns angular transport, and prescribes the radial evolution analytically. It therefore cannot reproduce off-circle observations. Both methods use identical three-layer MLPs, and 5,000 updates, with matched initialisation and training randomness. We evaluate against 1,024 noisy validation observations, using 256 sampling steps and sliced W2 metric.

Observation noiseAmbient FM SW₂ ↓DI-FM SW₂ ↓
00.081550.04784
0.0250.083440.04696
0.100.081100.05359
0.200.056610.09121
0.500.064700.24620

At this training budget, DI-FM performs better under small observation noise, but enforcing the circle becomes detrimental as the observations depart substantially from it.

Can you add a terminal-projection baseline on Stiefel and QM9,

Thank you for the suggestion. We assume by terminal projection you mean a PCFM-style projection onto the feasible set of same ambient FM samples, without retraining or resampling.

Stiefel. Projection enforces the constraint to numerical precision, but DI-FM retains the lowest distributional error.

MethodSliced Wasserstein distance ↓Gram constraint RMS ↓
Ambient FM0.0251.28268
Ambient FM + terminal projection0.0221.5510-16
DI-FM0.01851.7010-16

QM9. We applied terminal projection to the same 2,560 ambient FM candidates

MethodConstraint error ↓Validity (%) ↑Stability ↓Diversity ↑
Ambient FM10.052447.73−0.3008821.43901
Ambient FM + terminal projection8.2910-644.73−0.268224†1.49012

These results show that terminal projection is useful for enforcing constraints, but it can have an adverse effect on sample quality. It's unclear why this happens, but we believe it's because projections distort the distribution; see [1]

assess trajectory correlations

We evaluated spatial and temporal correlation MSE against simulator references. Results use eight conditions × 64 trajectories, 32 NFE.

TaskMethodSpatial correlation MSE ↓Temporal correlation MSE ↓
BurgersCFFM-small0.0042400.002367
BurgersCFFM-large0.0022760.000919
BurgersDSI0.0055250.001951
Reaction–diffusionCFFM-small0.0037790.001340
Reaction–diffusionCFFM-large0.0033400.000549
Reaction–diffusionDSI0.0029710.001229

DSI achieves the lowest spatial-correlation error on reaction–diffusion, while CFFM-large achieves lower temporal-correlation errors. We also found that terminal corrections damage baseline correlations, so these are without any projections.

How does accuracy compare at matched total training times, including corrected-path construction and caching, and what is the corresponding sampling cost?

General Summary of training and sampling cost overhead

For one flow-matching training update, using the same network and batch size:

+T_{\mathrm{path}} +T_{\mathrm{projection}}$$ Here, $T_{\mathrm{path}}$ covers the additional path construction (Algorithm 2, steps 4–6) and velocity target used in step 7, while $T_{\mathrm{projection}}$ covers the projection in step 7-9. For one sampling step, using the same network, batch size: $$S_{\mathrm{DI}}\approx S_{\mathrm{amb}} +S_{\mathrm{projection}} +S_{\mathrm{retraction}}$$ Whether the proposed algorithm saves compute depends on whether smaller networks or faster convergence from avoiding the singularity discussed in Appendix D can offset the overhead above; we investigate this trade-off empirically below. ## Empirical measurements We report mean ± standard deviation in milliseconds per batch over 10 seeds. **PDEs** — Training batch 8; sampling batch 8, 32 NFE. | Task | Method / implementation | Training total | Sampling total | |---|---|---:|---:| | Burgers | CFFM-small | 5.04 ± 0.05 | 47.22 ± 0.63 | | Burgers | CFFM-large | 7.41 ± 0.24 | 66.66 ± 0.77 | | Burgers | DSI structured — `jax.lax.scan` LDL solver | 7.98 ± 0.30 | 108.54 ± 0.72 | | Burgers | DSI structured — `jax.lax.linalg.tridiagonal_solve` | 5.16 ± 0.12 | 45.98 ± 0.56 | | Burgers | DSI LSMR | 5.35 ± 0.01 | 69.13 ± 0.16 | | RD | CFFM-small | 3.75 ± 0.08 | 32.04 ± 0.16 | | RD | CFFM-large | 5.07 ± 0.08 | 41.56 ± 0.20 | | RD | DSI structured — `jax.lax.scan` LDL solver | 8.41 ± 0.68 | 105.97 ± 0.75 | | RD | DSI structured — `jax.lax.linalg.tridiagonal_solve` | 5.62 ± 0.14 | 33.61 ± 0.17 | | RD | DSI LSMR | 6.21 ± 0.11 | 56.47 ± 0.14 | **QM9** — Training batch 64; sampling batch 80, 200 NFE. DSI uses the original near-manifold source. | Method | Training total | Sampling total | |---|---:|---:| | Ambient FM | 20.96 ± 0.10 | 1144.74 ± 8.98 | | Stiefel-FM | 29.05 ± 0.10 | 2210.02 ± 11.64 | | DSI structured | 22.93 ± 0.11 | 1362.49 ± 11.02 | | DSI LSMR | 62.95 ± 0.32 | 9407.37 ± 24.95 | Since the experiments use the same number of updates, end-to-end training costs are approximately a common multiple of the per-batch update times. We would like to make three additional observations about these measurements: 1. Explicit and implicit methods have different costs. Explicit methods are basically ordinary flow matching in lower-dimensional space, with only additional work being coordinate transformations, which are usually. The new CIFAR illustrates an inexpensive linear case. All the reported timings are for implicit methods, which are more interesting. 2. Path construction is an important opportunity for improvement. The computational overhead can be divided into path construction and one additional projection in the update. We find that path construction can dominate the additional cost, while projection is inexpensive. For example, in reaction–diffusion, path construction takes 2.97 ms per batch, whereas projection only adds 0.078 ms to a normal update. Our current interpolants enforce ($c(X_t)=B_t(c(X_0))$) along every path, a stronger requirement than separability in Definition 2.2. Exploring a broader class of interpolants could reduce construction costs significantly. 3. Exploiting the constraint Jacobian’s structure matters. Our PDE and QM9 constraints admit efficient structured solves. On QM9, the generic LSMR implementation takes approximately 2.9 times longer for training and 7.2 times longer for sampling than the structured implementation. Even while using structured solves the specific implementation matters. Scanning over the tridiagonal matrix is much slower than using jax.lax.linalg.tridiagonal_solve. In summary: The computational overhead in none of the tasks is prohibitive, but exploiting the structure in the constraints and implementation details matters a lot. For the future, we want to reduce the computational overhead by exploring a broader class of interpolants and improving the generic LSMR solver. References: 1. Xu, Jian, et al. "The Right Measure for Physics-Constrained Generation: A Co-Area Correction for Posterior-Consistent PDE Inverse Problems." arXiv preprint arXiv:2606.04804 (2026).
Reviewer_SiYuOpenReview ↗
Rating
3
Confidence
4
Informativeness
2
Presentation
3
Soundness
3
Read review
Paper summary

This paper proposes Disintegrated Stochastic Interpolants (DSI), a unified framework for exploiting known structural information, such as equality constraints, conservation laws, and symmetries, in flow-based and diffusion-based generative models. By disintegrating probability measures along the fibres of a structural map, the method decomposes the generative dynamics into a prescribed horizontal component and a learnable vertical component, reducing the burden on neural networks.

The authors establish theoretical guarantees for this decomposition, derive corresponding training objectives, and develop both explicit and implicit implementations. Experiments on PDE trajectory generation, QM9 molecular reconstruction, and Stiefel manifolds demonstrate improved distributional accuracy and constraint satisfaction compared with several baselines.

Critical strengths
  1. Solid theoretical foundation and generality. The paper provides a principled framework for separating known structural dynamics from learnable transport through measure disintegration. The theoretical results, particularly the separability condition and corresponding transport equations, offer a rigorous foundation for structured generative modeling. I consider this the paper's most significant contribution.

  2. Practical relevance and convincing empirical results. The proposed explicit and implicit methods connect the theory to practical algorithms without requiring explicit manifold parameterizations. Experiments across PDEs, molecular generation, and Stiefel manifolds demonstrate improved distributional accuracy and constraint satisfaction, supporting the practical value of the framework.

Critical weaknesses
  1. Computational overhead and practical efficiency. The implicit method introduces additional costs from Jacobian computations, projections, and retractions. Although some runtime results are provided, a more systematic comparison of end-to-end training and sampling costs would help assess its practical benefits. I do not expect the proposed method to outperform baselines in every setting, but a clearer analysis of when its learning advantages outweigh the computational overhead would strengthen the paper.

  2. Approximation in implicit diffusion. The projected Gaussian noise target in Algorithm 2 relies on a small-noise approximation, which may deviate from the true fibrewise score due to nonlinear retractions. The paper would benefit from further empirical analysis of how this approximation affects generation quality under different noise levels and constraint geometries. Understanding when the approximation works well, and when it becomes less reliable, would clarify the method's practical limitations.

Questions for authors
  1. Relationship to existing methods. Could the authors clarify the key theoretical and algorithmic differences between DSI and related approaches such as Neural SHAKE (2025) and Constraint-Aware Flow Matching (2026)? These works also incorporate geometric constraints and projections into generative dynamics or training. A more explicit comparison would help clarify the unique contributions of DSI.

  2. Applicability to image inverse problems. Could the proposed framework be extended to image inverse problems, where the observation operator provides known structural information? Related works such as DDNM (ICLR 2023) and FlowPET (ICML 2026) also exploit decompositions between constrained and unconstrained components. It would be interesting to understand whether DSI offers additional theoretical or practical advantages in these settings, particularly for noisy or nonlinear observations.

  3. Practical trade-offs. Could the authors provide further evidence on the end-to-end computational overhead and the sensitivity of implicit diffusion to the projected-noise approximation? The results need not demonstrate improvements in every setting; a clearer characterization of when the method is beneficial, and when its assumptions or computational costs become limiting, would be valuable.

4 comments

We thank the reviewer for the excellent review, and we are very grateful for a thorough engagement with our work and for highlighting its strengths. We completely agree with the reviewer's assessment that our main contribution is theoretical. Building better algorithms based on this theory is a future direction we are eager to pursue.

Relationship to existing methods. Could the authors clarify the key theoretical and algorithmic differences between DSI and related approaches such as Neural SHAKE (2025) and Constraint-Aware Flow Matching (2026)?

We thank the reviewer for highlighting these relevant and interesting works and will include them in the related-work discussion. The key differences are the following:

1. Projection and training objectives. CAFM projects predicted endpoints onto the feasible set inside its loss:

=\mathbb E\left\|\Pi_C\left(X_0+v_\theta(X_t,t)\right)-X_1\right\|^2.$$ For nonlinear constraints, this generally requires nonlinear optimisation and differentiation through the projection. By contrast, our approach only requires much cheaper linear projections. Neural SHAKE is a sampling-time intervention that is closer to PCFM than our disintegrated stochastic interpolants (DSI). It projects the updated state after each sampling step without modifying training. DSI projects the **velocity-regression residual onto the constraint Jacobian's null space**: $$\mathcal L_{\mathrm{DSI}} =\mathbb E\left\|P_{X_t}\bigl(u_\theta(t,X_t)-\dot X_t\bigr)\right\|^2.$$ Applying this projector requires a linear least-squares solve, which can be done with highly efficient solvers. **2. Distributional correctness.** CAFM doesn’t theoretically guarantee that zero loss implies the correct distribution at t=1. It is possible to construct counterexamples even with linear constraints. For brevity, we skip the details, but are happy to elaborate if the reviewer is interested. Neural SHAKE's Theorem 3 states that projected sampling produces the ambient density restricted and normalised to the constraint surface. But its proof identifies **projecting a process** with **restricting its density**. These operations need not agree: even with exact scores and linear constraints, projected sampling can produce the wrong final conditional distribution; see [1] for a detailed discussion of this. By contrast, DSI's Theorems 2.3–2.6 establish that regression recovers the target law under the stated assumptions—guaranteeing **the correct distribution and constraint satisfaction**. **3. Different goals.** DSI aims not only to satisfy constraints but also to simplify learning: known structural dynamics are prescribed analytically, leaving only within-fibre transport to be learned. > Applicability to image inverse problems This is a very interesting future direction we would like to explore and an ideal application for Disintegrate interpolants. Regarding comparisons to the related methods: We found that DDNM is extremely close to explicit DSI with linear observations (Ax=y) and a one-sided Gaussian interpolant, using the score-only diffusion version of Algorithm 1 (Appendix B.6). The disintegration gives DDNM’s range–null–space construction essentially. The remaining differences concern training and sampling heuristics: DDNM reuses a pretrained unconditional denoiser and corrects its clean-image predictions, whereas DSI (ours) learns the conditional fibre score and follows a prescribed observation path throughout sampling. FlowPET also uses a range–null-space decomposition, but combines it with Hamiltonian dynamics and symplectic integration. This suggests a conceptual similarity to DI, although we have not established a precise equivalence. Following reviewer feSr’s request for higher-dimensional experiments, we evaluated DSI on one of the experiments from DDNM, comparing against I²SB and DDNM on CIFAR-10 inpainting with a central 16×16 missing region. All methods restore the same 10,000 test images using 100 network evaluations per image. We use **explicit linear-fibre, DSI diffusion** (from algorithm 1), with a Gaussian source and an Euler–Maruyama sampler evolving the 768 missing coordinates. I²SB [2] uses the OT-ODE implementation from their release on this tasks and **DDNM uses the released `ema_cifar10` checkpoint from its official repository, kept frozen throughout**; | Method | FID | Accuracy ↑ | PSNR (dB) | SSIM | |---|---:|---:|---:|---:| | DDNM | 6.360 | 76.75% | 21.300 | 0.7421 | | DSI | 5.836 | 79.51% | 21.5472 | 0.7566 | | I²SB | **5.258** | **82.10%** | **22.1785** | **0.7783** | Sampling times per batch of 32 were 439 ms, 1,386 ms and 1,398 ms for DI, I²SB and DDNM, respectively; but this difference could simply be due to the difference between compiled JAX versus PyTorch implementations.

DSI offers additional theoretical or practical advantages in these settings, particularly for noisy or nonlinear observations.

We are also very interested in these settings because the key advantage of DSI is a rigorous way to handle nonlinearities with its geometric machinery and the implicit formulation. However, we could not find any experiments with nonlinear observations in DDNM, and it seems to be specifically designed to handle linear constraints.

Practical trade-offs. Could the authors provide further evidence on the end-to-end computational overhead …

General Summary of training and sampling cost overhead

For one flow-matching training update, using the same network and batch size:

+T_{\mathrm{path}} +T_{\mathrm{projection}}$$ Here, $T_{\mathrm{path}}$ covers the additional path construction (Algorithm 2, steps 4–6) and velocity target used in step 7, while $T_{\mathrm{projection}}$ covers the projection in step 7-9. For one sampling step, using the same network, batch size: $$S_{\mathrm{DI}}\approx S_{\mathrm{amb}} +S_{\mathrm{projection}} +S_{\mathrm{retraction}}$$ Whether the proposed algorithm saves compute depends on whether smaller networks or faster convergence from avoiding the singularity discussed in Appendix D can offset the overhead above; we investigate this trade-off empirically below. ## Empirical measurements We report mean ± standard deviation in milliseconds per batch over 10 seeds. **PDEs** — Training batch 8; sampling batch 8, 32 NFE. | Task | Method / implementation | Training total | Sampling total | |---|---|---:|---:| | Burgers | CFFM-small | 5.04 ± 0.05 | 47.22 ± 0.63 | | Burgers | CFFM-large | 7.41 ± 0.24 | 66.66 ± 0.77 | | Burgers | DSI structured — `jax.lax.scan` LDL solver | 7.98 ± 0.30 | 108.54 ± 0.72 | | Burgers | DSI structured — `jax.lax.linalg.tridiagonal_solve` | 5.16 ± 0.12 | 45.98 ± 0.56 | | Burgers | DSI LSMR | 5.35 ± 0.01 | 69.13 ± 0.16 | | RD | CFFM-small | 3.75 ± 0.08 | 32.04 ± 0.16 | | RD | CFFM-large | 5.07 ± 0.08 | 41.56 ± 0.20 | | RD | DSI structured — `jax.lax.scan` LDL solver | 8.41 ± 0.68 | 105.97 ± 0.75 | | RD | DSI structured — `jax.lax.linalg.tridiagonal_solve` | 5.62 ± 0.14 | 33.61 ± 0.17 | | RD | DSI LSMR | 6.21 ± 0.11 | 56.47 ± 0.14 | **QM9** — Training batch 64; sampling batch 80, 200 NFE. DSI uses the original near-manifold source. | Method | Training total | Sampling total | |---|---:|---:| | Ambient FM | 20.96 ± 0.10 | 1144.74 ± 8.98 | | Stiefel-FM | 29.05 ± 0.10 | 2210.02 ± 11.64 | | DSI structured | 22.93 ± 0.11 | 1362.49 ± 11.02 | | DSI LSMR | 62.95 ± 0.32 | 9407.37 ± 24.95 | Since the experiments use the same number of updates, end-to-end training costs are approximately a common multiple of the per-batch update times. We make three additional observations about these measurements: 1. **Explicit and implicit methods have different costs**. Explicit methods are basically ordinary flow matching in lower-dimensional space, with only additional work being coordinate transformations, which are usually not expensive. The new CIFAR experiment illustrates an inexpensive linear case. All the reported timings are for implicit methods, which are more interesting. 2. **Path construction is an important opportunity for improvement**. The computational overhead can be divided into path construction and one additional projection in the update. We find that path construction can dominate the additional cost, while projection is inexpensive. For example, in reaction–diffusion, path construction takes 2.97 ms per batch, whereas projection only adds 0.078 ms to a normal update. Our current interpolants enforce ($c(X_t)=B_t(c(X_0))$) along every path, a stronger requirement than separability in Definition 2.2. Exploring a broader class of interpolants could reduce construction costs significantly. 3. **Exploiting the constraint Jacobian’s structure matters**. Our PDE and QM9 constraints admit efficient structured solves. On QM9, the generic LSMR implementation takes approximately 2.9 times longer for training and 7.2 times longer for sampling than the structured implementation. Even while using structured solves the specific implementation matters. Scanning over the tridiagonal matrix is much slower than using jax.lax.linalg.tridiagonal_solve. In summary: The computational overhead in none of the tasks is prohibitive, but exploiting the structure in the constraints and implementation details matters.

Approximation in implicit diffusion

To measure the approximation error, let's consider a simple example where the fibres are circles of different radii. For a circle of radius R, tangent Gaussian noise followed by normalization gives θ=arctan(aZ)\theta=arctan(aZ), where a=ρ/Ra=\rho/R and Z∼N(0,1)Z\sim\mathcal N(0,1). With eθe_\theta the unit tangent, the exact conditional score and mean projected-noise approximation are

\qquad \widetilde s=-\frac{Z}{\sigma\sqrt{1+a^2Z^2}}\,e_\theta.$$ Using **one million Gaussian samples**, the relative RMS approximation error is: | Noise/radius /R | 0.05 | 0.10 | 0.35 | 0.70 | |---|---:|---:|---:|---:| | Relative score error | 1.1% | 4.3% | 39.7% | 81.6% | The approximation is accurate when noise is small relative to the curvature radius and deteriorates as this ratio increases. 1. Da Costa, NathaÃŦl, Marvin PfÃķrtner, and Jon Cockayne. "Constructive Disintegration and Conditional Modes." arXiv preprint arXiv:2508.00617 (2025). 2. Liu, Guan-Horng, et al. "I $^ 2$ SB: Image-to-Image Schr\" odinger Bridge." arXiv preprint arXiv:2302.05872 (2023).

I thank the authors for their detailed responses and additional experiments, as well as the other reviewers for their thoughtful feedback.

The rebuttal has addressed several of my concerns. While the practical efficiency of the implicit formulation still appears to depend on the constraint structure and solver implementation, I appreciate the additional evidence and clarifications. I continue to view the theoretical framework as the main contribution of this work and will maintain my original score of 3 (weak accept).

From OpenReview, CC BY 4.0. Authors as signed there; shown unedited.

Related papers

Loading the map…

Discussion (0)

Sign in to comment.