acceptodds
Under review as a conference paper at ICLR 2027

Constraint-Aware Flow Matching: Decision-Aligned End-to-End Training for Constrained Sampling

18 views

Abstract

Deep generative models provide state-of-the-art performance across a wide array of applications, with recent studies showing increasing applicability for science and engineering. Despite a growing corpus of literature focused on the integration of physics-based constraints into the generation process, existing approaches fail to enforce strict constraint satisfaction while maintaining sample quality. In particular, training-free constrained sampling methods, while providing per-sample feasibility guarantees, introduce a fundamental mismatch between the training objective and the constrained sampling procedure, often leading to performance degradation. Identifying this training-sampling misalignment as a central limitation of current constrained generative modeling approaches, this paper proposes , a novel end-to-end framework that explicitly incorporates constraint projections into the training objective. By aligning the model’s learned dynamics with the constrained sampling process, the proposed method mitigates distributional shift induced by projection-based corrections, enabling high-quality constrained generation. The proposed approach is evaluated on three challenging real-world benchmarks, illustrating the generality and efficacy of the method.

Primary area:
generative models
Keywords:
Flow Matching, Constrained Sampling, End-to-End Learning

Reviews

Among:
Shape:
1234this paper, Top 49%
Figure 1. Mean review rating of the reviewed papers at ICLR 2027. This paper averages 2.33 over 3 reviews, above 51% of the others.
Reviewer_6BdqOpenReview ↗
Rating
2
Confidence
2
Informativeness
2
Presentation
2
Soundness
2
Read review
Paper summary

This paper proposes Constraint-Aware Flow Matching (CAFM), which adapts flow matching training to constrained sampling by computing a reconstruction loss after projecting the predicted clean endpoint onto the constraint set. An instantaneous endpoint estimate avoids differentiating through the full ODE trajectory, while a differentiable projection layer propagates the training gradient. Inference uses the PCFM sampler. Experiments cover constrained PDEs, wind fields and porosity-constrained microstructures.

Critical strengths
  • The comparison with PCFM shows useful gains with the same sampler. Burgers-IC standard-deviation MSE (SMSE) falls from 0.101 to 0.0575 while preserving feasibility, and wind coherence violation decreases by more than an order of magnitude. This suggests a practical benefit from training for the constrained sampler.

  • The ablations provide relevant evidence about the training choices. Appendix E shows that the Burgers-IC SMSE gain persists without additional ReFlow and that full ODE rollout offers no consistent advantage over the instantaneous endpoint estimate. Figure 3 also compares continued FM training with transitions to CAFM, supporting the use of pretrained models for adaptation.

Critical weaknesses
  • The methodological contribution is narrower than the paper claims. Chen et al. [1] already train a biomolecular diffusion model with losses computed on projected denoiser outputs, differentiate implicitly through the projection, and compare fine-tuning against inference-only projection. CAFM extends this principle to flow matching and different scientific constraint settings, which is useful, but the “first end-to-end training approach” claim should be qualified. Given this overlap, I view the main contribution as an empirical extension, whose value depends on the evidence for better constrained generation.

  • The sampler that CAFM is trained for is not, in general, consistent with the learned flow. The reverse update in Section 4 re-interpolates between the initial noise and the projected endpoint, which follows the ODE trajectory only when the learned paths are straight. With an independent coupling, each conditional path is straight but the marginal trajectories are not, and the justification in PCFM [2] seems to assume straight trajectories. Since the CAFM objective trains the projected endpoint prediction but not this update, I am not sure that the claimed alignment of training and sampling extends to the generated distribution. Part of the quality loss attributed to training-free projection may also come from the re-interpolation itself.

  • The evidence for improved sample fidelity is incomplete, particularly for microstructures. Appendix B.3 measures each image’s MSE against a fixed mean image, so reducing sample variation can improve the score. Nearest-training-image MSE also does not measure coverage. Without generated examples or structural statistics, I cannot tell whether CAFM produces a better distribution of plausible microstructures. For PDEs, pointwise means and standard deviations are informative, but leave joint space-time structure untested, and no generated samples are shown for any of the tasks.

  • Some conclusions go further than the results. Section 6.1 states that CAFM consistently improves accuracy and feasibility, yet Burgers-BC SMSE worsens from 0.000460 to 0.00105, and PDE feasibility is already at numerical precision for both CAFM and PCFM. On wind, the large coherence gain comes with a higher variance MSE (0.417 to 0.473), which the authors acknowledge. I think the claims should be limited to the metrics where the gains are clear.

[1] Chen et al., “Physically Valid Biomolecular Interaction Modeling with Gauss-Seidel Projection”, ICLR 2026.

[2] Utkarsh et al., “Physics-Constrained Flow Matching: Sampling Generative Models with Hard Constraints”, NeurIPS 2025.

Questions for authors
  • Under what conditions on the learned flow, for example straighter paths after ReFlow, does the combined procedure target the data distribution? It would also help to compare the sampler with the projection disabled against plain ODE sampling of the same model.

  • On Burgers-IC, the PCFM results in Table 1 are much weaker than those reported in [2] under what appears to be the same setup. Could you explain the difference, and does CAFM still improve on PCFM when PCFM is configured as in [2]?

  • Section 5.2 presents Eq. (10), the 1-RF variant in Appendix E, as the method, but the Burgers-BC results in Table 1 match the 2-RF variant in Table 3. Could you state which variant was used for each task? Since 2-RF is trained on endpoints from a frozen PCFM teacher, part of its gain may come from these samples rather than from the projected loss. Is the continued-FM curve in Figure 3 also evaluated with the PCFM sampler?

6 comments

Many thanks to the reviewer for their careful consideration and detailed review of our paper. In particular, we are appreciative of the reviewer's acknowledgment of our strong empirical results, and their mention of relevant ablations. We have revised the manuscript accordingly, with changes highlighted in red, and address your points below.

W1: Chen et al. already train a biomolecular diffusion model with losses computed on projected denoiser outputs.

Thank you for calling this work to our attention. We agree that their work provides precedent for differentiating through projected end points, and we have adjusted claims such as "first end-to-end training approach" accordingly. However, we would like to clarify the methodological contributions of CAFM beyond extending this principle to additional scientific applications.

Chen et al. apply a differentiable projection to coordinates predicted by a diffusion denoiser and compute the training loss on the projected coordinates. CAFM instead formulates the downstream objective through the endpoint obtained by integrating a learned velocity field (Section 5.1). Evaluating and differentiating this projected endpoint requires an ODE rollout, introducing a computational challenge specific to this formulation. Section 5.2 develops an efficient approximation using the instantaneous velocity, and Appendix E evaluates this approximation against full-rollout training and additional ReFlow.

As discussed through our connection to decision-focused learning, differentiable optimization layers are well established; we do not claim these layers themselves as a contribution. Chen et al. are relevant prior work demonstrating their use in generative modeling. However, CAFM’s methodological contribution lies in formulating projection-aware training through the endpoint of continuous flow dynamics and developing and evaluating a practical approximation to that objective. We have revised the related-work discussion and contribution statements to make this distinction explicit.

W2 & Q1: The sampler that CAFM is trained for is not, in general, consistent with the learned flow.

We acknowledge the distinction drawn between the learned ODE trajectory and the re-interpolation update. Our experiments adopt the existing PCFM sampling implementation, with our contributions targeting adjustments to the training objective.

While we agree that straight conditional training paths do not guarantee straight marginal trajectories, it is worth clarifying:

  • Improving the sampler is outside the scope of the paper. CAFM explicitly incorporates the projected endpoint prediction into training. Our claim of alignment concerns this correction; it does not establish that the complete re-interpolation procedure reproduces the learned ODE trajectory. The PCFM comparison holds the sampling procedure fixed and assesses the benefit of changing training within that procedure.

  • ReFlow ablations are informative. Appendix E measures agreement between the instantaneous endpoint prediction and the endpoint obtained by integrating the learned dynamics. Their close agreement provides empirical support for approximately straight trajectories in the tested settings, supporting the approximation underlying the sampler.

We are also actively running the requested ablation for the PCFM sampler with the projection disabled, and we will follow-up in the thread as soon as we have completed this experiment.

[1/3]

W3: The evidence for improved sample fidelity is incomplete, joint space-time structure untested, and no generated samples are shown for any of the tasks.

Thank you for highlighting the importance of distributional evaluation. Our PDE experiments include both MMSE and SMSE, assessing agreement with the reference mean and standard-deviation fields. These metrics have follow prior evaluations for this class of constrained generative modeling tasks [1,2,3]. However, during this period, we extended this evaluation with joint space-time statistics and distributional comparisons under matching constraint specifications.

TaskMetricPCFMCAFM (1-RF)CAFM (2-RF)
Burgers-ICSWD0.41460.35420.2957
RMS ratio; target = 12.7232.4001.782
Burgers-BCSWD0.06860.08870.0509
RMS ratio; target = 10.9920.9850.922

The reduction in sliced Wasserstein distance indicates closer agreement between the generated and reference solution distributions beyond pointwise statistics. The RMS ratio complements this comparison by assessing whether generated samples exhibit a similar overall level of variability to the training distribution.

We also updated the appendix with generated examples to assist in interpretation of the results (Figure 3).

For microstructures, the prescribed porosity changes the generation target relative to the training distribution. Evaluating constraint satisfaction and nearest training sample MSE assesses adherence to the requested design property while retaining proximity to observed microstructures. These metrics provide evidence relevant to inverse design, although they do not fully characterize coverage of the constrained target distribution, for which a matched reference ensemble is not directly available. In other words, the training data does not satisfy the inverse design constraints, so metrics for training distribution coverage are not directly applicable here.

W4: Some conclusions go further than the results.

We would like to clarify the intended meaning of accuracy in this statement. Appendix B.1 defines MMSE as evaluating "how accurately the model reproduces the expected or average behavior of the target distribution." CAFM improves this metric over PCFM across all three PDE benchmarks, supporting the stated improvement in accuracy. SMSE is a secondary metric measuring standard deviation, and we acknowledge that improvements in mean-field accuracy do not imply uniform improvements in uncertainty statistics.

The feasibility results depend on the setting. We agree that both CAFM and PCFM achieve numerical-precision constraint satisfaction on the PDE benchmarks, where the appropriate conclusion is improved mean-field accuracy while preserving feasibility. In wind modeling, CAFM improves both mean-field accuracy and achieved feasibility: coherence violation decreases from 4.59×10−34.59\times10^{-3} to 3.60×10−43.60\times10^{-4}, approximately 12.8×12.8\times, under the same sampling-time projection setup. This gain accompanies the variance-MSE tradeoff already acknowledged in Section 6.2. However, to improve clarity, we have adjusted the wording to avoid overstating the empirical improvement realized by CAFM.

Q2: On Burgers-IC, the PCFM results in Table 1 are much weaker than those reported in [2] under what appears to be the same setup.

The PCFM results reported in Table 1 follow the exact setup used in the officially released code base. Indeed, we flagged this discrepancy between our reproduced numbers and the reported PCFM numbers and reached out to the authors regarding this; we have not recieved any response to our request for additional details on the setup used to achieve their reported numbers. Our reported comparison therefore uses our reproduced PCFM baseline under the shared evaluation setup.

Q3: Could you state which variant was used for each task?

Thank you for identifying this ambiguity. We are happy to clarify!

Equation (10) defines the projected-endpoint objective used by both 1-RF and 2-RF. Thus, we frame the method as general to both cases. We use ReFlow selectively in the PDE setting (Burgers-BC) where it improved performance, rather than as a requirement for CAFM. Wind and microstructure experiments use the objective without additional ReFlow. We have added these details to Appendix B.

We agree that teacher-generated pairs may contribute to the performance of the 2-RF variant. Appendix E also evaluates CAFM without additional recoupling, showing that useful improvements can be obtained without it. For example, Burgers-BC 1-RF improves both MMSE and SMSE relative to the reported PCFM baseline.

Finally, in Figure 3, all evaluations use the PCFM sampler, so the comparison holds the sampling procedure fixed while changing the training objective. We have made this explicit in the caption.

[2/3]


Thank you, again, for your insightful feedback on our work. We hope that these responses have helped address your remaining concerns surrounding the paper, but we are happy to continue the discussion throughout the response period if there are any remaining questions!

[1] Utkarsh, Utkarsh, et al. "Physics-constrained flow matching: Sampling generative models with hard constraints." Advances in Neural Information Processing Systems 38 (2026): 160217-160252.

[2] Cheng, Chaoran, et al. "Gradient-free generation for hard-constrained systems." International Conference on Learning Representations. Vol. 2025. 2025.

[3] Zhang, Yi, Peng Wang, and Difan Zou. "Physics-informed distillation of diffusion models for pde-constrained generation." arXiv preprint arXiv:2505.22391 (2025).

[3/3]

Reviewer_5A84OpenReview ↗
Rating
3(+1)
Confidence
3
Informativeness
2
Presentation
2
Soundness
1
Read review
Paper summary

This paper proposes Constraint-Aware Flow Matching (CAFM), which incorporates a differentiable constraint projection into the training objective to better align training with constrained sampling at inference time. It uses a single-step endpoint approximation to avoid backpropagating through a full ODE rollout and evaluates the approach on PDE-constrained generation, wind-field modeling, and microstructure inverse design.

Critical strengths
  1. The paper addresses a relevant problem: applying constraint corrections during sampling can affect generation quality, and accounting for these corrections during training is a reasonable approach.

  2. The single-step endpoint approximation provides a practical way to incorporate differentiable projection into training without backpropagating through the full ODE rollout.

  3. The experiments cover three scientific application settings with different constraints. Using the same sampling setup for CAFM and PCFM also helps assess the effect of constraint-aware training.

Critical weaknesses
  1. I have concerns about whether the new loss function guarantees the correct generated distribution. Under linear interpolation, the single-step endpoint loss without projection can be viewed as a time-weighted version of the standard CFM loss, which does not change the ideal optimal velocity field. However, once the projection operator is introduced, it remains unclear whether the model can still learn an appropriate velocity field and whether the actual sampling procedure produces the target distribution. In particular, straight conditional interpolation paths do not imply straight trajectories under the learned marginal flow. The current argument that a single-step extrapolation can replace the full ODE rollout is therefore not sufficiently convincing.

  2. Differentiable projection requires additional iterative optimization, making computational cost an important concern. Appendix D reports some timing results, showing that a CAFM training step takes approximately 2.3 times as long as a standard FFM training step. However, it remains unclear how large this overhead is on the other tasks. I would like the authors to report total training time and inference time, and compare the methods under the same computational budget.

  3. The paper lacks comparisons with methods that account for constraints during training. The main baselines, including PCFM, PDM, and ECI, handle constraints during sampling. However, the paper’s central argument is that incorporating constraints during training improves training–inference alignment, so comparisons with existing constraint-aware training methods, such as Gauge Flow Matching and PolyFlow, would be valuable. Although their scope differs from that of CAFM, comparisons could be conducted on linear or convex constraint settings where both approaches apply. If such comparisons are not feasible, I would like the authors to explain the specific reasons.

  4. The experiments lack metrics that directly assess how well the generated distribution matches the target distribution and how diverse the generated samples are. Under constraints, could the improvement in MSE come at the cost of an overly concentrated generated distribution? The authors should include distributional distance metrics and compare the Pareto frontiers of different methods in terms of constraint violation and task performance.

Questions for authors

See in weakness.

7 comments

Thank you for your thoughtful review and, in particular, for your recognition of the both the practical value of our endpoint approximation and the diversity of our evaluation. We have included explicit revisions to the manuscript in light of your comments, highlighted in green, and we reply to your main points below.

W1a: Whether the new loss function guarantees the correct generated distribution.

Thank you for raising this distinction. We agree that introducing the projection can change the optimal velocity field. Indeed, this is intentional. The standard CFM optimum is defined for generation through the learned ODE, as opposed to the PCFM sampler which iteratively projects the endpoint and re-interpolates. Hence, this work argues that learning the standard CFM optimum does not necessarily result in optimal performance under the constrained sampler.

Decision-focused learning literature has explored this exact problem extensively. If the prediction-focused loss is minimized to zero (i.e. perfect prediction), the decision-focused loss will also be perfect as the optimization or projection step is exact. However, in practice, the prediction-focused loss will always contain some residual loss, and prediction accuracy before correction and output quality after correction need not favor the same model (Figure 2). CAFM therefore permits the unprojected prediction to change in ways that improve the projected output, while retaining supervision against the data endpoint.

W1b: Straight conditional interpolation paths do not imply straight trajectories under the learned marginal flow.

While we agree that straight conditional training paths do not guarantee straight marginal trajectories, it is worth clarifying that our ReFlow ablations speak to this exact question. Appendix E evaluates the agreement between the instantaneous endpoint prediction and the endpoint obtained by integrating the learned dynamics. Their agreement provides direct support for approximately straight trajectories in the tested settings.

Furthermore, these ablations empirically validate the single-step extrapolation as providing competitive or superior performance to the full ODE rollout while reducing training cost. We observe improved SMSE and identical (within the confidence intervals) or improved MMSE in both experiments.

W2: Computational cost [is] an important concern.

We agree that computational cost is an important practical consideration, which motivated both our adoption of endpoint approximation (Section 5.2) and warm-starting for training (Appendix D). Importantly, Appendix D focuses on the highest complexity constraints in the paper, the Burgers BC case, where we have substantially improved efficiency over full ODE rollout training:

DatasetMetricStandard FFMCAFMIntegrated-ODE
Navier StokesPeak Memory20.84 GiB20.85 GiB40.59 GiB
Navier StokesHours / 1,000 steps0.0630.1173.683
Burgers BCPeak Memory21.02 GiB34.15 GiB51.37 GiB
Burgers BCHours / 1,000 steps0.1780.39617.131
Burgers ICPeak Memory21.02 GiB27.97 GiB45.45 GiB
Burgers ICHours / 1,000 steps0.2050.52514.467

These results are reported without applying warm-starting and without dedicated effort to improving training code efficiency, which further reduces the walltime requirements of CAFM training.

Such runtime improvements fall outside the scope of our work; rather, the paper demonstrates that CAFM provides improved performance over FFM training. Figure 3 additionally compares continued FM training with transitions to CAFM, showing that the observed gains are not reproduced by simply continuing the standard training objective over the evaluated horizon.

[1/3]

W3: The paper lacks comparisons with methods that account for constraints during training.

Thank you for your suggestion! We agree that these are relevant works within the space of constraint-aware training, and we have expanded our related works section to include them in our discussion.

As you have acknowledged, the scope of Gauge Flow Matching and PolyFlow differs from our work. Their applicability depends on the geometry of the constraint set:

  • Gauge Flow Matching: This approach requires formalization of a bijective gauge mapping between the constrained domain and a simpler reference domain. This addresses compact convex sets, with extensions to particular nonconvex geometries (e.g., star-convex and geodesic-convex sets).

  • PolyFlow: This method embeds polyhedral constraints into its architecture. It does not, however, accommodate the nonlinear constraints explored in our experiments.

This limits their applicability across the selected benchmarks. Our PDE benchmark enforces nonlinear conservation constraints. Wind modeling imposes a nonlinear coherence constraint, while microstructure generation enforces an exact count of threshold pixels. PolyFlow's polyhedral formulation does not directly represent these compelete constraint sets. Applying Gauge Flow Matching would require establishing that each set admits a geometric construction and developing the corresponding mapping; GFM's support of star-convex and geodesic-convex sets does not directly support whether it can be extended to these more complex constraint sets.

Instead, our evaluation has centered on the comparison between CAFM and PCFM, which isolates the training changes with a shared sampling procedure. This comparison speaks directly to our claim that constraint-aware training improves generation under the same constrained sampling procedure.

W4: The experiments lack metrics that directly assess how well the generated distribution matches the target distribution and how diverse the generated samples are

We appreciate the suggestion. Our PDE evaluation follows prior work in reporting MMSE and SMSE, which assess agreement with the reference mean and standard-deviation fields [1,2,3]. However, we have since expanded our evaluation to assess sample diversity and distributional coverage.

TaskMetricPCFMCAFM (1-RF)CAFM (2-RF)
Burgers-ICSWD0.41460.35420.2957
RMS ratio; target = 12.7232.4001.782
Burgers-BCSWD0.06860.08870.0509
RMS ratio; target = 10.9920.9850.922

The reduction in sliced Wasserstein distance indicates closer agreement between the generated and reference solution distributions beyond pointwise statistics. The RMS ratio complements this comparison by assessing whether generated samples exhibit a similar overall level of variability to the training distribution.

Additionally, to address the requested feasibility-performance comparison, we evaluated fixed checkpoints across different inference-time projection budgets on Burgers IC. Constraint targets, initial noise, 200 sampling steps, and terminal correction were held fixed across methods.

IterationsMethodMMSESMSEIC residualConservation residualSampling time (s)
0PCFM / CAFM0.0506266 / 0.05016850.1191760 / 0.08496681.015e-10 / 1.071e-102.690e-02 / 1.623e-0226.11 / 26.17
1PCFM / CAFM0.0402993 / 0.03862060.0961683 / 0.05736693.108e-22 / 3.570e-223.503e-15 / 3.145e-15122.93 / 122.91
3PCFM / CAFM0.0409786 / 0.03947360.1029172 / 0.05767354.926e-27 / 5.225e-272.898e-15 / 2.814e-15319.30 / 311.85

CAFM outperforms PCFM across varying constraint correction budgets for both MMSE and SMSE. In practice, we also observe that only one Newton-style constraint correction is necessary per diffusion step, with feasibility not improving substantially when raising the budget to 3.

For the microstructure experiments, the enforced porosity changes the generation target relative to the training distribution. Hence, we assess constraint satisfaction and nearest training sample MSE assesses satisfaction of the design task while retaining proximity to observed microstructures. However, since the training data does not satisfy the inverse design constraints, metrics for training distribution coverage are not directly applicable here.

[2/3]


Many thanks for your careful review of our paper. We believe our response addresses each point raised in your review, and we hope that you would reconsider your assessment in light of these additional results and clarifications. We are also happy to answer any remaining questions during the discussion window!

[1] Utkarsh, Utkarsh, et al. "Physics-constrained flow matching: Sampling generative models with hard constraints." Advances in Neural Information Processing Systems 38 (2026): 160217-160252.

[2] Cheng, Chaoran, et al. "Gradient-free generation for hard-constrained systems." International Conference on Learning Representations. Vol. 2025. 2025.

[3] Zhang, Yi, Peng Wang, and Difan Zou. "Physics-informed distillation of diffusion models for pde-constrained generation." arXiv preprint arXiv:2505.22391 (2025).

[3/3]

Reviewer_gDKYOpenReview ↗
Rating
2(+1)
Confidence
2
Informativeness
1
Presentation
1
Soundness
2
Read review
Paper summary

This paper studied flow matching for constrained generation. The author proposed to merge projection inside the training loss to improve performance.

Critical strengths

The proposed loss function is interesting.

Critical weaknesses

The notation is confusing, and not properly defined. I expect a more rigorous treatment on notation, definition, and mathematical equations.

I think formulations like equation 3 should be put in an earlier section. For example, earlier sections talked about domain-specific constraints, but didn't rigorously formulate it. Also around line 170, I think it's better to just write $C = ${...}, define it as a set (e.g., the equality and inequality constraint).

Line 110: I am not sure if "constrained sampling" is a good name. For example, reflected flow matching (https://arxiv.org/pdf/2405.16577) can also be considered as constrained sampling, but I think what the author means here is some kind of projection-based approach.

Line 188: the ODESolve notation should be defined mathematically. Equation 1 is an ODE. I see no Δ\Delta (the author wrote s=t+Δs=t+\Delta) in Equation 1 or in lines 182-185. I might guess in line 182 the author means start from initial value (zt,t)(z_t, t) to simulate ode with the learned field, perhaps through some kind of discretization. But I don't know what line 185 is doing.

Line 192: I don't understand the sentence "Then, the learned flow is updated such that the end point enforces feasibility".

Lines 196-198: when discussing existing works I think the author should be more explicit. There are some works considered, for example, a narrower class of set C, e.g., convex constraints. See for example https://openreview.net/pdf?id=vxq1OnaAMq What I mean is: since there are existing works on diffusion/flow, where generation feasibility can be guaranteed, it might not be good to say "existing diffusion and flow matching objectives are designed to...remaining misaligned with the downstream task of sampling from pCp_C".

Line 200: what is "deployment objective"? I don't think this term is defined. And I don't really get the point of figure 1b from the description in line 200-201. What I can guess is, the author means, when training procedure is not designed to know the constraint set, the output infeasibile point might lie in some regions, so that after projection, it is not good. And I don't understand (in Figure 1) what is “high-density data region", and in particular why it is not in the blue feasible set? Since the data, by design, should be feasible, I thought the data region should be a subset of feasible set.

The author should rigorously define which norm they are using. It seems to me that somewhere there should be a expectation but that was merged inside the norm (e.g., the flow matching loss).

From the perspective of flow matching, I think directly introduce the projection based loss is ok and intuitive, so I don't think providing intuition by introducing DFL is a better choice. Also, the last == in line 266 is confusing. If c^=z1^\hat c = \hat{z_1}, i.e., if they are the same thing why introduce one extra notation? And in line 266 there should be a expectation?

It's better to have a (easy to read) section discussing the novelty of this paper.

Questions for authors

See weakness.

In short, my main concern is the presentation, mathematical rigorous, and novelty. It's better to discuss and compare existing works (for readers not familar with those existing papers that author discussed).

It's possible that (as I mentioned I feel some parts difficult to understand) I didn't fully understand some part of this paper. If the rebuttal indicate my understanding is wrong, I might adjust my score based on the correct understanding.

8 comments

We appreciate the reviewer's consideration of our work, especially your interest in our proposed training objective. Your comments identified several places where our presentation could be strengthened to better communicate the formulation and contribution. We have revised the manuscript accordingly, with changes highlighted in blue, and address your points below.

Notation is confusing, and not properly defined.

We have strengthened the mathematical definitions and clarified the distinction between training states, predicted endpoints, and sampling updates:

  • Constraint set: Section 4 now explicitly defines C={x∈Rd:h(x)=0,g(x)≤0}C=\{x\in\mathbb R^d: \mathbf h(x)=0,\mathbf g(x)\leq0\}, with inequalities interpreted component-wise.

  • ODE solution operator: We now mathematically define ODESolve⁡(x,f,a,b)\operatorname{ODESolve}(x,f,a,b), including its initial condition and integration interval.

  • Sampling times and reverse update: We replaced the Δ\Delta-based description with t<s≤1t<s\leq1 and wrote the interpolation explicitly as zs=(1−s)z0+sz1z_s=(1-s)z_0+sz_1.

  • Norms and expectations: Section 3 now specifies Euclidean norms and distinguishes per-example losses, which do not include an expectation, from their expected training objectives.

Regarding the placement of Equation (3), we retain the formal constrained-generation setup immediately after the flow-matching preliminaries, allowing the model and trajectory notation to be defined first. We are open to deferring the related works to position them later in the paper if you believe that would improve the discussion, but we'd prefer to defer the discussion around Equation (3) until after the introduction and preliminaries as it constitutes its own section.

I am not sure if "constrained sampling" is a good name.

Our intended definition encompasses methods that impose constraints during sampling, rather than exclusively projection-based approaches. However, we agree that the current naming convention is too broad; to make this scope clearer, we renamed the category "Sampling-Time Constraints." Our argument concerns the mismatch arising when a standard training objective is retained while the sampling procedure introduces feasibility corrections.

I don't understand the sentence "Then, the learned flow is updated such that the end point enforces feasibility".

We agree this could be more clear and have replaced this sentence. Here, we are referring to the reverse update which interpolates backward in time. The revised text explains that the update interpolates from the projected endpoint toward the original noise state to obtain the next sampling state.

When discussing existing works I think the author should be more explicit.

Thank you for the suggestion. We have narrowed the introduction’s statement to "many prominent" constrained sampling approaches, rather than asserting that all such approaches are training-free. We also revised Section 5 to refer specifically to standard diffusion and flow-matching objectives adopted by sampling-time methods, and removed the broad statement that the challenge remains unaddressed throughout constrained generative modeling.

What is "deployment objective"? Why is the high-density data region not inside the feasible set in Figure 1?

We replaced the ambiguous deployment-objective wording with an explanation of how feasibility correction influences the sampled distribution. Generally, your interpretation is correct: the projection step can degrade the solution quality, especially since the model was trained agnostic to the constraint correction. The updated phrasing clarifies this further.

We also expanded the Figure 1 caption to distinguish the original data distribution from the requirements of a particular generation task. For example, a dataset can contain valid PDE solutions under different initial or boundary conditions, while the feasible set specifies particular conditions. Thus, the original data distribution need not lie entirely within that task-specific set. In other words, the data distribution may often include samples that are feasible for their own set of conditions, but no one sample is feasible for all sets of conditions. Hence, there may be high-density regions that are not feasible for a given task. Inverse-design tasks realize a similar mismatch between the general data distribution and a specific targeted property.

[1/2]

Directly introducing the projection-based loss is intuitive. Why introduce DFL and the additional notation c^=z^1\hat c=\hat z_1?

We agree that the projected loss admits a direct interpretation. The DFL formulation serves an additional purpose; it identifies the predicted clean endpoint as the parameter of a downstream optimization problem and its projection as the resulting decision. This connects constrained generative training to established principles in decision-focused learning and differentiable optimization. Furthermore, we believe this framing is generally valuable as it opens future opportunities to connect principles from DFL to constrained generative modeling.

We have added an explicit explanation of this correspondence. Additionally, we'd like to clarify that the equality c^=z^1\hat c=\hat z_1 specializes the general DFL parameter to the more specific flow-matching setting. Hence, these terms serve distinct purposes in the formalization of DFL and flow matching.

It is better to have an easy-to-read discussion of novelty.

Thank you for the suggestion. We rewrote the contributions paragraph to identify the methodological components explicitly. The revised paragraph discusses the DFL formulation of projected-endpoint training, the integrated-flow objective and efficient instantaneous approximation, the ReFlow ablations, and the evaluation using shared constrained sampling procedures.


We hope these revisions address the presentation, notation, and contribution concerns underlying your assessment. Given these concrete changes targeted each concern raised in your original assessment, and your statement that a clearer understanding could lead you to adjust your interpretation of the work, we respectfully ask you to reconsider the current assessment in light of these changes. If any concern remains after these clarifications, we are more than happy to engage in further discussion during this period. Many thanks!

[2/2]

Thanks for improving the presentation.

  • Still, I am confused about Figure 1. I am not from a PDE background, and the sentence "For example, valid PDE solutions may satisfy different initial or boundary conditions from those prescribed for the current task" is not clear to me (for example what is the current task?). In my understanding, the data itself, if valid, should satisfy the constraint, so in my understanding the data region itself should be feasible.

On the other hand, the proposed approach, although claimed as a general approach for constrained generative model, but it seems to be designed for PDE problems (or some physics related problems).

  • But as a general approach, I think a comparison against methods like [Gauge flow matching, Mirror flow matching, Polyflow, Reflected flow matching], for example on some synthetic Gaussian mixture data constrained on some polytope, should be conducted. I think such a comparison would better demonstrate whether the proposed approach applies more broadly (as another reviewer also mentioned).

Comments on the paper

1 comment
Area_Chair_1fRjOpenReview ↗

Reminder: Follow-up Feedback Before Discussion Closes

Dear reviewers,

With the discussion period nearing its end, I would appreciate it if you could review the authors’ responses and revised manuscript and share their follow-up feedback.

Please indicate which concerns have been addressed and which remain, and update your assessment and score where appropriate.

Thank you for your time and contributions to the discussion.

Best, AC

From OpenReview, CC BY 4.0. Authors as signed there; shown unedited.

Related papers

Loading the map…

Discussion (0)

Sign in to comment.