acceptodds
Withdrawn from ICLR 2027

Promoting Determinism Physics-Conditioned Video Generation via Convergent Flow

Withdrawn from ICLR 2027, so left out of the paper lists and the map.

Abstract

Physics-conditioned video generation is emerging as a promising paradigm for learning world models that can predict physically plausible futures from initial observations and explicit physical constraints. A distinctive property of these tasks is that the conditioning signals—such as the initial scene, object states, velocities, masses, and interaction parameters—often strongly constrain the future evolution, leading to conditional target distributions that are substantially more concentrated than those in generic video generation. Nevertheless, existing generative approaches typically retain a condition-independent standard Gaussian source. We argue that this default choice is poorly matched to strongly conditioned generation in two respects: its scale can be substantially misaligned with the remaining conditional variation, and its unstructured randomness fails to exploit information already available in the conditioning signal, imposing unnecessary learning burden on the generative flow. To address these limitations, we introduce SCFM, a task-aware source construction for conditional flow matching that adapts the source to both the residual uncertainty and the structural information provided by the condition. By starting the transport from a more informative yet non-degenerate distribution, SCFM enables the flow model to focus its capacity on modeling the remaining conditional dynamics rather than reconstructing structure already specified by the input. Beyond controlled physical supervision, we further introduce VPhysBench, a real-world benchmark for physics-conditioned video prediction, to evaluate whether such source design principles transfer to realistic physical interactions. Experiments on VPhysBench demonstrate that SCFM achieves better performance across multiple distributional metrics with greater learning efficiency, highlighting source distribution design as an important and underexplored dimension of efficient conditional video generation.

Primary area:
generative models
Keywords:
flow-matching, physical video generation model

Reviews

Among:
Shape:
1234this paper, Bottom 2%
Figure 1. Mean review rating of the reviewed papers at ICLR 2027. This paper averages 1.33 over 3 reviews, above 2% of the others.
Reviewer_NyhXOpenReview ↗
Rating
2
Confidence
4
Informativeness
2
Presentation
2
Soundness
2
Read review
Paper summary

This paper proposes SCFM, a source-adaptive conditional flow matching framework for physics-conditioned video generation. Motivated by the mismatch between standard Gaussian initialization and strongly constrained conditional target distributions, SCFM constructs condition-dependent source distributions by predicting physical outcome probabilities, branch-specific means, and within-branch variances. The proposed framework aims to reduce unnecessary transport complexity while preserving uncertainty across different physical outcomes.

Critical strengths
  • A meaningful and well-motivated problem formulation. The paper identifies an important mismatch between conventional Gaussian source distributions and the concentrated conditional target distributions encountered in physics-conditioned video generation. The observation that physically constrained future predictions may exhibit multiple narrow outcome modes provides a compelling motivation for adapting the source distribution to both outcome uncertainty and within-branch variation. This perspective highlights a potentially valuable direction for improving the efficiency of conditional flow matching beyond conventional conditioning mechanisms.
  • Promising empirical improvements supported by component-level ablations. SCFM demonstrates consistent improvements over standard Gaussian initialization, CPD, and CSFM across several distributional metrics under the evaluated training budget. The experiments also report improvements in physical trajectory alignment and visual similarity. Furthermore, the ablation studies provide useful evidence supporting the importance of branch-aware source construction and variance adaptation. These results suggest that incorporating structured conditional uncertainty into the source distribution can improve learning efficiency.
Critical weaknesses
  • Limited evidence of generalization and concerns regarding evaluation independence. The experimental evidence is insufficient to establish the generalizability of the proposed framework beyond the specific evaluation setting. Importantly, the authors explicitly acknowledge in Section 6.1 that the test split was inspected during method development, raising concerns about evaluation independence and potential bias in the reported results.
  • Strong reliance on repeated observations and outcome clustering limits practical applicability. SCFM relies on repeated future observations under the same recorded physical condition to estimate branch probabilities, branch-specific means, and within-branch variances. However, obtaining multiple realizations under sufficiently consistent physical conditions may be difficult or prohibitively expensive in practical video generation scenarios. Furthermore, the framework depends on clustering-derived outcome assignments, which introduces an additional assumption regarding the identifiability and consistency of physical outcome branches. The reliability of the estimated source distributions may therefore depend substantially on the availability of repeated observations and the quality of the clustering procedure.
  • Unresolved references and insufficient manuscript proofreading. The manuscript contains several unresolved cross-references, such as "Appendix ??" in Lines 82 and 107, as well as similar issues throughout the paper. These errors undermine the overall presentation quality and make it difficult to locate supporting theoretical derivations and implementation details. The authors should carefully proofread the manuscript and ensure that all cross-references are correctly resolved.
Questions for authors
  • Could the authors clarify how the inspection of the test split during method development may have influenced model selection or experimental design? Results on a strictly held-out evaluation set would help establish the reliability of the reported improvements.
  • How sensitive is SCFM to the number of repeated observations per condition and the quality of clustering-derived outcome assignments? In particular, can the framework maintain its advantages when only a limited number of future observations are available?
  • Inconsistency between Tables 1 and 2: Table 1 reports an Energy Score of 0.1273 ± 0.0012 for SCFM under PhyParam-style conditioning after 1,000 training steps, whereas Table 2 reports 0.1193 ± 0.0012 for the full model under the seemingly identical setting. A similar discrepancy appears under PhyCo-style conditioning (0.1270 versus 0.1170). Could the authors clarify whether these results correspond to different experimental configurations, evaluation protocols, or training settings? If the settings are identical, what explains the discrepancy?
Reviewer_P6jUOpenReview ↗
Rating
1
Confidence
3
Informativeness
1
Presentation
2
Soundness
2
Read review
Paper summary

SCFM employs "flow matching," starting from a distribution tailored to potential physical outcomes. It predicts the likelihood of each outcome, the central location of its source distribution, and the degree of permissible variation around that center. In a subsequent stage, these predicted parameters are held fixed, while a shared model learns to generate the final video. The paper also introduces VPhysBench, a benchmark comprising real-world recordings captured multiple times under identical conditions; furthermore, it provides a theoretical explanation for the phenomenon of "source variance collapse." Experimental results demonstrate that this method achieves superior performance in terms of distributional scores across two different experimental settings.

Critical strengths
  1. The paper does not assume that physical conditioning always determines a unique future; it focuses on separating uncertainty across qualitative outcomes.
  2. Experiments show its advantages in physical scenarios.
Critical weaknesses
  1. Only show advantages in self-designed physical benchmark.
  2. The main results are inconsistent across tables. Table 1 reports full-model ES values of 0.1273/0.1270, while Table 2 reports 0.1193/0.1170 under the same setup. 3.The test set was used during method development, as the paper acknowledges. 4.Supporting material is missing from the provided manuscript.
Questions for authors

1.Which full-model ES values are correct: Table 1 or Table 2? 2.How was the test split used during development, and if it was not used, which decisions were influenced? 3.What are the performances of the model on other physical benchmarks?

Reviewer_q7MkOpenReview ↗
Rating
1
Confidence
2
Informativeness
1
Presentation
1
Soundness
1
Read review
Paper summary

The paper proposes SCFM (Possibly: source-conditional flow matching? The acronym is never defined), an approach for improving video generation by replacing the typical Gaussian base distribution with a source distribution that depends on provided physical parameters. During training, multiple source videos with the same starting conditions are clustered into distinct "branches"; then, predictor models are trained to estimate branch probabilities, as well as means and variances for each branch. These predicted means and variances are used to parameterize the base distribution for a conditional diffusion model. The authors evaluate the proposed approach on a benchmark they introduce, called VPhysBench, and demonstrate that SCFM outperforms models trained with various baseline source distributions.

Critical strengths

None

Critical weaknesses
  1. Poor/missing justification of the proposed approach.

The paper assumes that physical parameters are provided at inference time and does not explain how they would be obtained for real-world video, or analyze how sensitive the proposed method is to noisy or poorly estimated parameters. Training also requires multiple source videos from the same initial conditions for branch clustering, which are not often available in existing real-world datasets, and the paper does not address how this approach could be extended for datasets without that structure.

  1. The paper contains dense, poorly written prose which is difficult to parse and reminiscent of AI writing, making it somewhat difficult to understand the proposed approach. In addition, the appendixes referenced in the paper, which would describe the method's theoretical justification as well as the majority of the implementation details for the experiments, are entirely missing.
Questions for authors
  1. Is it possible to extend this method to general real-world video in a sensible way, when physical parameters are not provided or when each physical scenario only corresponds to a single training video? Is the proposed method applicable only to the proposed VPhysBench, or could a justification for broader applicability be provided?

  2. Why does Table 1 report ES scores of 0.1273 and 0.1270 for SCFM, while Table 2 reports 0.1193 and 0.1170, despite the fact that both tables should be test set performance after 1000 steps for the proposed method?

  3. Is the learning efficiency comparison in figure 3 indicative of the actual, end-to-end training cost of the proposed approach, including the cost of training branch, mean, and variance predictors? Section 6.3 reports that source-only sampling already achieves an ES of 0.1408; is the performance gain of the proposed method actually attributable to more efficient learning by the flow model, or can it mostly be attributed to the separately trained predictors? Could controlled comparisons between end-to-end GPU-hour or FLOP counts for each method be provided?

  4. How does training beyond 1000 steps affect the relative performance of the proposed method with the baseline approaches? In other words, is the effect of the proposed method limited to only faster initial convergence, or is its advantage maintained in longer training runs?

  5. How was clustering for branch labeling performed? How was K selected? What was the distance metric / in what feature space was clustering performed? What clustering algorithm was used? Do the clusters actually qualitatively correspond to different modes of physical outcomes? Was clustering performed globally or independently per physical scenario?

From OpenReview, CC BY 4.0. Authors as signed there; shown unedited.

Related papers

Discussion (0)

Sign in to comment.