acceptodds
Under review as a conference paper at ICLR 2027

Risk Decomposition for Hallucination-Aware Video Sampling

14 views

Abstract

Despite achieving high visual fidelity, video models still suffer from physical hallucinations, limiting their reliability as world simulators. Existing methods leverage model-internal or external signals to improve physical plausibility, but their roles and limitations require a unified explanation. We propose a two-dimensional physical-hallucination risk framework that defines low-density sampling risk and high-density drifting risk. The first risk captures generations assigned low likelihood by the video model, for example, a ball suddenly teleporting despite the model favoring continuous motion, while the second risk captures physical inconsistencies in high-likelihood generations, for example, a ball bouncing progressively higher without additional energy. Our analysis explains how model-internal signals steer sampling toward higher-density regions of the learned distribution to address low-density sampling risk. However, higher model likelihood does not guarantee physical plausibility, motivating external guidance to address high-density drifting risk. Based on this framework, we develop Hallucination-Aware Sampler (HA-Sampler), which applies external physical guidance within latent refinement to generate physically plausible videos, and HA-Scorer, which derives a likelihood-based signal from the generator itself for evaluating generated videos using continuous normalizing flow (CNF), with coarse ODE integration and finite-difference divergence estimates. Experiments on Physics-IQ and VideoPhy benchmarks support our theoretical analysis and demonstrate that our methods outperform the compared sampling and scoring SOTA baselines.

Primary area:
generative models
Keywords:
Video generation, Physical hallucinations, Flow matching

Reviews

Among:
Shape:
1234this paper, Bottom 27%
Figure 1. Mean review rating of the reviewed papers at ICLR 2027. This paper averages 2.00 over 3 reviews, above 27% of the others.
Reviewer_o6PwOpenReview ↗
Rating
1
Confidence
3
Informativeness
1
Presentation
2
Soundness
1
Read review
Paper summary

This paper proposes a conceptual framework that splits hallucination in video generation into two kinds of risk: 1) low-density sampling risk and 2) high-density drifting risk. Then training-free methods are proposed, aiming to reduce the risks. HA-Sampler adds gradient guidance from an external loss (WMReward's score) to latent refinement. HA-Scorer estimates a video's likelihood under the generator through a continuous-normalizing-flow computation for candidate selection. Experiments are conducted on Physics-IQ and VideoPhy.

Critical strengths
  1. The method works with existing video models without retraining i.e. in a training-free manner.

  2. It gives some consistent, although marginal sometimes, improvements on two benchmarks and different backbones.

Critical weaknesses
  1. The two-risk formulation is like storytelling. It is highly conceptual, and too general to be wrong. It is not directly connected with the proposed method. This storytelling complicates the paper, and does not contribute much.

  2. The proposed method is not specific to physical hallucinations. HA-Sampler is a standard combination of latent refinement and gradient guidance. HA-Scorer is also a general method. The only physics-related element is the external loss from WMReward. It is unclear why the method is presented as a solution to physical hallucination rather than as a general sampler. If considered as a general sampler, the comparison coverage and novelty may be further limited.

  3. The experimental results are not very convincing. Many existing methods are missing in the comparisons. Also the method relies on WMReward loss, but only shows a marginal gain over it. Visual quality, diversity and compute cost are not reported.

  4. Section 3.4’s intention is unclear. This may be a writing issue as it is hard to understand.

  5. (Minor) There are some sentences that read kind of AI-flavoured.

Questions for authors

The authors can respond to my comments above.

Reviewer_bPWcOpenReview ↗
Rating
3
Confidence
3
Informativeness
2
Presentation
3
Soundness
3
Read review
Paper summary

This paper introduces a two-dimensional framework for physical hallucination risk. This paper proposes HA-Sampler for guided latent refinement and HA-Scorer for likelihood-based candidate selection. Its strengths include training-free applicability and improvements across two sampling backbones. However, the evaluation also lacks matched-compute comparisons and several closely related baselines. Overall, the results support the practical promise of the methods, but stronger evidence is needed to substantiate the proposed risk interpretation.

Critical strengths
  • The distinction between low model likelihood and physical inconsistency in high-likelihood samples is clear and useful. It explains why reinforcing a generator’s own preferences may help without guaranteeing physical correctness, motivating the combination of internal refinement and external guidance.

  • The proposed combination has concrete experimental support. HA-Sampler improves the overall Physics-IQ score on both Wan2.2 and SkyReels, while the controlled ablation shows that combining refinement and guidance performs better than either component alone.

Critical weaknesses
  • On the method part, the proposed risk decomposition is more of a conceptual partition than a quantitative decomposition of physical hallucination risk. Eq. (3) partitions the hallucination set, but the two surrogate risks in Eq. (5) and (7) are not shown to bound or recover hallucination probability. Appendix A analyzes an idealized continuous-time Langevin process, which does not establish risk reduction for the implemented finite-step sampler.

  • On the experimental side, the comparisons do not control for the additional computation introduced by refinement, gradient guidance, and likelihood estimation. Besides, no end-to-end runtime comparison is provided. Several gains over WMReward are small.

Questions for authors
  • Matched-compute comparison: Can you report end-to-end latency and peak memory, including gradient computation, VAE decoding, external evaluation, and scoring all eight candidates? At the same computational budget, does HA-Sampler outperform additional P&P iterations, and does HA-Scorer outperform using a cheaper scorer with more candidates?

  • Physical improvement versus trade-offs: Why does HA-Sampler improve aggregate Physics-IQ while reducing Spatiotemporal IoU and MSE Score on Wan2.2? Can you report semantic adherence, diversity, and quantitative motion magnitude to clarify what improves and what is sacrificed?

Reviewer_ZpoWOpenReview ↗
Rating
2
Confidence
3
Informativeness
2
Presentation
2
Soundness
2
Read review
Paper summary

The paper introduces HA-Sampler, training-free sampling method for physically plausible video generation. The authors argue that existing model-internal guidance methods, such as P&P, generate model-preferred stable outputs, but these outputs are not guaranteed to be physically plausible. External guidance can improve physical scores, but does not necessarily keep samples in regions well supported by the model. HA-Sampler therefore combines model-based refinement with external physical guidance to address both limitations. Specifically, it repeatedly adds noise and denoises the latent, using WMReward to guide the added noise toward better physical outcomes. The authors also introduce HA-Scorer, which estimates how likely a video is under the generator and uses this score to rank candidate videos.

Critical strengths
  1. Clear motivation. The paper clearly explains why existing methods (model-internal / external only) can be failed. This motivates combining model-internal refinement with external physical guidance.

  2. Useful model-based scoring. The paper develops a scorer based on VDM. The additional checks of numerical accuracy and ranking stability support the proposed approximation.

Critical weaknesses
  1. Unclear link between motivation and scorer. The paper argues that model-internal only method does not guarantee physical plausibility, however, HA-Scorer ranks videos using likelihood alone. It may favor model-preferred stable outputs.

  2. Limited evaluation of practicality and generalization. HA-sampler combines gradient updates and iterative refinement, and the HA-Scorer requires 48 NFEs per candidate. This suggests considerable inference overhead, while gradient computation may also increase memory use compared with refinement alone. Runtime and peak-memory comparisons are needed to assess the practical cost. The evaluation focuses on Physics-IQ and VideoPhy, with displayed examples mostly involving isolated object interactions. Broader tests on complex human motion and interactions, as explored in Self-Refining Video Sampling, would help establish generalization. Proposed method may show lower sample diversity and static motion than base ODE sampling, therefore, non-cherry picked multiple samples can clarify this concern.

Questions for authors
  1. Could the authors provide inference cost (runtime, peak memory) comparison?

  2. See weaknesses.

From OpenReview, CC BY 4.0. Authors as signed there; shown unedited.

Related papers

Loading the map…

Discussion (0)

Sign in to comment.