acceptodds
Withdrawn from ICLR 2027

From Pixels to Initial States: Geometry and Propagation of Source-Induced Sensitivity in Diffusion and Flow Models

Withdrawn from ICLR 2027, so left out of the paper lists and the map.

Abstract

Small source-image perturbations can substantially alter image-to-image diffusion and flow outputs. Output differences alone do not reveal the contributions of initial-state displacement magnitude, downstream growth, direction, or their interactions. Contrary to a simple explanation based on displacement magnitude and downstream growth, we argue that source-induced sensitivity requires separating the geometry of the induced initial-state change from its subsequent propagation. We examine this question at the source-conditioned initial state, holding stochastic inputs, conditioning, and generation settings fixed within each clean–perturbed pair. Across SD1.5, SDXL, and FLUX, native sources induce larger initial displacements than tested generic and optimized projection controls; direct interventions show larger mean responses at larger tested radii and different responses across directions at a common prescribed radius. Our primary bounded-source comparison uses 50 FFHQ images and approximately matches native and transferred initial-state radii before output evaluation. Native mean DINOv2 dissimilarity from paired clean outputs is larger in both SDXL and FLUX. Yet native mean normalized trajectory displacement is smaller than transferred in SDXL but larger in FLUX, yielding opposite trajectory-displacement orderings. This cross-target pattern persists under fourfold refinement of the executed integration schedules. The same output ordering thus arises under distinct downstream propagation regimes. A separate 200-image non-face COCO cohort extends the unmatched source-to-state and output comparisons without repeating matched-radius interventions.

Primary area:
generative models
Keywords:
Generative Models, Diffusion Models, Flow Matching, Image-to-Image Generation, Source-to-State Transfer, Conditioning Sensitivity, Latent Geometry, Causal Interventions

Reviews

Among:
Shape:
1234this paper, Bottom 27%
Figure 1. Mean review rating of the reviewed papers at ICLR 2027. This paper averages 2.00 over 3 reviews, above 27% of the others.
Reviewer_XxftOpenReview ↗
Rating
2
Confidence
3
Informativeness
2
Presentation
1
Soundness
2
Read review
Paper summary

Image-to-image generative models are highly sensitive to small perturbations in their input images. However, it remains unclear whether this sensitivity primarily arises from the magnitude of the initial latent displacement, its direction, or error amplification along the subsequent generation trajectory. This paper proposes an intervention framework based on the initial state. It decomposes the initial latent changes caused by input perturbations into magnitude and direction, then directly modifies the initial state to study their respective effects on the output. The findings show that sensitivity cannot be explained solely by the magnitude of the initial perturbation; its direction also matters. Moreover, a larger final output difference does not necessarily imply greater trajectory separation during generation.

Critical strengths
  1. The experimental design is relatively rigorous.
  2. The paper reports results that challenge a unified explanation without overstating its findings.
Critical weaknesses

The empirical findings are valuable, but is their significance sufficient? Can they inform future scientific practice?

Questions for authors

The main text and appendix acknowledge missing experimental recipes, missing records, and the lack of verified full-model replay. Why have these issues not been resolved?

Reviewer_dSZ8OpenReview ↗
Rating
2
Confidence
4
Informativeness
1
Presentation
2
Soundness
1
Read review
Paper summary

This paper investigates how small source-image perturbations affect image-to-image diffusion and flow models. It introduces an experimental framework to examine the roles of initial latent displacement magnitude, direction, and subsequent trajectory propagation. Experiments on SD1.5, SDXL, and FLUX suggest that initial displacement magnitude alone cannot explain output sensitivity, while trajectory displacement and semantic output differences do not always follow the same ordering.

Critical strengths

None

Critical weaknesses

The paper primarily confirms relatively expected effects of perturbation magnitude and direction, without establishing substantially deeper scientific insights. While the experimental analysis is systematic, it remains unclear what new understanding can be derived or how these findings could inform practical robustness improvements. This limits the significance of the contribution.

The analysis remains largely empirical, showing differences in sensitivity without adequately explaining their underlying causes. In particular, the roles of perturbation direction, text conditioning, and intermediate denoising dynamics are not sufficiently disentangled. As a result, the evidence supports the observed phenomena but provides limited insight into why they occur, weakening the paper's central explanatory claims.

Questions for authors

What deeper scientific insights do these findings offer beyond confirming the expected effects of perturbation magnitude and direction?

The analysis remains largely empirical. What explains why certain perturbation directions are more influential than others?

It would be helpful to see interventions at intermediate denoising steps to better understand perturbation propagation. Can these insights also help predict sensitivity patterns or inform robustness strategies?

How sensitive are the identified directions and main findings to different text prompts and guidance settings?

Reviewer_A9rGOpenReview ↗
Rating
2
Confidence
2
Informativeness
2
Presentation
1
Soundness
2
Read review
Paper summary

The paper studies why bounded (L∞ ≤ 8/255) source-image perturbations change image-to-image outputs in SD1.5, SDXL and FLUX. It treats the source-conditioned initial state as an intervention interface and splits displacement into radius and direction. Native perturbations give larger initial-state displacement than the tested random controls (P1). The primary result (P4) matches radius at ρ = 0.9 on 50 FFHQ images, and native beats transferred perturbations in DINOv2 dissimilarity (+0.0089 SDXL, +0.0081 FLUX). A trajectory-separation measure G is larger for native in FLUX but smaller for native in SDXL, which the authors read as a failure of a shared displacement-growth explanation. The paper is dense, heavily hedged and full of abbreviations (P1–P4, PD/ENC, ρ, FR-2). The structure is good, but it takes effort to follow.

Critical strengths
  • The experiments are carefully designed. Noise and conditioning are fixed per pair, the primary family is declared in advance with a fresh holdout, and exploratory analyses are labeled.

  • Unfavorable results are reported (a slightly negative, non-significant SDXL CLIP-image contrast; shuffled and isotropic directions beating native in SDXL state-space edits), so the claims are easy to audit.

  • Separating magnitude, direction and propagation is a sensible framing, and the random-sign control cleanly shows source RMS alone does not explain the state displacement.

Critical weaknesses
  1. The effects are small (about 0.008 DINO difference at N = 50), and only one primary radius and one primary endpoint are used. The SDXL CLIP-image contrast is null (slightly negative).

  2. Architecture and execution settings are confounded, so the cross-model difference in trajectory behavior can’t be attributed to architecture (the authors acknowledge this).

In general, the practical impact is unclear. It isn’t evident how this helps protection, editing or robustness. Also the paper is hard to read, with a very long appendix and many hedges.

Questions for authors
  1. What is the practical takeaway for building defenses or robust editors?

  2. Does the trajectory metric predict anything about the outputs, e.g. is it correlated with per-image DINO dissimilarity? If not, what does the sign flip tell us?

  3. Can you show a feature-space or decoded-state separation along the trajectory (not just latent RMS), and does the SDXL ordering still flip?

  4. What do the lower radii (ρ = 0.5, 0.7) look like, and how much does the result depend on ρ = 0.9?

  5. Can you add a third model or a second model pair to separate architecture from execution settings?

From OpenReview, CC BY 4.0. Authors as signed there; shown unedited.

Related papers

Discussion (0)

Sign in to comment.