acceptodds
Under review as a conference paper at ICLR 2027

From Seeds to Semantics: Measuring Semantic Accessibility in Deterministic Diffusion Models

Abstract

Diffusion models generate samples through a sequence of learned denoising steps, and recent work has studied how semantic structure appears along this sampling process. We study this question in deterministic samplers by measuring semantic accessibility: how much information about a final semantic property (e.g. an image class label or attribute) can be extracted from the seed and intermediate states along the trajectory that produces the sample. Using DDIM sampling, for which each initial noise seed maps to a single trajectory and final image, we train a separate classifier (a probe) at several points along the trajectory to predict a semantic property of the final image. We measure how well such property can be predicted from the state at that point using using top-1 accuracy and normalized mutual information. Across MNIST, Fashion-MNIST, CIFAR-10, and CelebA, class labels and image attributes can be predicted above chance from the initial noise seed, and along DDIM trajectories, this accessibility exceeds matched-noise forward baselines. We observe that accessibility is substantially higher in groups of trajectories whose final images are classified with high confidence than in those classified with low confidence. These measurements provide a quantitative view of when semantic properties can be recovered along deterministic generation.

Primary area:
generative models
Keywords:
diffusion models, deterministic sampling, semantic accessibility, mutual information, interpretability

Reviews

Among:
Shape:
1234this paper, Bottom 27%
Figure 1. Mean review rating of the reviewed papers at ICLR 2027. This paper averages 2.00 over 2 reviews, above 27% of the others.
Reviewer_DsiGOpenReview ↗
Rating
2
Confidence
3
Informativeness
1
Presentation
3
Soundness
2
Read review
Paper summary

This paper investigates whether semantic properties of images generated by deterministic diffusion samplers can be directly predicted from the initial noise and intermediate states. The authors conduct three sets of experiments to demonstrate that deterministic sampling leads to greater semantic predictability and to examine how this predictability relates to that of the final clean images.

Critical strengths

The paper is generally well-written and clearly organized. The experimental design is also fairly comprehensive.

Critical weaknesses

My main concern is that I do not feel I have gained many new insights from this paper. While the experiments are comprehensive, most of the observations and conclusions seem relatively intuitive to me.

For a deterministic sampler, we can view the generation process as learning a transport map from the noise distribution to the data distribution. We might reasonably expect the learned transport to exhibit some degree of geometric regularity. Therefore, if a semantic property exhibits a relatively simple structure in the data space (e.g., clustering or some degree of linear separability), it seems natural that its preimage in the noise space could also exhibit some degree of predictability. From this perspective, I do not find the observation that semantic properties can be predicted from the initial noise particularly surprising. What I would find more interesting is understanding what properties of the learned transport make such predictability possible, and how the geometry of the transport affects the accessibility of semantic information. However, the paper does not seem to investigate these questions in much depth.

Relatedly, the relationship between semantic predictability and the confidence of the final clean image, examined in the second set of experiments, also seems relatively intuitive. Low-confidence samples are often closer to the decision boundaries of the classifier, and their preimages under a deterministic transport may likewise lie near more ambiguous regions in the noise space. Therefore, the observation that such samples are harder to predict from the initial noise does not seem particularly unexpected. Similarly, the near-chance prediction accuracy under DDPM is understandable given the additional randomness introduced during stochastic sampling. More generally, stochastic and deterministic diffusion processes can share the same time-dependent marginal distributions while inducing different couplings between the initial noise and the final samples. This distinction already provides some intuition for why deterministic sampling may allow stronger seed-level predictability.

Overall, I feel that the paper mainly provides empirical confirmation of phenomena that are already broadly consistent with our understanding of diffusion models, rather than offering substantially new insights. I would be more interested in understanding what properties characterize the ODE transport induced by diffusion models, how these properties relate to semantic predictability in the noise space, and how different training methods and sampling configurations influence the learned transport. Such an investigation would, in my view, provide a deeper understanding of the observed phenomena. As it stands, I do not find the current contributions sufficiently compelling to justify acceptance.

Questions for authors

I have two additional questions about the experiments.

  1. I'm a bit concerned about the confidence-stratified experiments. For low-confidence samples, the pseudo-labels given by the classifier may be less reliable, and even small perturbations could change the predicted labels. This also means that learning to predict these labels could be inherently more difficult. Since the paper always uses classifier-predicted labels as ground truth, I'm wondering how much of the observed difference in seed-level predictability actually comes from the transport, rather than the difficulty of predicting these labels in the first place.

  2. I'm also curious about the LDA results in Table 8. Why is the LDA accuracy so much higher than that of the CNN probes in Table 7?

2 comments
AuthorsOpenReview ↗

Response to Q1 - Q2 / W1

We thank the reviewer for the thoughtful assessment. Our study quantifies practical semantic recoverability from raw generation states. We group the contribution and confidence concerns below, while distinguishing the measured patterns from the open question of their geometric explanation.

W1. Contribution and transport geometry

We agree that regularity of the learned transport provides a plausible intuition for seed-level predictability. However, this intuition does not establish how accurately a finite-data probe can predict endpoint semantics or how accuracy changes during generation. Our contribution is a systematic measurement of this recoverability along realized reverse trajectories.

Although every DDIM state determines the endpoint label, probe accuracy varies substantially across sampling times and confidence strata. The new CIFAR-10 DDIM–DDPM comparison shows that the difference between samplers is largest early in sampling and nearly closes after 80 reverse updates. The confidence controls below test the robustness of the variation across strata.

We regard these results as quantitative evidence about practical semantic recoverability. Characterizing which geometric properties produce these patterns remains an important open direction. We will make this scope and the empirical contribution explicit in the revision.

Q1 / W1. Confidence hierarchy and label ambiguity

We agree that the difficulty of reproducing the assigned labels can contribute to the confidence hierarchy. We examined this using comparisons where final-image label recovery is nearly complete, normalization by endpoint accuracy, and an independent confidence classifier.

On MNIST, seed accuracy drops from 84.19% at level 1 to 56.29% at level 7, while final-image probe accuracy stays at or above 99.96% across these levels (Table 7). On Fashion-MNIST, seed accuracy drops from 81.40% to 56.21% across the same levels, while final-image probe accuracy stays at or above 98.71%. Thus, substantial seed-level differences occur even among strata where the final-image probe recovers almost all assigned labels.

For CIFAR-10, we normalize within each confidence level ℓ\ell and paired replicate rr using

Rℓ,r(t)=Aℓ,r(t)−cAℓ,r(0)−c,c=0.10.R_{\ell,r}(t) = \frac{A_{\ell,r}(t)-c}{A_{\ell,r}(0)-c}, \qquad c=0.10.

Here t=1t=1 is the seed and t=0t=0 the final image. This measures the fraction of above-chance final-image accuracy recovered at time tt. We compute the ratio within each replicate before averaging. At the seed, mean normalized accuracies for levels 1, 5, and 10 are 41.9%, 32.6%, and 25.8%, respectively. The hierarchy persists after this adjustment.

We also kept classifier A's final-image labels as targets and formed confidence deciles within each A-assigned class using an independently trained classifier B's top-two logit margin. Mean held-out seed accuracies at levels 1, 5, and 10 were 45.7%, 36.8%, and 19.8%, respectively. The ordering therefore persists when a separate classifier supplies confidence.

We will incorporate these controls and normalized results into the revision and discuss their implications for interpreting the confidence hierarchy.

Q2. LDA statistic versus CNN accuracy

We thank the reviewer for identifying this inconsistency. Table 8 reports an LDA discriminability statistic based on the ratio of between-class to within-class scatter. We incorrectly described it as classification accuracy in the table and accompanying text. Its numerical values measure class separation, so they do not establish higher predictive accuracy than the CNN results in Table 7. We will correct the terminology and provide the precise calculation and scaling used in Table 8.

Thank you for the rebuttal. My main concern remains. While the results provide empirical measurements, I still feel that a deeper investigation into the underlying mechanisms is needed to offer sufficiently meaningful new insights. I am inclined to maintain my original assessment.

Reviewer_2FFsOpenReview ↗
Rating
2
Confidence
4
Informativeness
2
Presentation
3
Soundness
2
Read review
Paper summary

It defines 'semantic accessibility' as held out probe acc/NMI for predicting a final image label or attribute from the seed or intermediate state of a deterinistic DDIM trajectory. On MNIST, Fashion-MNIST, CIFAR-10 and CelebA it shows (i) labels are predictable above chance from the pure-noise seed, (ii) DDIM states beat a matched-noise forward-corruption baseline, and (iii) trajectories whose final images get high classifier confidence are far more accessible at every timestep.

Critical strengths
  • Seed-level result is a clean, surprising observation.
  • The experimental protocol is careful and fully specified.
Critical weaknesses
  • Matched-noise baseline is rigged by construction. Under DDIM, Y is a deterministic fucntion of X_T (the papers own prop 1), whereas under forward corruption Y is not a function of x_t^fwd at all, at t=1 the forward baseline is literally pure noise independent of x_0, so 50% is guaranteed. The 'ddim states are more accessible' claim therefore mostly restates determinism rather than showing anything about how the sampler organizes trajectories. A fair baseline would be another trajectory-valued process (ex. predicting the final image's attribute from ddpm intermediate states, or a different deterministic solver)
  • The confidence-stratification result is confounded by label noise. Target and confidence come from the same classifier, so low-confidence trajectories have unreliable targets, the t=0 probe on the clean image itself only reaches 41% on CIFAR-10 level 10. The seed-level gap across levels could be entirely explained by target noise, not by different 'regions of the trajectory map'. Section 3.4 acknowledges this but no experiment separates the two
  • Beyond those two claims the contribution is thin. Prop. 1 is a one-line observation, 'semantic accessibility' is standard probing, and the trajectory curves just rise smoothly with no connection drawn to the speciation/phase-transition literature the paper positions against.
  • (minor) "Eq. equation 2" appears several times. Figure 22(b) uses DDIM color limits so the DDPM panel is uniformly pale, which hides the actual spread; a native color scale would be more honest.
Questions for authors
  • What does the DDIM-vs-forward comparison show beyond Prop. 1? Can you compare against predicting the final image’s label from DDPM intermediate states (not just the seed)?
  • Does the confidence hierarchy survive if confidence is computed by a second, independently trained classifier, or if targets come from a class-conditional sampler?
  • How much of the seed-level accuracy gap across confidence levels is explained by the t=0 label-recovery ceiling (i.e., target noise)?
  • Any quantitative link to the speciation/transition time predicted by Biroli et al. or Sclocchi et al. on the same models?
3 comments
AuthorsOpenReview ↗

Response Part 1: Q1 / W1 and Q2–Q3 / W2

We thank the reviewer for the careful assessment. Our contribution is a quantitative study of how much endpoint semantics a finite-data probe can recover along generation trajectories. Below we group each question with the corresponding weakness and report the additional sampler and confidence controls.

Q1 / W1. DDIM, the forward baseline, and DDPM

Proposition 1 establishes deterministic dependence and perfect prediction in principle. It leaves open how accurately a probe trained with finite data can recover the final-image label and how its accuracy compares with probes trained on other processes. Our experiments quantify this practical recoverability under the specified architecture and training protocol. The matched-noise baseline measures label recovery from the same final images after independent forward corruption at the same nominal noise levels, showing where and by how much DDIM states support higher probe accuracy.

We agree that YY is determined by the DDIM seed and is independent of the forward baseline's input in the pure-noise limit, despite both inputs being Gaussian. However, we disagree that accessibility therefore mostly restates determinism. Above-chance held-out generalization demonstrates learnable structure in the sampler's seed-to-label mapping.

We ran the suggested comparison on CIFAR-10 using 100-step DDIM and 100-step DDPM with the same denoiser, initial seed pool, and timestep schedule. Separate ResNet-18 probes predict each sampler's own final-image labels, assigned by the same classifier. Held-out accuracies were:

Reverse updatesDDIMDDPM
0 (seed)31.8%10.3%
2032.6%12.9%
5039.9%32.3%
8061.0%60.8%

Chance accuracy is 10%. DDIM's advantage is largest early in sampling and narrows as denoising progresses. This comparison between two reverse samplers extends the evidence beyond the pure-noise endpoint of the original forward baseline.

Q2–Q3 / W2. Confidence hierarchy and final-image recovery

We agree that endpoint label ambiguity can affect the confidence hierarchy. We examined this using an independent confidence classifier and comparisons accounting for final-image probe performance.

Independent confidence classifier (Q2).

We ran the suggested two-classifier control on CIFAR-10. We kept classifier A's final-image labels as the probe targets and used an independently trained classifier B to compute the top-two logit margin. Following Section 3.4, we formed confidence deciles within each class assigned by A, ranking images by B's margin. At levels 1, 5, and 10, mean held-out seed accuracy was 45.7%, 36.8%, and 19.8%, respectively, with 10% chance accuracy. The ordering reported in Section 5.2 therefore persists when a separate classifier supplies the confidence score.

Final-image recovery and normalization (Q3).

The MNIST and Fashion-MNIST results in Table 7 provide a complementary check. Between levels 1 and 7, MNIST seed accuracy drops from 84.19% to 56.29%, while final-image probe accuracy stays at or above 99.96% across those levels. On Fashion-MNIST, seed accuracy drops from 81.40% to 56.21% across the same levels, while final-image probe accuracy stays at or above 98.71%. On both datasets, seed accuracy varies substantially even across strata where the final-image probe recovers almost all assigned labels.

To account for the different CIFAR-10 final-image accuracies in Table 1, we use the following normalization within each confidence level ℓ\ell and paired replicate rr.

Rℓ,r(t)=Aℓ,r(t)−AchanceAℓ,r(0)−Achance.R_{\ell,r}(t) = \frac{A_{\ell,r}(t)-A_{\mathrm{chance}}}{A_{\ell,r}(0)-A_{\mathrm{chance}}}.

Here Aℓ,r(t)A_{\ell,r}(t) is held-out probe accuracy at time tt, with t=1t=1 at the seed and t=0t=0 at the final image. Achance=0.10A_{\mathrm{chance}}=0.10 for balanced CIFAR-10. The ratio sets chance to zero and the observed final-image accuracy to one. For example, a value of 0.42 means that the seed probe achieves 42% of the final-image probe's accuracy gain above chance. We compute the ratio within each replicate before averaging. At the seed, the mean normalized accuracies for levels 1, 5, and 10 are 41.9%, 32.6%, and 25.8%, respectively.

The hierarchy is therefore present among strata with nearly equal final-image recovery and persists after the endpoint adjustment on CIFAR-10.

We will incorporate these controls and normalized results into the revision and discuss their implications for interpreting the confidence hierarchy.

Our response continues in Part 2: Q4 / W3; W4.

AuthorsOpenReview ↗

Response Part 2: Q4 / W3 and W4

Continuation of Part 1.

Q4 / W3. Contribution and transition literature

We agree that Proposition 1 is straightforward. It motivates measuring recoverability from raw states along actual generation trajectories, using the semantic label of the image each trajectory ultimately produces. The empirical contribution is the systematic characterization across sampling times and confidence strata, including the time-varying DDIM–DDPM difference reported in Q1/W1. These properties are not specified by deterministic dependence alone.

Biroli et al. study when class structure emerges during generation, using a distribution-based speciation criterion and class agreement between stochastic continuations from a shared state. With identical deterministic DDIM continuations, this agreement is always one and therefore provides no distinction across timesteps or deterministic samplers. In contrast, our semantic accessibility can distinguish deterministic samplers with different probe recoverability.

Sclocchi et al. study the noise threshold beyond which denoising a corrupted image loses its original class while some lower-level features persist. Their forward–backward experiment remains meaningful with a deterministic reverse sampler. We have not quantified this link on our models. Establishing it would require estimating the speciation or reconstruction-transition time using the corresponding criterion and comparing it with our accessibility curves at matched noise levels.

We will clarify this contribution and expand the discussion of how probe recoverability relates to speciation and transition measurements.

W4. Equation references and Figure 22(b)

We thank the reviewer for pointing these out. We will correct the duplicated equation references and give the DDPM panel in Figure 22(b) its own clearly labeled color scale.

Reviewer_2FFsOpenReview ↗

Comment

I appreciate the additional experiments, but I will keep my score. DDIM's advantage disappears at 80 steps (61.0% vs. 60.8%), the seed interpolation control outperforms the DDIM state (70% vs. 63.2%), and the confidence gap on CIFAR-10 shrinks considerably after normalization. In my view, these results weaken rather than support the paper's main claim.

Comments on the paper

3 comments
AuthorsOpenReview ↗

Response Part 1: W1.1, Q1 / W1.2, and Q2 / W2 (scope and applications)

We thank the reviewer for the detailed suggestions. Our contribution is a systematic quantitative characterization of final-image class and attribute accessibility across seeds, intermediate states, and confidence strata. Below we acknowledge the relevant precedents and group the requested experiments with the concerns they address.

W1.1. Prior work

We agree that seed-level predictability has important precedents. Wang & Vastola analyze early feature formation and trajectory geometry; Zhang et al. study consistency of seed-to-image mappings across models; and Ban et al. demonstrate learned prediction of object positions from initial noise. We will discuss these connections explicitly and avoid presenting seed-level predictability itself as unprecedented.

Q1 / W1.2. Seed-noised interpolation and the original baseline

DDIM does not remain more accessible at the two tested checkpoints. However, caution is necessary when interpreting this result: it does not invalidate our original finding that DDIM states are more accessible than independently corrupted endpoints at matched nominal noise levels.

On CelebA Smiling, the seed-noised control achieves 70% accuracy after 20 reverse updates and 82% after 50 updates. At 20 updates, DDIM achieves 63.2%, compared with 50.3% for independent forward corruption.

The proposed control preserves the seed–endpoint pairing, but constructs intermediate representations by directly interpolating the completed image with its generating seed. Its higher accuracy establishes that these representations are easier to probe than the corresponding DDIM states. Calling this construction “a fair control” requires further justification: preserving the pairing alone does not establish why it is the appropriate fairness criterion for the original comparison; that justification is missing.

We agree that independent corruption removes the informative pairing and guarantees chance accuracy in the pure-noise limit. Our comparison intentionally measures accessibility relative to this reference. Deterministic dependence alone does not determine how accurately a finite-data probe can learn the seed-to-label mapping. The interpolation results do not support an additional accessibility advantage of the intermediate DDIM states over this particular construction. We will report them and distinguish the interpretations of the two comparisons.

We also compared two actual reverse samplers on CIFAR-10: with the same denoiser and 100-step schedule, separate ResNet-18 probes predict each sampler's own final-image labels, assigned by the same classifier. Seed accuracies were 31.8% for DDIM and 10.3% for DDPM, becoming 61.0% and 60.8% after 80 updates (10% chance).

Q2 / W2 (scope and applications). Other samplers, models, and potential uses

We have not yet measured accessibility across DDIM step counts, different deterministic solvers, or independently trained denoisers. Our deterministic-trajectory experiments use a fixed 100-step DDIM schedule and remain limited to small pixel-space models. We agree that further comparisons would establish how accessibility depends on sampling and training choices. The DDIM–DDPM comparison above does not establish variation among deterministic samplers.

Each additional configuration requires generating new trajectories, assigning endpoint labels, and training matched probes. Repeating the full experimental grid would require substantial computation. During the rebuttal period, we prioritized the requested baseline comparisons, confidence controls, learning curves, and projection-based probes. We plan to compare deterministic samplers with the denoiser and probe protocol held fixed, and to repeat the measurements across independently trained checkpoints. We will clarify this scope and these planned comparisons in the revision.

We also see concrete potential applications, including conditional generation through seed selection while leaving the original generation map unchanged. This paper focuses on establishing and characterizing the measurement; validating an application requires developing and evaluating its full pipeline. We acknowledge the absence of a demonstrated application without viewing it as evidence that the measurement lacks applicability. We will briefly explain this motivation and distinguish potential uses from demonstrated results.

Our response continues in Part 2: Q3 / W2 (training budget); Q4 / W1.3.

AuthorsOpenReview ↗

Response Part 2: Q3 / W2 (training budget) and Q4 / W1.3

Continuation of Part 1.

Q3 / W2 (training budget). Learning curves and linear projections

We ran the requested learning-curve and projection experiments on CIFAR-10 at confidence levels 1, 5, and 10. Chance accuracy is 10% throughout.

For the seed learning curves, we varied the number of training examples per class while keeping the ResNet-18 architecture and validation/test sets fixed within each confidence level. The table reports mean held-out accuracy over three replicates; the 3,000-per-class results are from the original experiment. Column headings give the number of training examples per class.

Level3001,0001,5002,0002,5002,8003,000
116.7%34.2%39.8%43.0%45.6%45.6%47.0%
514.8%25.9%30.4%34.9%35.1%35.8%36.4%
1011.5%15.1%16.1%17.2%18.1%18.2%18.1%

Accuracy depends on the training budget, and the confidence ordering holds at every tested budget. Accordingly, accessibility is measured relative to a specified probe and training protocol. Gains from 2,000 to 3,000 examples per class are smaller than those from 1,000 to 2,000.

For the top-PC experiment, we fitted PCA on training seeds within each confidence level and trained logistic regression probes using the same splits and 3,000 training examples per class. Mean held-out accuracies over three replicates were:

LevelFull seed (3,072 features)Top 48 PCsTop 192 PCsTop 768 PCs
128.4%11.5%13.7%17.7%
523.4%11.8%13.1%17.6%
1014.3%10.9%11.3%12.4%

The tested leading-PC projections recover only part of the predictability achieved by the full-seed linear probe.

For low-frequency probes, we trained logistic regression on the lowest-frequency 8×88\times8 DCT coefficients per RGB channel, giving 192 features. We used the same splits and training budget as the full-state ResNet-18 probes. The following results are from one matched replicate; each cell gives low-frequency linear / full-state ResNet-18 accuracy.

Reverse updatesLevel 1Level 5Level 10
0 (seed)40.6% / 47.0%33.4% / 36.2%19.1% / 18.9%
2040.9% / 47.0%33.5% / 34.8%20.0% / 20.9%
5044.6% / 55.1%37.4% / 46.4%21.7% / 21.7%
8054.3% / 85.4%46.3% / 71.0%24.6% / 32.6%

These results support the reviewer's suggestion that a simple low-frequency linear readout can recover much of the measured seed-level accessibility. As denoising progresses, the full-state ResNet-18 achieves a growing advantage at all three levels. At level 1, the gap increases from 6.4 percentage points at the seed to 31.1 points after 80 updates.

We will incorporate the learning curves and projection comparisons into the revision and discuss the substantial contribution of low-frequency features to seed predictability.

Q4 / W1.3. Confidence hierarchy and endpoint normalization

We agree that differences in final-image label recoverability complicate interpretation of the confidence hierarchy.

Following the reviewer’s suggestion, we normalized accuracy within each confidence level ℓ\ell and paired replicate rr after subtracting chance:

Rℓ,r(t)=Aℓ,r(t)−AchanceAℓ,r(0)−Achance.R_{\ell,r}(t)= \frac{A_{\ell,r}(t)-A_{\mathrm{chance}}} {A_{\ell,r}(0)-A_{\mathrm{chance}}}.

Here, t=1t=1 denotes the seed, t=0t=0 denotes the final image, and Achance=0.10A_{\mathrm{chance}}=0.10 for balanced CIFAR-10. This ratio sets chance performance to zero and each stratum’s observed final-image probe accuracy to one. We compute the ratio within each replicate before averaging.

At the seed, mean normalized accuracies for confidence levels 1, 5, and 10 are 41.9%, 32.6%, and 25.8%, respectively. The ordering therefore persists after this adjustment. We will include the normalized curves alongside the raw accuracies in the revision.

We also ran a two-classifier control on CIFAR-10. We kept classifier A's final-image labels as the probe targets and used an independently trained classifier B to compute the top-two logit margin. Following Section 3.4, we formed confidence deciles within each class assigned by A, ranking images by B's margin. At levels 1, 5, and 10, mean held-out seed accuracy was 45.7%, 36.8%, and 19.8%, respectively, with 10% chance accuracy. The ordering reported in Section 5.2 therefore persists when a separate classifier supplies the confidence score.

These checks support the robustness of the observed association. We will incorporate the independent-classifier control alongside the normalized results.

Area_Chair_ZELKOpenReview ↗

Author-Reviewer Discussion

Dear Reviewers,

Thank you for your time and thoughtful reviews. With the author–reviewer discussion period ending on November 18 (AoE), please review the authors' rebuttal and share your updated assessment. Specifically, let us know whether your main concerns have been resolved, and if not, which points remain open.

Your input before the deadline is greatly appreciated!

AC

From OpenReview, CC BY 4.0. Authors as signed there; shown unedited.

Related papers

Loading the map…

Discussion (0)

Sign in to comment.