acceptodds
Under review as a conference paper at ICLR 2027

On Oracle Design for Guided Diffusion in the Low-Data Regime

Abstract

In this work, we propose to use the reverse dynamics of diffusion models to train oracles suited for steering in the low-data regime. Steering diffusion models toward desired properties at inference time requires oracles that properly map intermediate noised states to end-state properties. In the scarce labelled data regime, however, learning such a mapping is challenging. Crucially, the reverse process dynamics carry valuable learning signal on how predictions should evolve across timesteps, one that can be leveraged without any labels. We exploit this by training oracles in the original data space while constraining their predictions on consecutive intermediate noised states to adhere to reverse process dynamics, requiring no additional labelling. Optimizing this objective is non-trivial (its loss nests an expectation over the stochastic reverse dynamics), for which we develop an anchored-target scheme to facilitate a principled optimization. In a systematic benchmark of thousands of experimental configurations, spanning regression and classification tasks in both continuous and discrete design spaces, our approach achieves superior steering quality across varied labelled data availability. These results establish dynamics-aware oracle training as the right strategy in the label-scarce regime, broadly applicable across diffusion and flow-based models. Code is available at https://anonymous.4open.science/r/code_dapp-0F53/.

Primary area:
applications to physical sciences (physics, chemistry, biology, etc.)
Keywords:
low data regime, guidance of diffusion models

Reviews

Among:
Shape:
1234this paper, Bottom 49%
Figure 1. Mean review rating of the reviewed papers at ICLR 2027. This paper averages 2.25 over 4 reviews, above 49% of the others.
Reviewer_SZopOpenReview ↗
Rating
1
Confidence
4
Informativeness
1
Presentation
1
Soundness
2
Read review
Paper summary

The paper studies how to train the property oracle used for inference-time steering of diffusion and flow models when labelled data are scarce. It proposes the Dynamics-Aware Predictor (DAP), which combines a supervised loss on the small labelled set of clean samples with a loss on unlabelled reverse trajectories of the pretrained model. The second loss requires the oracle’s prediction at each noisy state to match its prediction one reverse step later. To avoid the bias of the single-sample estimate, the target is computed with an exponential-moving-average copy of the network. Convergence is analyzed under linear function approximation. Experiments on a 2-D toy dataset, MNIST, and molecular benchmarks report better steering quality than oracles trained in clean space, on forward-noised data, or on pseudo-labelled trajectories.

Critical strengths

The paper addresses a practically relevant question: how to train the oracle that inference-time steering relies on when labels are scarce, a component that is usually taken for granted. The inter-step consistency constraint is a correct property of the ideal intermediate oracle, and the paper correctly identifies the bias of the single-sample estimator. The evaluation covers classification and regression targets, continuous and discrete generative models, and both particle-based and gradient-based steering, albeit on small-scale benchmarks.

Critical weaknesses

First, the methodological novelty is limited:

The inter-step identity presented as the key insight, E[y∣xt]=E[E[y∣xt−1]∣xt]\mathbb E[y\mid x_t] = \mathbb E\big[\mathbb E[y\mid x_{t-1}]\mid x_t\big], is the standard Doob hh-transform / Bellman recursion underlying twisted SMC, FK steering, value-based decoding, and diffusion-as-MDP formulations. Several of these works are cited, but the connection is not made. Estimating this fixed point by regression is standard value learning, and DAP is a textbook instance of it: semi-gradient TD(0) with a Polyak-averaged target network, i.e., fitted value evaluation. The paper proposes neither a new target nor a new estimator. The related work omits semi-supervised learning, to which the setting (abundant unlabelled samples, scarce labels) belongs. It also describes SVDD only through its posterior-mean variant, although, to my knowledge, SVDD also learns value functions by Monte Carlo regression on rollouts.

Second, the method and theory do not exploit the structure of the problem:

The reverse chain is finite-horizon and acyclic, with supervision only at t=0t=0. The fixed point can therefore be obtained by backward induction or by Monte Carlo regression onto the endpoint prediction, without bootstrapping. The transition p(xt−1∣xt)p(x_{t-1}\mid x_t) can be sampled repeatedly from the generative model, which makes double sampling or multi-sample targets readily available to remove the bias of Eq. (6). App. A.5 dismisses double sampling by citing RL literature, without an experiment. Theorem 1 restates TD convergence under linear function approximation. It bounds the distance to the solution of the constraint, not to E[y∣xt]\mathbb E[y\mid x_t]. Because labels exist only on Ds\mathcal D_s, this solution is not unique on unlabelled trajectory endpoints and depends on the function class. The inter-step loss carries no label information: at convergence, DAP approximates E[f0(x0)∣xt]\mathbb E[f_0(x_0)\mid x_t] for a clean-space predictor f0f_0 fit on Ds\mathcal D_s. For the regression tasks, where the reward is the likelihood of a target value, enforcing consistency on the predicted mean does not yield the required potential E[p(y⋆∣x0)∣xt]\mathbb E[p(y^\star\mid x_0)\mid x_t].

Third, the experiments do not support the claims:

The oracle is trained on unguided prior trajectories but deployed along guided trajectories. Prediction quality is evaluated only on unguided trajectories, leaving this distribution shift untested, and no analysis of error propagation over 200–1000 bootstrapped steps is given. Key ablations are missing: the non-anchored estimator (discussed only theoretically in App. A.3), double sampling, Monte Carlo regression onto DAP’s own endpoint prediction, and backward induction. The necessity of the anchored-target scheme, the main technical component, is therefore not established. The comparison with IDP is confounded by the different endpoint predictors, and the gap shrinks as NτN_\tau grows (Fig. 3). Standard baselines are absent: semi-supervised methods, and a noisy classifier trained with all labels as in standard classifier guidance. The “universe” oracle is applied through the posterior mean E[x0∣xt]\mathbb E[x_0\mid x_t], which makes it a weak upper reference, and it also serves as the evaluator for reward and SWD. All benchmarks are low-dimensional (2-D data, MNIST, SMILES with 2-layer MLPs, a 1024-dimensional VAE latent), with at most 3000 trajectories. Label scarcity is simulated: MNIST is fully labelled, and the molecular properties are computed with RDKit.

Minor:

Eq. (3) uses “:=:=” for a constraint and the same fθf_\theta for all time steps. “Upper-bounds” is used throughout to mean “performs best”.

Questions for authors

None

Reviewer_dsCYOpenReview ↗
Rating
3
Confidence
2
Informativeness
2
Presentation
3
Soundness
3
Read review
Paper summary

We want to do inference-time steering of diffusion models. However, we only have minimal data samples to train our oracle. The paper finds that the best approach is to train a network to match the minimal data samples during the zero noise timestep, and enforce a consistency loss on the network's value function predictions for the previous timesteps. This works better than the standard forward noised approach, which only trains on the diffusion rollouts from the minimal dataset, or the IDP approach which uses an NN trained on the minimal dataset to impute y-values, and then assigns these y-values to the rollouts. This is validated on toy tasks as well as a molecule steering task.

Critical strengths

I like the empirical section. DAP is able to show clear improvements on a toy crescent moon dataset. Furthermore, these gains persist on more complex datasets such as MNIST, discrete flow guidance, and latent space steering. Importantly, the gains increase with more trajectories. This persists across metrics; wasserstein distance & hyper-vol index.

Critical weaknesses

The paper's central findings are that 1) the consistency loss works better than the IDP/standard foward noised approach, and 2) this can be trained via a lagging student-teacher training setup.

When evaluating the novelty of 1 and 2, 2 seems standard in the literature already. 1 has not been applied to this problem before, but is a standard concept in the literature.

Thus, I feel the paper could have provided more conceptual novelty.

Questions for authors

The paper's claim is fundamentally an empirical one. Thus, it would be nice, if possible, to see more real-world experiments (proteins, DNA, etc.). The current ones seem specific to molecules.

More generally, it would be helpful if you could articulate the conceptual novelty of your paper in the context of prior work - see my comments above on critical weaknesses.

Reviewer_Nqt7OpenReview ↗
Rating
2
Confidence
3
Informativeness
2
Presentation
2
Soundness
1
Read review
Paper summary

The paper proposes Dynamics-Aware Property Oracle (DAP), a method for improving property oracle training in low-label regimes by exploiting the reverse dynamics of pretrained diffusion and flow-based generative models. The key idea is to leverage intermediate states from reverse-generation trajectories as additional training signals, enforcing consistency between oracle predictions at consecutive timesteps without requiring additional ground-truth labels.

The method combines a supervised boundary loss on the available labeled dataset with an intermediate dynamics-consistency loss. To address the bias introduced by the naive single-sample estimator of the stochastic reverse-dynamics objective, the authors introduce an anchored-target training scheme based on an exponential moving average (EMA) target network. They further provide theoretical analysis of the resulting optimization procedure under linear function approximation.

The approach is evaluated through approximately 9,000 experimental configurations across Half-Moon, MNIST, and molecular benchmarks, covering both continuous diffusion and discrete flow models. The experiments investigate different labeled dataset sizes, numbers of augmented trajectories, and reference oracle quality. DAP is compared with several oracle baselines and evaluated in both prediction and downstream steering tasks, including Feynman–Kac-based steering and gradient-based predictor guidance. Overall, the results show that DAP generally achieves better prediction and steering performance than non-universe oracle baselines, with improvements from additional reverse trajectories and robustness to degraded reference oracle quality.

Critical strengths

Strengths

  1. Novel methodology: The paper proposes a novel approach to low-label oracle training by exploiting reverse diffusion trajectories as additional supervision without requiring extra annotations.

  2. Theoretical contribution: The anchored-target training scheme addresses the bias of the naive estimator and is supported by theoretical convergence analysis.

  3. Comprehensive evaluation: The paper provides extensive experiments across approximately 9,000 configurations, covering multiple datasets, prediction tasks, and both continuous diffusion and discrete flow models.

  4. Robustness and data efficiency: The experiments demonstrate that DAP generally improves prediction and steering performance with more augmented trajectories and is robust to degradation of the reference oracle.

Critical weaknesses
  1. Strong dependence on pretrained generative models: DAP relies on a pretrained generative model trained on a large in-domain dataset to provide meaningful reverse trajectories. The paper does not investigate whether the method remains effective when such a pretrained model is unavailable, poorly trained, or learned from limited data, which limits its applicability to genuinely low-data generation scenarios.

  2. Limited evaluation on complex image generation tasks: The image experiments are mainly conducted on synthetic Half-Moon and MNIST datasets, without evaluation on more challenging high-resolution natural image datasets such as FFHQ. Furthermore, comparisons with established few-shot image generation approaches, such as StyleGAN2-ADA, are missing, leaving the effectiveness and scalability of DAP in realistic low-shot image generation scenarios unclear.

Questions for authors

1. Dependence on the pretrained generative prior.

The proposed method relies on reverse trajectories generated by a pretrained generative model trained on a large in-domain dataset. However, the low-data setting considered in this work primarily concerns the availability of labeled data for oracle training, rather than the training data available to the generative model. Could the authors clarify how sensitive DAP is to the quality of the pretrained generative prior? In particular, does the method remain effective when the generative model is trained with limited data or exhibits poor sample diversity?

2. Impact of oracle guidance on generative quality.

While the paper evaluates steering performance using metrics such as HVI and SWD and includes unguided sampling as a baseline, it remains unclear how oracle guidance affects the overall quality and diversity of generated samples. Could the authors provide further analysis of the trade-off between target property satisfaction, sample fidelity, and diversity under different guidance strengths? This would help establish whether the improved steering performance comes at the cost of degraded generation quality or reduced mode coverage.

3. Generalization to more complex image generation settings.

The image-based evaluations are primarily conducted on Half-Moon and MNIST, which may not fully demonstrate the effectiveness of DAP on high-dimensional and more diverse image distributions. Could the authors discuss the expected scalability of the proposed approach to more challenging datasets, such as FFHQ? Furthermore, a comparison with established few-shot generation approaches, such as StyleGAN2-ADA, under comparable data constraints would provide additional insights into the practical relevance of DAP in low-data image generation settings.

Reviewer_65gZOpenReview ↗
Rating
3
Confidence
1
Informativeness
2
Presentation
2
Soundness
2
Read review
Paper summary

I served duties for ICLR and is supposed to get exempt from paper review every time. The system seems bugged this year. Please ask AE to find another reviewer. Please ignore my rating

Critical strengths

na

Critical weaknesses

na

Questions for authors

na

From OpenReview, CC BY 4.0. Authors as signed there; shown unedited.

Related papers

Loading the map…

Discussion (0)

Sign in to comment.