acceptodds
Under review as a conference paper at ICLR 2027

Diverse Low-Temperature Diffusion Sampling through Replica Exchange

Abstract

Diffusion models achieve strong generative performance across domains. However, achieving high sample quality under hard physical and dynamical constraints, as in protein design and robotic manipulation, remains difficult. Existing inference-time methods either require a costly or unavailable verifier, or avoid a verifier at the cost of reduced diversity. To relieve this tension, we introduce Score Tempering with Exchange between Exploratory Replicas (STEER), a verifier-free inference-time method that replaces single-chain sharpening with global search using multiple diffusion chains. During sampling, neighboring chains, each at different sharpening strengths, exchange particles according to an energy-based acceptance rate, propagating exploration to outputted samples. Across a diverse range of domains, from synthetic multimodal distributions to image generation, STEER achieves a better fidelity–diversity trade-off and improves success rates on continuous-control tasks in both verifier-based and verifier-free settings. Our findings suggest that effective diffusion tempering requires going beyond local updates to population-based global search that enhance fidelity and diversity.

Primary area:
generative models
Keywords:
diffusion models, inference-time scaling, replica exchange, verifier-free sampling, fidelity-diversity trade-off

Reviews

Among:
Shape:
1234this paper, Top 15%
Figure 1. Mean review rating of the reviewed papers at ICLR 2027. This paper averages 3.00 over 3 reviews, above 85% of the others.
Reviewer_fZ55OpenReview ↗
Rating
3(+2)
Confidence
3
Informativeness
2
Presentation
3
Soundness
2
Read review
Paper summary

This paper proposes STEER, a verifier-free inference-time steering method that samples the tempered distribution p1/λp^{1/\lambda} of a pretrained diffusion model. STEER runs MM diffusion chains at different temperatures in parallel and rescales the score of each chain to its temperature. STEER adapts parallel tempering, a technique that MCMC uses for mode mixing, to the inference of pretrained diffusion models. The authors prove that the swap rule is exact for isotropic Gaussian data and does not amplify the bias of score rescaling in general. They validate STEER on 2D Gaussian mixtures, Stable Diffusion 3, protein backbone generation, PointMaze planning, and Robomimic manipulation.

Critical strengths

S1. STEER is a training-free, verifier-free plug-in with no extra network evaluations. The swap rule reuses scores already computed for denoising, so its cost equals that of independent chains. The same algorithm applies to images, proteins, planning, and robot policies.

Critical weaknesses

W1. The advantage over existing verifier-free population methods is insufficiently established. As acknowledged in Appendix A, Feynman–Kac Correctors already provide a verifier-free framework for targeting annealed marginals proportional to ptβp_t^\beta using pretrained scores and SMC resampling. However, FKC is absent from the experimental comparisons. The memory-based argument against SMC is also insufficient: STEER maintains MM replicas, so restricting interactions to adjacent replicas does not eliminate population storage costs. Without a comparison under matched computational budgets and measured memory usage, it remains unclear what practical advantage STEER offers over this relevant prior approach. This omission weakens the paper’s justification for its claimed gap in existing verifier-free methods.

W2. Robustness of the swap mechanism to score estimation error is not established. Hot replicas can visit regions that are poorly represented under the model’s training marginals, where score accuracy may be poorly controlled. Because STEER estimates swap acceptance probabilities from these learned scores, estimation errors can distort acceptance decisions and bias the target replica. The exactness result assumes the true score under standard Gaussian data, while the non-amplification proposition requires a target-preserving swap kernel. Neither establishes robustness of the practical kernel to score error. Although the pretrained-model experiments demonstrate empirical utility, they do not isolate this sensitivity. A controlled study comparing exact and learned scores on the toy distributions, or systematically perturbing score accuracy, would help establish when replica exchange remains beneficial.

Questions for authors

Q1 (experiment request). Could you compare STEER with FKC temperature annealing at the same number of particles and the same compute, on the toy mixtures and at least one high-dimensional task?

Q2 (clarification). How does the memory cost of keeping MM replicas in STEER differ from the cost of keeping KK particles in SMC?

Q3 (experiment request). On the toy mixtures, where the true score is known, how much does the swap acceptance from the learned score deviate from the acceptance from the true score? Alternatively, how do STEER and TSR compare when the base model is undertrained?

Q4 (clarification). Could you report error bars or per-seed results for the protein and SD3 experiments?

3 comments
AuthorsOpenReview ↗

Summary + Comparison to FKC + Robustness to score error

Summary.

  • STEER beats FKC at matched compute. It has lower error on the toy at every budget and higher success on all three robomimic tasks (W1, Q1).
  • Memory is of the same order as SMC. We agree STEER and SMC both store M states, and have removed the memory argument from the paper (W1, Q2).
  • STEER is robust to score error. With exact vs. learned scores and across training checkpoints, the learned score barely changes STEER, and swaps help even an undertrained network (W2, Q3).

W1 & Q1 & Q2: Comparison with FKC; memory

We add FKC (and SMC with RNE weights) as baselines under matched NFE, and report measured wall-clock time and memory per output sample. Toy results are on the 40-mode GMM with a learned score, 50,000 samples, 5 seeds (mean ± SE). For SMC and FKC, M is the number of particles.

MethodMTNFEWall-clock (µs)Peak memory (KB)Distance TVD ↓
FKC85040034.63.30.139 ± 0.001
SMC85040039.33.30.162 ± 0.002
STEER410040028.216.90.123 ± 0.001
  1. More accurate at the same compute. STEER has lower error than FKC at every budget, with both the learned score and exact energies (Fig. 2d, e). We include additional results with W2 distance in the revised paper (Fig. 10), with the same ordering. We also test at 200 NFE, where STEER has the lowest mode-weight error (Fig. 2a; HAee W2).
  2. Slightly faster. At matched NFE, STEER is slightly faster than FKC and SMC.
  3. Memory of the same order. We agree with the reviewer: STEER stores M replicas, and its per-sample memory is higher than FKC's in our implementation. We have removed the memory-based argument against SMC from the paper. The advantage we claim instead is accuracy at matched compute, from a swap mechanism that needs only adjacent pairs and no population-wide weights.

Q1. FKC at the same particles and compute. On robomimic, FKC and SMC run 4 particles, the same as STEER's 4 replicas, at matched NFE, and STEER has a higher success rate on all three tasks (Fig. 6; HAee W2, item 3). On the toy, every method receives the same NFE, and FKC's particle count was tuned at that budget (8 particles × 50 steps, table above); STEER has lower error than FKC at every budget (Fig. 2d, e; HAee W2).

Q2. Memory of M replicas vs. M particles. Both store M states, so the memory is of the same order (item 3 above).

W2 & Q3: Robustness to score error

We ran both controlled studies the reviewer proposes on the 40-mode GMM, at 400 NFE per sample; they also answer Q3.

1. The learned score barely changes STEER. 5 seeds.

ScoreSwap energyDistance TVD, no swapsDistance TVD, STEER
ExactExact0.1530.084
ExactPath integral through the exact score0.1530.122
Learned (200k)Path integral through the learned score0.1530.123

Standard errors are at most 0.002.

  • Moving from the exact to the learned score barely changes STEER's error, so score error has little effect on the swaps; the gap to exact energies comes from the quadrature (HAee W3).
  • STEER clearly beats the swap-free chain in every setting.

2. Swaps help even an undertrained network. We train networks for an increasing number of iterations and report TVD of the final samples (seed 0; Fig. 2c).

Training iterationsEnergy TVD, no swapsEnergy TVD, STEERDistance TVD, no swapsDistance TVD, STEER
20,0000.2580.1950.1480.121
50,0000.2480.1820.1490.113
100,0000.2470.1810.1500.114
150,0000.2360.1610.1500.112
200,0000.2420.1740.1530.119
  • STEER outperforms the swap-free chain at every checkpoint, including the least-trained network.
  • The size of the improvement stays roughly constant across checkpoints, so the swaps' benefit does not depend on a well-trained score.

3. Swap acceptance barely changes. Replacing the exact score by the learned score in STEER's swap energy raises the error of the swap acceptance probability by at most 0.006 (e.g. 0.180 to 0.181 at 100 NFE per state), and of the energy difference by at most 0.04 (Fig. 9b, c). With either score, the path integral has about half the energy-difference error of the RNE estimator used by CREPE, SMC and FKC (9ZEq W2).

Q4: Error bars for protein and SD3

Protein designability is reported with 95% Wilson intervals (Fig. 16): STEER improves pooled designability from 35.9% to 44.2% (56/156 vs. 69/156). Each SD3 point is a single run of 10,000 samples. On FLUX.1, we report CLIP with standard errors over 1,000 prompts (Table 2), and every toy and robomimic result is reported as mean ± SE over five seeds.

Thank you for the detailed rebuttal and additional experiments. I appreciate the authors' efforts to address my concerns.

W1: The new comparisons with FKC demonstrate STEER's empirical advantage at matched compute. Although STEER has higher memory usage, the authors acknowledge this limitation and have revised their claims accordingly. I consider W1 addressed.

W2: The experiments across training checkpoints provide convincing evidence that STEER remains effective with undertrained score networks. This substantially addresses my main concern about robustness to score estimation error.

However, two concerns remain:

  1. Comparison with single-chain TSR: My original Q3 requested a compute-matched comparison with single-chain TSR across training checkpoints. The swap-free temperature ladder is a useful ablation, but it does not replace this comparison.

  2. Uncertainty in high-dimensional experiments: Figure 4 shows that improvements vary across configurations. In particular, STEER does not consistently outperform the baseline in protein generation. Given the stochastic nature of this kind of method, multi-seed evaluations are important to establish the reliability of these improvements. While confidence intervals were added for protein designability, the main SD3 and protein results still lack uncertainty estimates. I understand that these experiments may be computationally expensive, but if the runtime is reasonably manageable, I would encourage multi-seed evaluations for at least representative configurations.

Given that the rebuttal has substantially addressed my main concerns, I am open to raising my initial rating.

We thank the reviewer for the careful follow-up, and are glad the FKC comparison and checkpoint study addressed W1 and W2. We address the two remaining points below.

1. Compute-matched single-chain TSR across checkpoints. We apologise for the confusing label: the swap-free baseline in our W2 table is single-chain TSR, given the same NFE as STEER. Relabelled, it is the comparison the reviewer requested in Q3:

Training iterationsEnergy TVD, TSR (400 NFE)Energy TVD, STEER (400 NFE)Distance TVD, TSR (400 NFE)Distance TVD, STEER (400 NFE)
20,0000.2580.1950.1480.121
50,0000.2480.1820.1490.113
100,0000.2470.1810.1500.114
150,0000.2360.1610.1500.112
200,0000.2420.1740.1530.119
  • At matched NFE, STEER beats single-chain TSR at every checkpoint, on both metrics, including the least-trained network.
  • Giving TSR more steps does not close the gap. Its error barely changes from 100 to 400 NFE (9ZEq W1), so the gain comes from the swaps, not from extra compute.

We have updated the paper to label this baseline as single-chain TSR at matched NFE (Fig. 2c).

2. Multi-seed evaluation in high dimensions. We agree that multiple seeds are needed, and ran a multi-seed protein study on FoldingDiff, comparing STEER with single-chain TSR at the same NFE on long sequences across target temperatures λ1\lambda_1. We report FID at matched designability, as mean ± SE across 5 seeds.

FID at matched designability (each row compares the TSR and STEER settings with about the same designability):

Designable %FID ×10⁶ ↓, TSRFID ×10⁶ ↓, STEER
~28%4.58 ± 0.024.46 ± 0.03
~33%4.89 ± 0.034.66 ± 0.04
~36%5.09 ± 0.024.89 ± 0.06
~41%not reached5.14 ± 0.04
~45%not reached5.36 ± 0.05
  • At matched designability, STEER has lower FID at every level. Its whole trade-off curve between designability and FID lies below TSR's.
  • STEER reaches higher designability. TSR does not exceed 36% designable at any λ1, while STEER reaches 45%.
  • STEER is more designable at every λ1. Pairing STEER and TSR on the same seed, batch and length, STEER gains 5.7 points on average.

This addresses the inconsistency the reviewer noted in Fig. 4: with multiple seeds and matched compute, STEER's protein improvement holds at every operating point we test. We have added these results to the revised paper, with error bars across seeds. For text-to-image, the FLUX.1 results report CLIP score with standard errors over 1,000 prompts (Table 2).

We thank the reviewer again for engaging with our rebuttal and for being open to raising their rating.

Reviewer_HAeeOpenReview ↗
Rating
3(+2)
Confidence
4
Informativeness
2
Presentation
1
Soundness
1
Read review
Paper summary

This paper introduces STEER, an inference-time technique for sampling from tempered distributions p1/λp^{1/\lambda} using a pretrained diffusion model, without the need for retraining or a verifier. In STEER, each sample is created through a single reverse-diffusion pass over TT steps, running MM parallel particles at temperatures λ1<⋯<λM\lambda_1 < \dots < \lambda_M. Each particle is denoised using temporal score rescaling (TSR), which rescales the learned score by rt(λ)=1/(αt2λ+σt2)r_t(\lambda) = 1/(\alpha_t^2\lambda + \sigma_t^2). At each step, neighboring particles attempt to swap states following a deterministic even-odd scheme. The swap acceptance assumes the particle at λk\lambda_k follows pt rt(λk)p_t^{\,r_t(\lambda_k)} and estimates the energy difference via a two-point line integral of the learned score. At t=0t=0, only the coldest particle is kept. Theoretical results show the swap rule achieves detailed balance for isotropic Gaussian data and does not increase total variation distance from the joint target. Experiments on 2D Gaussian mixtures, Stable Diffusion~3, FoldingDiff, PointMaze, and Robomimic demonstrate improved fidelity-diversity trade-offs and task performance compared to TSR, CNS, Langevin correctors, and CREPE.

Critical strengths
  • Verifier-free low-temperature sampling is an important problem, and the method is simple: swaps reuse score evaluations, so they add no network evaluations beyond those of the MM particles.
  • Incorporating replica exchange into the sequential denoising process is a novel contribution.
  • The experiments cover a diverse selection of tasks and domains.
Critical weaknesses
  • STEER operates as a non-equilibrium particle method, so parallel tempering theory is not strictly applicable. Parallel tempering requires both the local moves and swap steps to preserve a shared invariant distribution, with equilibrium sampling. In contrast, STEER runs a single forward pass with one swap per step, so particles never reach equilibrium. Theoretical guarantees (Thm. 2.1, Prop. 2.1) only apply if particles are already distributed according to the target, which is not achieved in practice. With exact scores, swaps have no effect on the output, so any gains must come from correcting TSR’s error. The paper does not justify why swaps, themselves built on this approximation, would reliably do so.

  • With even-odd swaps, moving a state from the hottest to coldest replica requires at least ∼2M\sim 2M rounds, even with perfect swap acceptance. STEER only performs TT rounds (e.g., 3030 for SD3 with M=6M=6), where the target changes each round and early rounds contribute little since rt(λ)≈1r_t(\lambda) \approx 1 for tt near TT. Each sample requires M×TM \times T score evaluations and yields just one biased, non-equilibrium sample. For the same computational budget, SMC or Feynman–Kac correctors (Skreta et al., 2025) could return MM weighted samples that consistently target p1/λp^{1/\lambda} as MM increases. STEER’s swaps act as unweighted, pairwise selection by energy, and any real advantage over SMC (e.g., reducing weight degeneracy) must be clearly demonstrated. In all but one experiment (PointMaze), STEER is compared to single-chain baselines using only 1/M1/M of its compute; fair comparisons to compute-matched SMC and Best-of-MM-by-energy baselines are needed.

  • The swap rule assumes πt(k)∝pt rt(λk)\pi_t^{(k)} \propto p_t^{\,r_t(\lambda_k)}. In reality, it follows the unknown marginal from TSR, and these match only for isotropic Gaussian data. Finer time grids, more chains, or denser ladders do not resolve this discrepancy. Additional bias comes from the two-point quadrature and treating the learned score as an exact energy gradient. No ablation study separates these sources, though such analysis would be feasible in the Gaussian-mixture experiments.

  • Tempering alters both mode weights and shapes, so hot particles only enhance exploration if they are sufficiently flat to cross energy barriers. In most experiments, the hottest particle’s temperature is very close to the target (λM=λ1+0.2\lambda_M = \lambda_1 + 0.2 for SD3 and FoldingDiff, 1.11.1 for Robomimic, λ1+0.01N\lambda_1 + 0.01N for PointMaze), minimizing potential benefits. Additionally, since rt(λ)→1r_t(\lambda) \to 1 as t→Tt \to T, swaps are almost always accepted early and only differentiate near t=0t=0, after samples are committed to modes. This challenges the claim that exchange aids early exploration; Fig.~3 supports this, showing similar global structure for samples with 1 and 4 replicas.

  • The even-odd swap schedule (Okabe et al., 2001) is implemented without citation. The paper's assertion that adding more replicas leads to overmixing (Sec. 3.2) is incorrect; with even-odd swaps, parallel tempering performance improves, then plateaus as more replicas are added. The 20% acceptance rate in Fig. 12 comes from reversible PT and is not meaningful here (Lingenheil et al., 2009). Since STEER does not reach equilibrium, these theoretical metrics may not apply. All temperature ladders are linearly spaced with manually chosen endpoints, and Appendix D.3 shows this can lead to poor balance.

  • CREPE is an MCMC method that performs replica exchange along the diffusion-time path and produces samples sequentially. With even-odd swaps, its replicas traverse between an exactly sampled Gaussian reference and the target. It is neither fundamentally local nor an SMC variant. Comparing CREPE under matched wall-clock time—a metric sensitive to implementation—against a single-pass particle method is not meaningful. The paper does not report ladder configuration, number of rounds, burn-in, or adaptation to the tempered target. CREPE’s near-identical results to TSR in Fig. 2 suggest it was run far from equilibrium. If used as a baseline, error should be reported as a function of compute for both methods.

  • Despite familiarity with the annealing and replica-exchange literature, I found the paper difficult to follow. Figure 1 would be much clearer if it depicted replica trajectories over denoising time, contrasting the dynamics of STEER, CREPE, and SMC-based methods.

Questions for authors
  • Why use replica exchange rather than SMC, given that STEER spends M×TM \times T score evaluations per sample? How does it compare at matched compute to Feynman--Kac correctors and to Best-of-MM selection by estimated energy?
  • The acceptance rule contains no reward term. How does STEER target the reward-tilted distribution in App.~D.1 (Fig.~9)?
  • Can you explain how the design choices affect performance or quantify the resulting bias?
  • Can you explain discuss how MM and the ladder should be tuned?
6 comments
AuthorsOpenReview ↗

Summary + Non-equilibrium; why swaps correct TSR's error

Summary.

  • Each swap moves replicas toward their targets. We agree the guarantees hold only at the target, and now present STEER as a principled heuristic backed by this per-step argument and an error-source ablation (W1, W3).
  • STEER beats the population baselines at matched compute. It beats SMC and FKC at every budget and CREPE at small and moderate budgets, and swaps beat resampling the same chains (W2, W6).
  • A simple rule sets the ladder. It fixes the hottest temperature and the number of replicas; we have also corrected citations and claims, and redrawn Figure 1 as trajectories over denoising time (W4, W5, W7).

W1: Non-equilibrium; why swaps correct TSR's error

We agree that STEER is a non-equilibrium method and that Thm. 2.1 and Prop. 2.1 hold only at the target. We have updated the paper to state this explicitly (Sec. 2.4 and App. B.5).

The full algorithm therefore has no end-to-end guarantee at finite compute, and we now present it as a principled heuristic: each component rests on an exact result (the swap kernel's invariance and the score-based energy estimate), and the combination is validated empirically. The same is true of the samplers STEER builds on and competes with, including TSR itself, which is exact only for isotropic Gaussian data. We support STEER in two complementary ways: a per-step argument for why swaps move the replicas toward the tempered targets (below), and controlled measurements of every remaining error source (W3).

With exact scores, swaps do have an effect. That premise holds only for isotropic Gaussian data, where TSR is exact. For a non-Gaussian target, swaps correct the error introduced by TSR's marginal, even with the exact score. On the 40-mode GMM with the exact score and exact energies, swaps nearly halve TSR's error, from 0.153 to 0.084 TVD (full table in W3).

Why swaps correct the error. With exact energies, each swap is a Metropolis step that leaves the product of the tempered targets, ∏kpt rt(λk)\prod_k p_t^{\,r_t(\lambda_k)}, invariant. A Markov kernel that leaves a distribution invariant cannot increase the divergence (KL or TV) to that distribution. So a swap moves the replicas' joint law toward the product of tempered targets, or leaves it unchanged, whatever TSR's error was before the swap. Two caveats apply:

  1. The guarantee needs accurate energies. With estimated energies, the swap preserves a slightly different distribution; W3 measures how much this costs.
  2. The targets move with t, so the guarantee holds per step. Each denoising step can add new error, which the next swap again pulls back.

Below we measure this empirically on the same 40-mode GMM (Erives et al. 2025, He et al. 2026, Midgley et al. 2023), at 400 NFE per sample, for two sources of error:

  1. Score-rescaling error from TSR's approximate tempered dynamics.
  2. Score network error.

The swap-free baseline is a single TSR chain with all 400 steps, i.e. TSR at the M-fold budget. This is equivalent to the coldest replica of the same ladder with swaps turned off, given STEER's total NFE.

Swaps stop score-rescaling error from growing. The table reports TVD between each chain's state at diffusion time t and the true tempered marginal at that noise level (learned score, 5 seeds; Fig. 2b). t = 1 is pure noise and t = 0 is the final samples.

Diffusion time tEnergy TVD, STEER − no swapsDistance TVD, STEER − no swaps
1.0 (noise)0.0000.000
0.6+0.003+0.011
0.33−0.056−0.028
0.0 (samples)−0.062−0.032

Negative values mean STEER has lower error. Standard errors are at most 0.002.

  • While the modes still overlap (t ≥ 0.6), the two chains have similar error, with STEER slightly higher in distance TVD.
  • Once the modes separate, the chain without swaps keeps drifting, while STEER's swaps stop the error from growing.

Swaps help with every score network we trained. We repeat the comparison with score networks trained for 20k to 200k iterations (Fig. 2c). STEER has lower error than the swap-free chain at every checkpoint, in both energy and distance TVD. The full table is in our response to fZ55 W2.

AuthorsOpenReview ↗

Compute-matched SMC, FKC and Best-of-M

W2 & Q1: Compute-matched SMC, FKC and Best-of-M

While SMC or FK correctors could return MM weighted samples, we show that under a fixed compute budget, STEER has lower error and lower mode-weight error. We compare against SMC (with RNE weights, He et al. 2026) and FKC (Skreta et al. 2025), with every method given the same total NFE per output sample. This also answers Q1 (why replica exchange rather than SMC, and Best-of-M by estimated energy).

1. Lower error at every budget, and better mode weights. 40-mode GMM, learned score, 5 seeds (Fig. 2e).

NFE per sampleSMCFKCSTEER
2000.1790.1640.141
4000.1620.1390.123
8000.1480.1170.103
16000.1370.0990.087

Distance TVD ↓; standard errors are at most 0.002. We include additional results with W2 distance in the revised paper (Fig. 10b), where STEER again has lower error than SMC and FKC at every budget.

  • The ordering also holds with exact energies (Fig. 2d; 9ZEq W2).

At 200 NFE per sample, STEER also covers the most modes and has the lowest mode-weight error. We plot 180 samples from each method (seed 0; Fig. 2a); mode-weight TV uses all 50,000 samples.

MethodModes covered (of 40) ↑Mode-weight TV ↓
CREPE390.248
SMC350.257
FKC330.231
TSR390.260
STEER400.212

2. Swaps beat resampling the same chains. We fix the dynamics to 4 chains × 100 TSR steps (400 NFE) and change only the swap mechanism. Resample weights the 4 chains as an SMC population and resamples them; Best-of-M keeps the chain with the highest log⁡p\log p. Best-of-M needs the exact density to rank samples, so it acts as an oracle verifier; it is the oracle version of Best-of-M by estimated energy, whose ranking can only be as good as its energy estimate (9ZEq W2, item 2).

Swap mechanismEnergy usedDistance TVD ↓
None (100-step TSR)n/a0.159
ResampleEstimated from the learned score0.128
Best-of-MExact log⁡p\log p (oracle)0.059
Swap (STEER)Estimated from the learned score0.123

Learned score, 5 seeds; standard errors are at most 0.001.

  • Without a verifier, swaps beat resampling the same chains.
  • With exact energies, resampling and swaps perform about the same. Best-of-M is best overall, but only because it ranks samples by the exact density, which a pretrained model does not provide.

3. Higher success on robomimic. Success rate (%) at matched NFE per action chunk, 5 seeds × 50 rollouts (mean ± SE; Fig. 6). SMC and FKC run 4 particles on the policy's 100-step schedule (384 NFE). Wall-clock and memory are per action chunk, averaged over tasks.

MethodNFECanSquareTool hangWall-clock (ms)Memory (MB)
TSR (M-fold)40058.8 ± 2.280.8 ± 1.641.2 ± 2.6306721.7
SMC38451.6 ± 2.481.6 ± 3.736.8 ± 2.264022.3
FKC38454.0 ± 2.178.4 ± 5.336.8 ± 4.354622.3
CREPE33853.2 ± 3.476.0 ± 1.439.2 ± 4.585627.1
STEER40060.8 ± 2.983.2 ± 2.042.8 ± 4.389523.7
  • STEER has the highest success rate on every task. SMC is within error of STEER on square, and SMC and FKC fall below single-chain TSR on can and tool hang.

On the ~2M rounds needed for transport. STEER does not need to carry one state from the hottest to the coldest replica. All M replicas start from fresh noise, and each swap only has to move the better state one rung down. The no-swap ablation above shows that most of the correction happens once the modes separate (t ≤ 0.6), which on the toy still leaves ~60 of the 100 rounds.

AuthorsOpenReview ↗

Separating error sources + Hyperparameters

W3 & Q3: Separating error sources

We thank the reviewer for this suggestion and ran the proposed ablation on the 40-mode GMM, turning on one error source at a time (400 NFE per sample, 5 seeds). This also answers Q3.

ScoreEnergy differenceSwapsError sources presentDistance TVD ↓
Exactn/ano (TSR)TSR marginal mismatch0.153
ExactExactyesSwap-rule mismatch only0.084
ExactPath integralyes+ quadrature0.122
Learned (200k)n/ano (TSR)+ score error, no swaps0.153
Learned (200k)Path integralyesAll sources (STEER in practice)0.123

Standard errors are at most 0.002.

  1. Swaps correct TSR's marginal mismatch. Even with exact energies, the swap rule assumes πt(k)∝ptrt(λk)\pi_t^{(k)} \propto p_t^{r_t(\lambda_k)} while chains follow TSR's marginal, yet swaps nearly halve TSR's error. Finer time grids do not resolve this mismatch for TSR, whose error stays flat as its steps grow, but more replicas do help STEER (Fig. 14a; W5).
  2. The trajectory integral needs no extra NFE. STEER integrates the score along each replica's own denoising trajectory, using the two scores at the ends of each step, which the replica has already computed, so swaps add no NFE. This quadrature is the largest added error, yet it is still more accurate than RNE with four times the NFE (9ZEq W2). More points per step would shrink it further, but at the cost of extra NFE.
  3. Score error barely matters. Computing the path integral with the learned score instead of the exact score barely changes STEER's error, and STEER stays well below the swap-free chain with the same network (Fig. 2c; fZ55 W2).

W4 & Q4: Hottest temperature, when swaps act, and how to set M and the ladder

We agree that the paper's ladders, set from small ablations, kept the hottest replica close to the target. We acknowledge that the choice of these hyperparameters was unclear, so we ran additional ablations of the hottest temperature λ_M and the number of replicas M at four target temperatures, on the toy and on robomimic (Figs. 9a and 15). They also answer Q4:

  1. Hottest temperature: λ_M ≈ 0.5 + 2λ_1, well above the target. Hotter targets want a hotter top replica, and the optimum is flat: any λ_M between 1.5 and 3 is close to the best at every target temperature. Our main experiments now use this wider range: on robomimic, λ_M = 2.4 against 1.1 before (App. C.5).
  2. Number of replicas: M = 4. With this ladder, four replicas balance the number of rungs against the steps each replica receives at a fixed budget: on robomimic they reach 74% success against 67% with two, and the rule is within 2 points of the best configuration at every target temperature (Fig. 15).

Swaps act as modes separate. The no-swap ablation (Fig. 2b; W1) supports the reviewer's reading: STEER and the swap-free chain have the same error while the modes overlap (t ≥ 0.6), and STEER's advantage appears as the modes separate. The SD3 acceptance rates (Fig. 17, left) show the same pattern at scale: the inner pairs of replicas swap almost always, while the outer pairs fall towards zero by mid-trajectory. We have revised the claim that exchange aids early exploration: swaps act during mode commitment, moving samples between modes at the point where a single chain would lock in.

W5: Number of replicas and ladder

We thank the reviewer for these corrections. We have:

  1. Cited Okabe et al. (2001) for the even–odd schedule.
  2. Removed the overmixing claim from Sec. 3.2.
  3. Removed the 20% acceptance-rate target from Fig. 12 (Fig. 17 in the revision), citing Lingenheil et al. (2009).

Our guidance: add replicas when the budget allows; at a fixed budget, use M = 4 with λ_M ≈ 0.5 + 2λ_1 (W4). With 100 steps per replica, more replicas improve the toy steadily and plateau on robomimic, as the reviewer predicts, with no overmixing (Fig. 14):

MToy, exact energies (TVD ↓)Toy, learned score (TVD ↓)Can (success % ↑)Square (success % ↑)Tool hang (success % ↑)
20.1250.14157.6 ± 3.778.4 ± 2.245.2 ± 2.6
40.0840.12360.8 ± 2.983.2 ± 2.042.8 ± 4.3
80.0480.10357.6 ± 2.181.2 ± 4.842.0 ± 5.9
160.0360.087
  • On can and square, success peaks at M = 4 and plateaus within error at M = 8. On tool hang, success is within error across M; single-chain baselines are in Fig. 6.

On the ladder. Instead of hand-picked endpoints, the ladder now follows the rule above, and its optimum is flat, so STEER does not need a finely tuned ladder (W4). We agree that adaptive ladders, such as those used in non-reversible PT, are a natural next step, and have added this to the limitations (Sec. 4).

Reviewer_9ZEqOpenReview ↗
Rating
3(+1)
Confidence
2
Informativeness
2
Presentation
3
Soundness
2
Read review
Paper summary

This paper studies how to improve sample quality in pretrained diffusion models through low-temperature sampling while preserving diversity and avoiding external verifiers. It is motivated by two limitations: verifier-based methods require an external objective that can be expensive or hard to specify, while existing temperature-scaling methods search only locally and may reduce diversity or distort mode weights. The proposed method runs multiple sampling trajectories at different temperatures and exchanges states according to a designed rule to facilitate global exploration. Samples from the low-temperature trajectory are retained as the final outputs. The paper evaluates the method on synthetic multimodal distributions, image generation, protein backbone generation, maze tasks, and robotic imitation learning, reporting improvements in sample quality while maintaining diversity.

Critical strengths
  1. The evaluation covers low- and high-dimensional settings, with and without verifiers, and includes both generation and control tasks. In particular, the low-dimensional results illustrate how sampling at multiple temperatures and exchanging states helps cover distant, isolated modes. These results support the motivation for improving diversity, while the broader evaluation demonstrates the method's relevance across varied settings.
Critical weaknesses
  1. In the verifier-free experiments (toy, SD3, protein, Robomimic), STEER uses M chains per output sample while TSR/CNS use a single chain, so these comparisons are not NFE-matched. The claim of no significant wall-clock overhead (Sec. 2.3) is not supported by measurements. Please report NFE, wall-clock time, and memory per output sample for all methods, and/or compare against baselines given the same M-fold budget.
  2. The core idea of coupling diffusion trajectories through replica exchange has been explored in prior work, including CREPE (which the paper compares against) and several 2026 methods such as Source Parallel Tempering, PATHS, and PT-MDM, which are not discussed. The distinct contribution appears to be a verifier-free, single-pass, temperature-indexed exchange using a score-based energy estimate. Please position the method relative to these works, and include an ablation with the same temperature ladder but no swaps to isolate the contribution of the exchange step.
Questions for authors
  1. Since FID does not separately quantify quality and diversity, could you include explicit diversity results for image generation in the main text(instead of appendix) ? Could you also provide a direct diversity evaluation for protein backbone generation? These results would make the evidence for preserving diversity easier to assess.

  2. How are the method's hyperparameters selected, and how sensitive are the results to these choices? Some settings appear to perform worse than single-chain sampling; could you clarify when this occurs and provide practical selection guidance?

  3. Across a sweep of fixed hyperparameter settings for each method, how do their quality-diversity Pareto frontiers compare under matched computational budgets? This comparison would clarify whether the advantage persists across operating points rather than only at selected settings.

  4. The role of state exchange in exploration could be investigated more directly. A case study tracing exchanges between high- and low-temperature chains could show whether additional high-temperature exploration reaches the low-temperature output. This would strengthen the empirical explanation for why the method improves diversity.

  5. The exchange criterion is shown to satisfy detailed balance in the isotropic Gaussian case, which provides a sanity check that exchanges do not corrupt the tempered targets; however, this analysis does not cover the multimodal regime where the claimed exploration benefit arises.

6 comments
AuthorsOpenReview ↗

Summary + Response to NFE-matched comparisons, cost, and diversity

Summary.

  • STEER beats TSR at the same NFE and wall-clock. It wins on the toy, robomimic, and FLUX.1 (W1).
  • Swaps drive the gains, not extra compute. Removing swaps on the same ladder raises the error, and STEER is the only verifier-free, training-free method that exchanges across temperatures in one pass (W2).
  • One hyperparameter, and diversity is preserved. STEER needs only the target temperature (Q2), covers all 40 modes on the toy, and raises recall without lowering precision on SD3 (Q1).

W1 & Q1 & Q3: NFE-matched comparisons, cost, and diversity

We now compare STEER with single-chain TSR at the same NFE, and report wall-clock time and memory per output sample. TSR (paper) is the paper's single chain with T steps (1/M of STEER's NFE); TSR (M-fold) runs M·T steps, matching STEER's NFE. We compare against TSR rather than CNS because STEER builds on TSR, the stronger single-chain method (Xu et al. 2025).

Toy: 40-mode GMM (Erives et al. 2025, He et al. 2026, Midgley et al. 2023). 50,000 samples at target temperature 1/3, learned score, 5 seeds (mean ± SE).

MethodMTNFEWall-clock (µs)Peak memory (KB)Distance TVD ↓
TSR (paper)11001007.54.20.159 ± 0.001
TSR (M-fold)140040029.74.20.153 ± 0.001
STEER410040028.216.90.123 ± 0.001
  • STEER keeps improving with compute; TSR does not. Extra steps barely change TSR's error, and its mode-weight error stays at 0.26 from 200 to 1600 NFE, while STEER's falls from 0.22 to 0.11 (Fig. 2b, 2e). STEER also covers all 40 modes in randomly drawn samples, where TSR misses one (Fig. 2a).

Robomimic. Success rate (%), 5 seeds × 50 rollouts (mean ± SE); cost per action chunk.

TaskMethodMTNFEWall-clock (ms)Memory (MB)Success (%) ↑
CanTSR (paper)110010077821.958.4 ± 3.1
TSR (M-fold)1400400305121.958.8 ± 2.2
STEER410040089323.960.8 ± 2.9
SquareTSR (paper)110010078122.280.8 ± 1.7
TSR (M-fold)1400400307622.280.8 ± 1.6
STEER410040089824.283.2 ± 2.0
Tool hangTSR (paper)110010077221.039.6 ± 0.7
TSR (M-fold)1400400307421.041.2 ± 2.6
STEER410040089422.942.8 ± 4.3
  • Higher success at the same NFE on all three tasks (Fig. 6).
  • 3.4× faster than a single chain at the same compute. TSR spends extra compute on more sequential steps (400), while STEER spends it on replicas run as one batch (100 steps): 895 vs. 3,070 ms per action chunk.

Images: FLUX.1-dev, with the same sampler as above, on 1,000 COCO prompts at 1024 × 1024 and target temperature 0.95. Costs are per image on one H100; memory is above the model weights.

MethodMTNFEWall-clock (s)Memory (MB)CLIP ↑
TSR (M-fold)111611623.547530.74 ± 0.10
STEER42911622.9188830.92 ± 0.10
  • When the model already saturates the GPU, STEER matches TSR's wall-clock time at equal NFE, with a CLIP score at least that of TSR (Table 2). Memory grows linearly with M, as the replicas form one batch.

These results are in the revised paper (Fig. 2, Fig. 6, Table 2), and Sec. 2.3 now states the cost: M× the NFE and memory of a single chain, with the replicas run in parallel.

Q1. Diversity. On SD3, raising the number of replicas from 2 to 6 increases recall without lowering precision (Fig. 17, right), which we now state in the main text (Sec. 3.2). On the toy, STEER covers all 40 modes and has the lowest mode-weight error of all methods (Fig. 2a; HAee W2). For proteins, STEER matches or exceeds TSR's designability in every length bin (Fig. 16); a direct structural-diversity measure is not part of this revision.

Q3. Frontiers at matched compute. The SD3 frontiers (Figs. 4 and 17) compare against a single chain at 1/M of STEER's compute. At matched compute, STEER beats TSR across budgets on the toy, beats CREPE, SMC and FKC at every budget with exact energies (Fig. 2d; W2), and is at least on par with TSR on FLUX.1 (Table 2), so its advantage is not tied to one operating point.

AuthorsOpenReview ↗

Positioning, no-swap ablation, and detailed balance

W2 & Q4 & Q5: Positioning, no-swap ablation, and detailed balance

Many of the prior works on replica exchange diffusion sampling require an external verifier function. We have updated the related work section (App. A) to position STEER against them, as summarised below.

MethodHow it differs from STEERLimitation
Source PTSwaps initial noise, not trajectory statesNeeds external reward
PATHSReward-tempered chains on initial noiseNeeds reward; initialization only
PT-MDMReward-tempered masked diffusion for sequencesNeeds external reward
PT-DenoiseUses exact problem energyNeeds exact energy; no guarantee
CREPESwaps across diffusion times via MCMCBurn-in; accumulating acceptance errors
GePTTransport-based swaps; needs known densityNeeds density and training
PTSDTraining-time method; needs target energyNeeds energy and training

CREPE is the only one of these methods that needs neither a verifier nor additional training. CREPE belongs to a family of tempering algorithms, along with SMC- and FKC-based algorithms, that use the Radon–Nikodym Estimator (RNE) to estimate energy differences during diffusion by integrating over timesteps. These verifier-free tempering algorithms can be split into two parts: the swap mechanism that acts on an energy difference, and how that energy difference is computed. We isolate STEER's advantage on each empirically, then ablate the swaps themselves. All three experiments use the 40-mode GMM from the replica exchange literature, including the RNE paper (Erives et al. 2025, He et al. 2026, Midgley et al. 2023), with 5 seeds of 50,000 samples.

1. A better swap mechanism. We give every method the same exact energy difference and the same total NFE (score evaluations). This isolates the swap mechanism: exchange (STEER), adjacent-timestep swap (CREPE), or reweight-and-resample (SMC, FKC). Under exact energies, STEER outperforms the other three at every budget (Fig. 2d).

~ NFECREPESMCFKCSTEER
2000.1600.2080.1630.125
4000.1500.1780.1370.084
8000.1250.1540.1160.048
16000.1000.1280.0960.036

Distance TVD ↓, means over 5 seeds; standard errors are at most 0.003. We include additional results with W2 distance in the revised paper (Fig. 10a), on which STEER similarly has the lowest error at every budget.

We also compare swap mechanisms on identical dynamics in HAee W2, item 2.

2. A more accurate energy estimate. STEER's swap rule costs no extra NFE: it reuses the scores each replica already computes for its own denoising step. Both STEER and RNE compute the energy difference by integrating the score, but along different paths:

  • STEER (path integral): integrates along each replica's own denoising trajectory back to its initial noise, where the two replicas are compared directly. Every point on this path already has a score.
  • RNE (time integral): simulates a new forward-noising path from each state up to t = T and integrates along it, which needs new score evaluations. We use the RNE kernels from CREPE's released code.

Adding points to either integral (more NFE) reduces its error. We use the exact score to isolate how each integral accumulates error. We describe this estimator in App. D.1, and the comparison is in Fig. 9b, c.

NFE per stateEnergy-difference error ↓Acceptance-probability error ↓
RNE (time)STEER (path)RNE (time)STEER (path)
1001.7640.7920.2280.157
1991.3000.5870.1920.123
3970.9420.4260.1570.092
7930.6770.3050.1260.067

Mean absolute error between adjacent replicas, against the exact log⁡pt\log p_t; standard errors are below 0.004.

  • STEER's path integral has about half of RNE's error at every budget.

3. Swap mechanism reduces score-rescaling error amplification. We run the same temperature ladder with swaps turned off and track the error along the sampling trajectory (Fig. 2b). In mid-generation, the error from score rescaling keeps growing in the swap-free chain, while STEER's swaps stop its growth. STEER finishes with an energy TVD of 0.178, against 0.240 without swaps. The full trajectory is in our response to HAee W1.

AuthorsOpenReview ↗

Detailed Balance cont. + Hyperparameter selection and sensitivity

Q4. Tracing exchanges from hot to cold replicas. We tag every sample with the replica it started in and follow it through the swaps (40-mode GMM, M = 4, 400 NFE, learned score, 5 seeds). STEER's final samples are drawn almost evenly from all four starting replicas (25.0%, 25.0%, 25.0% and 25.0%, coldest first; standard errors at most 0.1%). About 75% of the outputs therefore began in a hotter replica and reached the target temperature through swaps.

The no-swap ablation (Fig. 2b; HAee W1) shows when this happens: the swap-free chain and STEER have the same error while the modes overlap, and STEER's advantage appears as the modes separate in later timesteps of sampling.

Q5. Detailed balance beyond the isotropic Gaussian. We agree that beyond the isotropic Gaussian, STEER's practical swap does not satisfy detailed balance exactly. With exact energies, the swap kernel satisfies detailed balance for any pair of densities (App. B.2), but Thm. 2.1, which makes the score-based energy estimate exact, holds only for isotropic Gaussian data. We have stated this limitation explicitly in the revised paper (App. B.5).

Exact tempered sampling from a pretrained diffusion model is, however, an open problem for general targets, and every verifier-free alternative is also an approximation: TSR and CNS are exact only for Gaussian data, and CREPE, SMC and FKC rely on RNE energy estimates. STEER does the best it can within this setting, and we measure how close it gets on the 40-mode GMM:

  • With exact energies, swaps nearly halve TSR's error (0.153 to 0.084 TVD; HAee W3).
  • With the learned score, STEER beats the swap-free chain at every score checkpoint (Fig. 2c; fZ55 W2).
  • Its energy estimate has about half the error of RNE's (item 2 above).

Q2: Hyperparameter selection and sensitivity

We acknowledge that the submission did not make clear how to choose STEER's hyperparameters, so we ran additional ablations of the hottest temperature λ_M and the number of replicas M at four target temperatures, on the toy and on robomimic (Figs. 9a and 15). They show that STEER effectively has one hyperparameter, the target temperature λ_1, which any temperature sampler requires; λ_M and M follow from it:

  • Hottest temperature: λ_M ≈ 0.5 + 2λ_1. The top replica must be hot enough to move between modes, but not so hot that colder neighbours rarely accept its states. On robomimic, the best λ_M rises with the target temperature, from 1 at λ_1 = 0.3 to 2 at λ_1 = 0.8, and this rule tracks it (Fig. 15). Results are not sensitive to the exact value: any λ_M between 1.5 and 3 is close to the best at every target temperature.
  • Number of replicas: M = 4. Under our λM\lambda_M rule, four replicas work best. We fix the budget in our comparison: we set more replicas to mean fewer steps each. Following this guidance, STEER matches or exceeds a single chain at matched compute on every task we evaluate (Fig. 6, Table 2). In the original submission, its gain shrank with two replicas on SD3. Here, every swap exchanged states between the two ends of the ladder. We believe these states were far apart, and in a high-dimensional space like SD3's, the energy integral between distant states become harder to estimate, so swaps become less reliable.

Comments on the paper

2 comments

We thank the reviewers for detailed and constructive reviews. The main additions, all now in the revised paper:

  • STEER wins at matched compute. We now compare every method at equal NFE. On equal-compute experiments for a 40-mode GMM and robomimic, STEER reaches lowest TVD and highest success rate. These comparisons include SMC, FKC and CREPE. Its replicas run as one batch, so at equal NFE it is up to 3.4× faster than a single chain, which must spend the extra compute on sequential steps.
  • Swaps correct TSR's error. Turning on one error source at a time shows that swaps nearly halve TSR's error even with exact energies, that STEER's energy estimate has about half the error of the RNE estimator behind CREPE, SMC and FKC, and that the gain persists with an undertrained score network.
  • STEER needs one hyperparameter. New ablations over the number of replicas and the hottest temperature, at four target temperatures on the toy and robomimic, show that the target temperature λ1\lambda_1 sets the remaining hyperparameters. Four replicas, with the hottest at λM≈0.5+2λ1\lambda_M ≈ 0.5 + 2\lambda_1, balance the width of the ladder against the steps each replica receives at a fixed budget.
Area_Chair_MviFOpenReview ↗

Reminder: Please Respond to the Authors’ Rebuttal

Dear Reviewers,

Thank you for your time and effort in reviewing this submission. Please take a moment to read the authors’ responses and follow up with any remaining questions or concerns. If the responses have changed your assessment, please update your review and score accordingly.

Your engagement is greatly appreciated and will help us reach a well-informed decision.

Best,

AC

From OpenReview, CC BY 4.0. Authors as signed there; shown unedited.

Related papers

Loading the map…

Discussion (0)

Sign in to comment.