- Rating
- 3(+2)
- Confidence
- 3
- Informativeness
- 2
- Presentation
- 3
- Soundness
- 2
Read review
Paper summary
This paper proposes STEER, a verifier-free inference-time steering method that samples the tempered distribution of a pretrained diffusion model. STEER runs diffusion chains at different temperatures in parallel and rescales the score of each chain to its temperature. STEER adapts parallel tempering, a technique that MCMC uses for mode mixing, to the inference of pretrained diffusion models. The authors prove that the swap rule is exact for isotropic Gaussian data and does not amplify the bias of score rescaling in general. They validate STEER on 2D Gaussian mixtures, Stable Diffusion 3, protein backbone generation, PointMaze planning, and Robomimic manipulation.
Critical strengths
S1. STEER is a training-free, verifier-free plug-in with no extra network evaluations. The swap rule reuses scores already computed for denoising, so its cost equals that of independent chains. The same algorithm applies to images, proteins, planning, and robot policies.
Critical weaknesses
W1. The advantage over existing verifier-free population methods is insufficiently established. As acknowledged in Appendix A, Feynman–Kac Correctors already provide a verifier-free framework for targeting annealed marginals proportional to using pretrained scores and SMC resampling. However, FKC is absent from the experimental comparisons. The memory-based argument against SMC is also insufficient: STEER maintains replicas, so restricting interactions to adjacent replicas does not eliminate population storage costs. Without a comparison under matched computational budgets and measured memory usage, it remains unclear what practical advantage STEER offers over this relevant prior approach. This omission weakens the paper’s justification for its claimed gap in existing verifier-free methods.
W2. Robustness of the swap mechanism to score estimation error is not established. Hot replicas can visit regions that are poorly represented under the model’s training marginals, where score accuracy may be poorly controlled. Because STEER estimates swap acceptance probabilities from these learned scores, estimation errors can distort acceptance decisions and bias the target replica. The exactness result assumes the true score under standard Gaussian data, while the non-amplification proposition requires a target-preserving swap kernel. Neither establishes robustness of the practical kernel to score error. Although the pretrained-model experiments demonstrate empirical utility, they do not isolate this sensitivity. A controlled study comparing exact and learned scores on the toy distributions, or systematically perturbing score accuracy, would help establish when replica exchange remains beneficial.
Questions for authors
Q1 (experiment request). Could you compare STEER with FKC temperature annealing at the same number of particles and the same compute, on the toy mixtures and at least one high-dimensional task?
Q2 (clarification). How does the memory cost of keeping replicas in STEER differ from the cost of keeping particles in SMC?
Q3 (experiment request). On the toy mixtures, where the true score is known, how much does the swap acceptance from the learned score deviate from the acceptance from the true score? Alternatively, how do STEER and TSR compare when the base model is undertrained?
Q4 (clarification). Could you report error bars or per-seed results for the protein and SD3 experiments?
3 comments
Thank you for the detailed rebuttal and additional experiments. I appreciate the authors' efforts to address my concerns.
W1: The new comparisons with FKC demonstrate STEER's empirical advantage at matched compute. Although STEER has higher memory usage, the authors acknowledge this limitation and have revised their claims accordingly. I consider W1 addressed.
W2: The experiments across training checkpoints provide convincing evidence that STEER remains effective with undertrained score networks. This substantially addresses my main concern about robustness to score estimation error.
However, two concerns remain:
-
Comparison with single-chain TSR: My original Q3 requested a compute-matched comparison with single-chain TSR across training checkpoints. The swap-free temperature ladder is a useful ablation, but it does not replace this comparison.
-
Uncertainty in high-dimensional experiments: Figure 4 shows that improvements vary across configurations. In particular, STEER does not consistently outperform the baseline in protein generation. Given the stochastic nature of this kind of method, multi-seed evaluations are important to establish the reliability of these improvements. While confidence intervals were added for protein designability, the main SD3 and protein results still lack uncertainty estimates. I understand that these experiments may be computationally expensive, but if the runtime is reasonably manageable, I would encourage multi-seed evaluations for at least representative configurations.
Given that the rebuttal has substantially addressed my main concerns, I am open to raising my initial rating.
We thank the reviewer for the careful follow-up, and are glad the FKC comparison and checkpoint study addressed W1 and W2. We address the two remaining points below.
1. Compute-matched single-chain TSR across checkpoints. We apologise for the confusing label: the swap-free baseline in our W2 table is single-chain TSR, given the same NFE as STEER. Relabelled, it is the comparison the reviewer requested in Q3:
| Training iterations | Energy TVD, TSR (400 NFE) | Energy TVD, STEER (400 NFE) | Distance TVD, TSR (400 NFE) | Distance TVD, STEER (400 NFE) |
|---|---|---|---|---|
| 20,000 | 0.258 | 0.195 | 0.148 | 0.121 |
| 50,000 | 0.248 | 0.182 | 0.149 | 0.113 |
| 100,000 | 0.247 | 0.181 | 0.150 | 0.114 |
| 150,000 | 0.236 | 0.161 | 0.150 | 0.112 |
| 200,000 | 0.242 | 0.174 | 0.153 | 0.119 |
- At matched NFE, STEER beats single-chain TSR at every checkpoint, on both metrics, including the least-trained network.
- Giving TSR more steps does not close the gap. Its error barely changes from 100 to 400 NFE (9ZEq W1), so the gain comes from the swaps, not from extra compute.
We have updated the paper to label this baseline as single-chain TSR at matched NFE (Fig. 2c).
2. Multi-seed evaluation in high dimensions. We agree that multiple seeds are needed, and ran a multi-seed protein study on FoldingDiff, comparing STEER with single-chain TSR at the same NFE on long sequences across target temperatures . We report FID at matched designability, as mean ± SE across 5 seeds.
FID at matched designability (each row compares the TSR and STEER settings with about the same designability):
| Designable % | FID ×10⁶ ↓, TSR | FID ×10⁶ ↓, STEER |
|---|---|---|
| ~28% | 4.58 ± 0.02 | 4.46 ± 0.03 |
| ~33% | 4.89 ± 0.03 | 4.66 ± 0.04 |
| ~36% | 5.09 ± 0.02 | 4.89 ± 0.06 |
| ~41% | not reached | 5.14 ± 0.04 |
| ~45% | not reached | 5.36 ± 0.05 |
- At matched designability, STEER has lower FID at every level. Its whole trade-off curve between designability and FID lies below TSR's.
- STEER reaches higher designability. TSR does not exceed 36% designable at any λ1, while STEER reaches 45%.
- STEER is more designable at every λ1. Pairing STEER and TSR on the same seed, batch and length, STEER gains 5.7 points on average.
This addresses the inconsistency the reviewer noted in Fig. 4: with multiple seeds and matched compute, STEER's protein improvement holds at every operating point we test. We have added these results to the revised paper, with error bars across seeds. For text-to-image, the FLUX.1 results report CLIP score with standard errors over 1,000 prompts (Table 2).
We thank the reviewer again for engaging with our rebuttal and for being open to raising their rating.
Summary + Comparison to FKC + Robustness to score error
Summary.
W1 & Q1 & Q2: Comparison with FKC; memory
We add FKC (and SMC with RNE weights) as baselines under matched NFE, and report measured wall-clock time and memory per output sample. Toy results are on the 40-mode GMM with a learned score, 50,000 samples, 5 seeds (mean ± SE). For SMC and FKC, M is the number of particles.
Q1. FKC at the same particles and compute. On robomimic, FKC and SMC run 4 particles, the same as STEER's 4 replicas, at matched NFE, and STEER has a higher success rate on all three tasks (Fig. 6; HAee W2, item 3). On the toy, every method receives the same NFE, and FKC's particle count was tuned at that budget (8 particles × 50 steps, table above); STEER has lower error than FKC at every budget (Fig. 2d, e; HAee W2).
Q2. Memory of M replicas vs. M particles. Both store M states, so the memory is of the same order (item 3 above).
W2 & Q3: Robustness to score error
We ran both controlled studies the reviewer proposes on the 40-mode GMM, at 400 NFE per sample; they also answer Q3.
1. The learned score barely changes STEER. 5 seeds.
Standard errors are at most 0.002.
2. Swaps help even an undertrained network. We train networks for an increasing number of iterations and report TVD of the final samples (seed 0; Fig. 2c).
3. Swap acceptance barely changes. Replacing the exact score by the learned score in STEER's swap energy raises the error of the swap acceptance probability by at most 0.006 (e.g. 0.180 to 0.181 at 100 NFE per state), and of the energy difference by at most 0.04 (Fig. 9b, c). With either score, the path integral has about half the energy-difference error of the RNE estimator used by CREPE, SMC and FKC (9ZEq W2).
Q4: Error bars for protein and SD3
Protein designability is reported with 95% Wilson intervals (Fig. 16): STEER improves pooled designability from 35.9% to 44.2% (56/156 vs. 69/156). Each SD3 point is a single run of 10,000 samples. On FLUX.1, we report CLIP with standard errors over 1,000 prompts (Table 2), and every toy and robomimic result is reported as mean ± SE over five seeds.