acceptodds
Under review as a conference paper at ICLR 2027

Risk-Calibrated Proposal Transport for Finite-Particle Diffusion Steering

Abstract

Inference-time steering combines pretrained diffusion experts or rewards without retraining by changing the dynamics that transport noise to data. Feynman-Kac correction compensates for proposal mismatch through importance-weighted sequential Monte Carlo (SMC), whose finite-particle behavior depends on the proposal. Variance-controlling guidance (VCG) improves that proposal by fitting a linear drift correction to minimize empirical log-weight-rate variance. Although its population optimum cannot worsen residual variance, finite-particle VCG can nearly eliminate its fitting residual while increasing residual risk on new states by orders of magnitude. The resulting update can degrade unweighted generation or accelerate particle collapse. We show that the centered Feynman-Kac rate is the normalized transport residual and that expected out-of-fit benefit is exactly population headroom minus coefficient-estimation penalty. Under regularity assumptions, a Wasserstein analysis bounds the unweighted proposal's terminal error using this residual. These results motivate Risk-Calibrated Proposal Transport (RCPT), which uses deletion leave-one-out residuals to calibrate the retained fraction of the VCG update, adding no model calls and only small linear-algebra overhead. Experiments on 2D checker distributions, scaffold decoration, molecular property optimization, and class-conditional CIFAR-10 generation demonstrate recovery from harmful fitted updates. Across molecular and image domains, RCPT consistently lowers mean residual risk and improves a broad range of terminal metrics relative to uncalibrated VCG.

Primary area:
generative models
Keywords:
diffusion models, inference-time steering, Feynman-Kac correction, sequential Monte Carlo, proposal adaptation, variance-controlling guidance, finite-particle estimation, leave-one-out calibration, risk calibration, importance sampling, transport error, molecular generation

Reviews

Among:
Shape:
1234this paper, Top 15%
Figure 1. Mean review rating of the reviewed papers at ICLR 2027. This paper averages 3.00 over 3 reviews, above 85% of the others.
Reviewer_CK4JOpenReview ↗
Rating
3
Confidence
3
Informativeness
2
Presentation
2
Soundness
2
Read review
Paper summary

This paper studies overfitting in variance-controlling guidance (VCG) for diffusion steering. VCG fits a linear drift correction by minimizing the empirical variance of the Feynman–Kac log-weight rate. With limited fitting particles, this can lead to poor generalization beyond the fitting population. The authors propose RCPT, which uses leave-one-out residuals to estimate a scalar shrinkage factor for the VCG correction. Experiments cover synthetic distributions, molecular generation, and CIFAR-10, showing promising results.

Critical strengths
  • The paper identifies a relevant finite-particle failure mode of VCG.
  • RCPT is simple and computationally inexpensive, requiring no additional model evaluations.
Critical weaknesses
  • Limited novelty and unclear advantage over regularized VCG: RCPT is essentially a scalar shrinkage of the fitted VCG coefficients, and the risk decomposition and oracle shrinkage rule largely follow from the quadratic regression objective. More importantly, Eq. (8) already includes ridge regularization, which controls overfitting during fitting and shrinks different eigendirections by different amounts, whereas RCPT applies a single scalar after fitting. Although Table 1 shows that RCPT outperforms LOO Ridge, it remains unclear whether this comes from a better risk estimator or from differences in how regularization is selected. A more controlled comparison with adaptively tuned ridge would help establish the benefit of RCPT.
  • The risk estimator may be unreliable for online interacting particles: RCPT estimates the coefficient-estimation penalty from leave-one-out residuals and the ESS of the importance weights (Eq. 10). In online SMC, however, particles are dependent due to resampling and the shared adaptive proposal. The ESS does not account for this dependence, and deleting one particle does not remove its influence on the remaining population. The penalty may therefore be underestimated, so that RCPT retains too much of the VCG update. The paper acknowledges that its frozen-schedule guarantees do not extend to online interacting particles, but this is an important limitation since RCPT is also proposed for online deployment.
Questions for authors
  • Can the authors compare RCPT with ridge-regularized VCG across a range of regularization strengths, using the same fitting particles and evaluation protocol on all benchmarks?
  • How accurate is the estimated penalty in online SMC, where particles are dependent? In particular, can the authors compare it against the empirical estimation error and check whether particle dependence leads to underestimation?
  • In the online QM9 experiment, VCG and RCPT perform similarly. How often does VCG produce harmful updates in online settings, and in which regimes does RCPT give a clear improvement?
4 comments
AuthorsOpenReview ↗

[1/2] Response to Reviewer CK4J

Thank you for asking for matched regularization comparisons and direct tests of online reliability. We identify when finite fitting reverses the benefit of an FK correction, explain how the error affects sampling, and use that analysis to choose how much correction to deploy without extra model calls. On frozen scaffold and online QM9, RCPT improves generation over both Anchor and VCG. The dependence audits show where the online penalty can fail. References are to the revised manuscript. Intervals are nominal paired 95% intervals over whole fits, runs, or pockets.

1. What RCPT adds to regularized VCG (W1, Q1)

A fitted FK correction can have nearly zero fitting residual yet increase independent residual risk and worsen generated samples. Theorem 1 explains this reversal through P−HP-H. Theorem 4 and Proposition 1 connect the residual to unweighted transport error and local correction-weight inflation. Building on classical shrinkage, our structured deletion-LOO rule uses this deployment risk to choose how much correction to retain, with no additional model calls.

For fixed features before safeguards, ridge scales covariance eigendirections by 1/(cj+η)1/(c_j+\eta), while RCPT scales the fitted direction by α\alpha. For α>0\alpha>0, matching RCPT with a new ridge would require η′=(cj+η)/α−cj\eta'=(c_j+\eta)/\alpha-c_j for every active eigenvalue cjc_j. In general no single value does so. Which choice works better is an empirical question.

Fixed ridge. We deployed four strengths (10−6,10−4,10−2,0.110^{-6},10^{-4},10^{-2},0.1) on the same ten paired CIFAR runs. Stronger ridge closes much of the gap, but at least six of seven endpoint means favor RCPT at every strength. At 0.1, ridge reaches 60.219% independent accuracy, 0.2583 KID and 78.9632 FID, versus RCPT's 60.906%, 0.2381 and 78.5020. DLSM-classifier accuracy favors ridge at 0.01 and 0.1 (91.469% and 91.844% vs 91.375%). Table L.7 gives the full results. Anchor (α=0\alpha=0) does better than RCPT on six endpoints and has nearly identical mean FID. Original DLSM is a separate baseline (Table 5).

Checker and scaffold also have strength-by-strength reference-risk comparisons (Figure L.1, Table L.8). On eight scaffold pockets with four fits each, ridge alone has lower mean reference risk than RCPT applied to the same candidate throughout the grid. Fixed-strength molecular deployments cover 10−3,10−2,10−110^{-3},10^{-2},10^{-1} on ten scaffold pockets and in a four-seed, three-field QM9 audit (Tables I.11–I.12).

Selected ridge. We minimize deletion loss on the relative grid {0,10−6,10−4,10−2,0.1,1,10,100}\{0,10^{-6},10^{-4},10^{-2},0.1,1,10,100\}, scaled by the training covariance trace per coefficient and added to the numerical ridge. Selection uses fitting data alone. Independent states and terminal outcomes in all four domains are reserved for evaluation (Appendix L.6).

ComparisonResult
Checker, N=16N=16RCPT Risk/Anchor 0.4931 vs selected ridge 0.8306, difference −0.3375 [−0.7484, −0.0510] (Table 1).
Same ridge candidate, Checker N=16N=16RCPT changes risk by −0.4301 [−0.5788, −0.2721] at ridge 0.01 and +0.0746 [0.0432, 0.1067] at ridge 1 (Table L.8).
QM9At N=36N=36, mean MAE and success favor RCPT over selected ridge, with unresolved differences. RMSE improves by −0.0101 [−0.0191, −0.0013]. At N=144N=144, ridge has lower MAE (RCPT minus ridge +0.0041 [0.0002, 0.0080], Table L.1).
Scaffold, 76 pockets, fit-16Unique-useful yield is 0.2508 for RCPT vs 0.2475 for selected ridge. The interval spans zero (Table L.5).
CIFAR, ten paired runsAll seven means favor RCPT over selected ridge on the same fixed-guidance base, with resolved gains in KID (−0.0240 [−0.0409, −0.0077]) and FID (−0.4608 [−0.7096, −0.2369]) (Table L.7).

The risk and regret comparisons with Direct LOO use the same fitted direction to isolate amplitude selection. At Checker N=16N=16, RCPT-minus-Direct risk is −0.6331 [−1.3630, −0.0669]. Holding direction and safeguards fixed, the criterion-by-continuity factorial attributes most of the N=16N=16 gain to the structured criterion. Continuity adds resolved benefits at N=8,32N=8,32 and in all eight saved online budget/driver conditions (Tables L.16–L.17). At N=64N=64, Direct LOO and selected ridge can have lower risk.

Frozen scaffold deployment improves useful yield over both Anchor and VCG. At fit-64 (12 coefficients), unique-useful yield is 27.1% vs 23.6% for Anchor and 20.0% for VCG across 76 pockets. The paired gains are +3.45 pp [0.66, 6.17] over Anchor and +7.07 pp [3.04, 11.10] over VCG (Table 3). Both intervals exclude zero at this larger fitting budget. At fit-16, guarded VCG and Direct LOO have higher means than RCPT (0.2656 and 0.2771 vs 0.2508), with intervals spanning zero. Guarded VCG's recovery shows that safeguards explain much of the gain over raw VCG (Appendix L.4). The matched Checker factorial and the QM9 comparisons provide evidence specific to the selection rule.

AuthorsOpenReview ↗

[2/2] Response to Reviewer CK4J

2. Online penalty accuracy and dependence (W2, Q2)

Your reliability concern is confirmed: the penalty is not quantitatively calibrated across online populations, and the implemented rule does not guarantee online safety. Equal weights and observation-wise deletion can overstate information when resampling leaves correlated copies.

In the controlled experiment, nominal NN and weight ESS remain 64. Reducing independent ancestors from 64 to 16 increases RCPT Risk/Anchor from 0.2639 to 16.4102. All 100 ensembles are harmful in the latter case. With eight ancestors, the estimated-penalty/reference-error ratio is about 3.15×10−83.15\times10^{-8} and mean amplitude is 0.99997, despite Risk/Anchor 75.7924 (Table L.11). Deleting one copy leaves its relatives in the fitting data.

On RCPT-driven Checker histories (Table L.12):

NNEstimated penalty / reference error, 95% CIFull VCG riskRCPT risk, 95% CIHarmful integrated RCPT runs
80.1047 [0.0156, 0.2644]510.96.042 [1.533, 14.095]7/10
160.2131 [0.1017, 0.6544]50.420.977 [0.620, 1.506]3/10
320.4415 [0.1081, 1.218]5.0410.929 [0.540, 1.561]2/10
640.3149 [0.2042, 0.4477]0.70930.3814 [0.3452, 0.4292]0/10

Risk is normalized by Anchor. Ratios use run means after time integration. On separately VCG-driven histories at N=8,16,32N=8,16,32, penalty/error ratios are 3.402, 8.488, and 2.475. QM9 ratios range from 1.5–13.7 at N=18N=18 and 0.032–0.263 at N=144N=144, across drivers and reference banks (Table L.13). Thus underestimation occurs, but its direction depends on the fitting population.

Reference coefficient error includes bias and evaluation-law mismatch, rather than only sampling variance. Whole-run intervals condition on finite reference banks. Appendix L.8 reports both banks and leave-one-population-out sensitivity. Some RCPT–VCG contrasts become unresolved under this sensitivity.

The exact fixed-law H/P identity permits dependent fitting data. The iid approximation motivating the proxy and the conditional bound in Theorem 3 do not establish accurate estimation for arbitrary online particles. We therefore evaluate online use by measuring both local harm and terminal generation outcomes.

3. Online harmful updates and a clear-benefit regime (Q3)

Full VCG is frequently locally harmful at low QM9 support, and RCPT reduces that frequency. At N=36N=36, RCPT also improves actual generation over both Anchor and VCG. Local harm means reference residual risk above Anchor on the same states. It is measured at 21 fixed nodes per run with time weights summing to one, separately from terminal failure probability.

Eight runs per driver and budget give these local-harm rates (Table L.13):

NN, driverBankFull VCG, 95% CIRCPT, 95% CI
18, RCPTA54.4% [49.4, 60.1]9.7% [3.8, 16.2]
18, RCPTB57.6% [53.8, 61.3]11.6% [6.9, 16.5]
18, VCGA55.7% [50.7, 61.3]6.9% [3.1, 10.0]
18, VCGB60.7% [53.8, 67.5]10.6% [6.9, 14.4]
144, RCPTA6.2% [4.4, 7.5]3.8% [1.6, 5.9]
144, RCPTB10.6% [8.1, 13.1]9.1% [5.9, 12.5]
144, VCGA6.2% [4.4, 7.5]5.3% [3.4, 6.6]
144, VCGB10.6% [8.1, 13.1]9.7% [7.2, 12.2]

Bank A combines four independent 144-particle systems. Bank B combines two 288-particle systems. Each totals 576 reference particles. The intervals use whole runs.

In the separate eight-run generation comparison at N=36N=36, terminal-weighted MAE is 0.7926 for RCPT, 0.9403 for Anchor, and 0.8238 for VCG. Paired differences are −0.1477 [−0.2827, −0.0038] and −0.0311 [−0.0476, −0.0138]. Terminal-weighted target success improves by +20.94 pp [6.00, 35.58] and +4.53 pp [2.25, 6.69], and mean distinct successes tie VCG at 12.875. Mean amplitude is 0.733 (Table 4). Both endpoints improve over each comparator with paired intervals excluding zero. VCG itself improves on Anchor here, so RCPT improves an already useful correction.

At N=144N=144, RCPT retains mean amplitude 0.945 and all 144 lineages, with 45.50 distinct successes versus VCG's 45.25. At N=18N=18, two RCPT runs lose ancestry, its MAE contrast with VCG is unresolved, and it has fewer distinct successes. The complete eight-run-per-budget sweep includes these outcomes (Appendix L.2). The evidence supports useful online operation in measured regimes, while leaving the reliability limitation intact.

The practical finding and its theoretical explanation remain useful even where another regularizer is competitive: a low fitting residual is insufficient evidence to deploy an FK correction. The matched comparisons show when the proposed low-cost decision rule helps, and the molecular endpoints establish benefits beyond that diagnostic observation.

Thank you for the detailed response and the additional experiments.

The new ridge comparisons and dependence audits address my questions, and the QM9 results give a clear example where RCPT improves over both Anchor and VCG.

At the same time, it shows that the advantage over tuned ridge depends on the regime, and that the penalty can be underestimated in online SMC. I think these limitations should be stated clearly in the main text. I will keep my score.

We are glad that the new ridge comparisons and dependence audits addressed your questions. Following your suggestion, we have revised Section 7 to make both limitations clear in the main text. Thank you again for the constructive review and helpful suggestions.

Reviewer_abQuOpenReview ↗
Rating
3(+1)
Confidence
2
Informativeness
2
Presentation
1
Soundness
2
Read review
Paper summary

The paper studies inference-time steering of pretrained diffusion models, where a cheap combined drift (the "Anchor") is corrected with Feynman–Kac (FK) importance weights and sequential Monte Carlo (SMC). It focuses on variance-controlling guidance (VCG) from DriftLite, which fits a linear drift correction by least squares on the FK log-weight rate.

The authors observe that with few fitting particles (e.g., 16 particles for 12 coefficients), VCG overfits: in-sample residual nearly vanishes while out-of-sample residual can grow by orders of magnitude. They decompose expected out-of-fit risk into population headroom H minus estimation penalty P, and propose RCPT, which scales the VCG coefficient.

Experiments cover a 2D checker target, 76-pocket CrossDocked scaffold decoration, QM9 reward steering, and class-conditional CIFAR-10.

Critical strengths
  1. The paper identifies a real failure mode of VCG at small fitting budgets and documents it clearly. This is a useful warning for practitioners of FK-corrected steering.
  2. The method is simple and cheap. It adds no expert or denoiser calls, and the reported overhead is 0.081% of QM9 generation time.
  3. The empirical scope is broad, spanning a toy target, two molecular tasks, and images. Several comparisons use paired bootstrap intervals.
Critical weaknesses
  1. The writing is very hard to follow. Particles, importance weights, ESS, the propagate–reweight–resample loop of SMC, FK weighting, and genealogy collapse are all assumed. Eq. (2) introduces the FK rate g without explaining that it is the rate at which each particle's log-weight grows. Without this, Sections 3–4 cannot be read on their own. I recommend a preliminaries section with the introduction for FK-corrected SMC that shows where VCG and RCPT enter. Terms such as "fit-to-deployment inversion", "Proposal-Only", "common-state vs. active-path risk", "realized ray", "reference-ray oracle", and "structured deletion-LOO" are introduced quickly or only in the appendix. Many sentences are heavily compressed and hedged, which makes the claims hard to extract.

  2. Once the notation is unpacked, according to my understanding, RCPT decides whether, and how much, to accept a proposed update to the sampler. This is conceptually close to the accept/reject logic of rejection sampling and Metropolis-type moves: keep a candidate only if it is judged to improve on the current state, otherwise fall back. The positive part sets α = 0 whenever the estimated penalty exceeds the fitted gain. Mean amplitudes are 0.19 on scaffold and 0.11 on CIFAR, so the update is mostly rejected and the sampler reverts toward the Anchor.

  3. Most reported gains are relative to a VCG fit that is failing, not relative to the uncorrected Anchor. Against the Anchor, RCPT's benefit is marginal and sometimes negative, as the paper's own numbers show. For instance, over the Anchor RCPT raises scaffold validity only from 0.425 to 0.434 (Table 3), and its CIFAR-10 independent-accuracy gain is a single image out of 320 (Table 5), whereas the Anchor beats RCPT in SMC W2 at every common-fit Checker budget (Table I.4).

Questions for authors
  1. Can you report paired RCPT-minus-Anchor differences with confidence intervals for every main-text endpoint, not only RCPT-minus-VCG?
  2. How does a binary accept/reject rule (α ∈ {0, 1} chosen by LOO residual against the Anchor) compare with the structured α?
  3. Could you provide a more readable writing framework to introduce your work?
4 comments
AuthorsOpenReview ↗

[1/2] Response to Reviewer abQu

Thank you for identifying the need for complete Anchor comparisons, a binary baseline, and clearer sampling preliminaries. The central finding is that a population-improving FK correction can become harmful after finite-particle fitting. The revision connects this practical failure to its headroom–estimation-error explanation and to inexpensive control with demonstrated generation benefits. We address your three questions below, grouping the related weaknesses with them. References are to the revised manuscript. Intervals are nominal paired 95% intervals over whole fits, runs, or pockets.

1. Benefit beyond Anchor and complete uncertainty (Q1, W3)

The revision demonstrates benefit beyond both Anchor and VCG in online QM9 and frozen scaffold deployment. Table L.14 reports every main-text Anchor contrast, including unfavorable and unresolved outcomes.

On QM9 at N=36N=36, eight paired runs give the following means. MAE and target success use terminal SMC weights.

OutcomeAnchorVCGRCPTRCPT − Anchor, 95% CIRCPT − VCG, 95% CI
MAE ↓0.94030.82380.7926−0.1477 [−0.2827, −0.0038]−0.0311 [−0.0476, −0.0138]
Target success ↑29.18%45.60%50.13%+20.94 pp [6.00, 35.58]+4.53 pp [2.25, 6.69]
Distinct successes ↑4.3812.8812.88+8.50 [5.25, 11.63]0.00 [−0.50, +0.50]

RCPT improves error and target attainment and matches VCG's mean number of distinct successes (Table 4). Its mean amplitude is 0.733, and the deployed coefficient vector is zero on only 10.24% of logged steps across eight 1,000-step runs. This is a step-level correction statistic, not a particle acceptance rate. The nonzero vectors on the remaining steps show that the controller usually retained some fitted correction (Table L.18).

For frozen scaffold fit64, 76 pockets each use 64 fitting particles and 16 fresh outputs. Unique-useful yield is 27.1% for RCPT, 23.6% for Anchor, and 20.0% for VCG. Paired gains are +3.45 pp [0.66, 6.17] and +7.07 pp [3.04, 11.10], respectively (Table 3). Both yield gains are statistically resolved at the larger of the two main-table fitting budgets, with 64 particles for 12 coefficients. The +1.56 pp validity mean gain over Anchor remains unresolved [−2.22, 5.35]. Fit16 practical alternatives and safeguards are compared in Appendix L.4.

These molecular results answer why calibration can be useful beyond falling back to Anchor: it improves generation in both frozen deployment and online SMC. On QM9, full VCG already improves on Anchor, so the additional gain is not confined to rescuing a failed VCG baseline. The two deployment modes provide complementary evidence of practical value.

The original Checker counterexample remains valid. In Table I.4, Anchor has lower mean SMC W2W_2 at every common-fit budget. At N=32N=32, RCPT-minus-Anchor is +0.0817 [0.0051, 0.1629]. Table 2's favorable mean W1/W2W_1/W_2 comparisons come from a different deployment cohort. They do not overturn that result. Lower residual risk need not improve a corrected-SMC endpoint. Likewise, the small fit16 scaffold gains over Anchor have intervals spanning zero.

CIFAR answers the repeated fitted-controller comparison. Ten paired fitting-and-generation runs, each with 320 images per method, give RCPT better means on all seven endpoints than VCG, Direct LOO, Binary LOO, and selected ridge sharing its fixed-guidance base. Against ridge, KID and FID differences are −0.0240 [−0.0409, −0.0077] and −0.4608 [−0.7096, −0.2369]. Anchor (α=0\alpha=0) is stronger on six endpoints. RCPT's independent accuracy difference is −1.59 pp [−2.03, −1.19]. Mean FID is 78.5015 for Anchor and 78.5020 for RCPT, with an unresolved paired difference. Your concern that the base proposal can suffice applies here. Original DLSM is reported separately. At each of the four fixed ridge strengths, the reported means favor RCPT on at least six of seven endpoints (Table 5 and Appendix L.5).

The complete Anchor table uses ten paired fits for Checker risk, ten paired runs for Checker generation, 76 paired scaffold pockets, eight paired runs per QM9 protocol, and ten paired CIFAR runs. It includes every main-table outcome, both scaffold budgets, and separate image contrasts against DLSM. Favorable means remain distinguished from intervals excluding zero.

AuthorsOpenReview ↗

[2/2] Response to Reviewer abQu

2. Binary selection, fractional retention, and the accept/reject analogy (Q2, W2)

The requested Binary LOO is a strong comparator, and fractional retention helps in specific regimes. Binary LOO retains full VCG only when deleted prediction loss is lower than the zero-correction base's, choosing that base on ties. It deploys the full-data coefficient and shares the fitting information and safeguards with the other selectors (Appendix E.1).

At common-fit Checker N=16N=16, RCPT has Risk/Anchor 0.4931 versus Binary LOO's 0.7837, with difference −0.2906 [−0.8005, −0.0263]. Proposal-Only W2W_2 improves by −0.0277 [−0.0516, −0.0070]. At N=32N=32, binary instead has better SMC W2W_2: RCPT-minus-binary is +0.0406 [0.0031, 0.0704] (Table L.4).

That comparison changes both criterion and continuity. We therefore cross them on identical fitted directions and safeguards. The entries give mean Risk/Anchor at N=16N=16 (lower is better).

CriterionBinary amplitudeContinuous amplitude
Deleted prediction lossBinary LOO, 0.784Direct LOO, 1.126
Structured quadraticStructured binary, 0.493RCPT, 0.493

Structured binary accepts iff S>2P^+10−12S>2\widehat P+10^{-12}. At common-fit N=16N=16, structured binary and RCPT have almost equal risk: difference −0.00016 [−0.0373, 0.0618]. Here the criterion accounts for most of the gain. Continuity adds a resolved benefit at N=8,32N=8,32: differences −0.0834 [−0.1252, −0.0462] and −0.0418 [−0.0753, −0.0034]. It also lowers risk in all eight saved-online budget/driver conditions, each with a favorable nominal interval (Tables L.16–L.17). Those online comparisons evaluate counterfactual decisions on saved fits, not separately generated endpoint cohorts.

On QM9 N=36N=36, RCPT has favorable means over structured binary: −0.00394 MAE and +1.05 success points, with intervals [−0.01270, 0.00526] and [−0.59, 2.60]. Both yield 12.875 distinct successes. Endpoint superiority over structured binary is unresolved. All four budget-wise MAE contrasts against ordinary Binary LOO also span zero. At N=18N=18, binary yields 1.875 more distinct hits [0.125, 4.000]. Across 76 scaffold pockets, RCPT-minus-binary unique-useful yield is +0.0041 [−0.0238, 0.0321]. CIFAR KID favors RCPT over binary by −0.0237 [−0.0426, −0.0075] (Appendices L.2–L.5).

Operationally, RCPT changes a shared drift, rather than accepting or rejecting individual particle moves. All particles propagate under that drift. FK weighting and resampling remain separate. Zero amplitude retains the fitting base, which includes fixed guidance in CIFAR. Mean amplitude is therefore not an acceptance probability, and its computation adds no model calls. Across QM9 budgets 18,36,72,14418,36,72,144, it rises from 0.318 to 0.733, 0.879, and 0.945. At N=144N=144, both fitted methods preserve all 144 lineages. At N=18N=18, two RCPT runs lose ancestry.

Under the stated B=0B=0, H,P>0H,P>0 assumptions, the expected-risk oracle α0=H/(H+P)\alpha_0=H/(H+P) beats Anchor by H2/(H+P)H^2/(H+P) and full VCG by P2/(H+P)P^2/(H+P). This explains why an interior choice can help. The empirical comparisons determine when the implemented selector does so (§4.2). The factorial therefore clarifies what calibration contributes: the structured criterion matters at N=16N=16, and continuous retention adds value in other measured regimes. A strong binary alternative makes that attribution more informative without removing the underlying failure or the benefit of controlling the fitted update.

3. Sampling preliminaries and reading order (Q3, W1)

Section 3 now introduces the sampling loop before the risk analysis. It defines particles as candidate samples, importance weights, and propagate–reweight–resample. Immediately after Eq. (2), gtv(Xt)g_t^v(X_t) is defined as the rate of growth of unnormalized log-weight. Weight ESS measures weight concentration, and resampling can reset weights while losing ancestors.

The section then locates each method: the base proposal supplies the drift, VCG fits an additive correction, and RCPT chooses its retained fraction. Proposal-Only deployment omits weighting and resampling. Section 5 defines deletion LOO and the fitted ray. Appendix H.4 distinguishes common-state evaluation from diagnostics on each sampler's trajectory. The CIFAR fitting base and original DLSM reference are now explicitly distinguished.

The reading order is the finite-budget observation (§1), sampling preliminaries (§3), H/P explanation (§4), selection rule and guarantees (§5), and generation evidence (§6). Figure 1 describes its measured quantity as “Lower out-of-fit residual risk.”

The revised evidence shows useful partial correction improving generation beyond both Anchor and full VCG, with no additional model calls. The matched binary comparisons separate the roles of criterion and continuity, while the sampling preliminaries make the failure, its explanation, and the practical response easier to assess.

Thanks for the revision from the authors. The revised manuscipt has a clearer introduction about the background and the problem formulation. The newly provided experiments addressed most of my concerns. I will increase my score.

Thank you for taking the time to reassess the paper. We are glad that the revised manuscript is clearer and that the additional experiments addressed most of your concerns. Your questions and feedback helped us explain when RCPT improves on both Anchor and VCG.

Reviewer_YtjgOpenReview ↗
Rating
3
Confidence
2
Informativeness
3
Presentation
2
Soundness
2
Read review
Paper summary

The paper tackles the general setting of inference-time diffusion steering. Diffusion steering produces proposal mismatch, which DriftLite’s variance-controlling guidance (VCG) tries to minimize this mismatch by regression. However, finite-particle regression can overfit. The proposed method, Risk-Calibrated Proposal Transport (RCPT), estimates how much to shrink the correction throughout denoising so that the inference-time steering preserves useful guidance while avoiding harmful updates. The core operation is scalar shrinkage of a fitted coefficient vector, with a penalty estimate motivated by regression theory. The meaningful difference between RCPT and VCG is that VCG estimates first-order regression quantities, whereas RCPT attempts to estimate the estimation risk of the fitted coefficients. The latter is generally a harder statistical problem.

Critical strengths
  • The abstract goes into great details at times, often unnecessarily, somewhat defeating its purpose.
  • Figure 1: useful and clear
  • Theory seems OK (the logical chain of arguments is good, easy to follow, and makes sense) although I am not too comfortable with this sub-domain of machine learning.
  • The paper offers a coherent methodological contribution.
  • The authors are transparent about the limitations of the method (both on theory and empirical fronts); they consistently acknowledge exceptions, where RCPT is not always the best approach. E.g. in Appendix I.12, fixed shrinkage with α=0.5\alpha=0.5 achieves a unique useful yield of 0.406 versus 0.306 for RCPT on a ten-pocket scaffold subset. In Appendix I.13, a particular ridge configuration achieves slightly better molecular property metrics than RCPT, though only four seeds are used. And in Appendix I.4, there are Checker settings where RCPT improves reference residual risk but has worse corrected-SMC terminal W2W_2 than Anchor or selected ridge.
  • I appreciate the use of short and to-the-point sentences.
  • VCG fits the lambda coefficients; RCPT additionally computes NN leave-one-out regression residuals and a scalar amplitude. Crucially, the leave-one-out fits, penalty estimation, and shrinkage factor assembly do not require reevaluating the expensive diffusion networks. The extra overhead compared to VCG is therefore minimal. The compute costs are dominated by the evaluation of the neural networks for each particle at each diffusion step.
Critical weaknesses
  • CIFAR: The results are favorable to RCPT, but the improvements over Anchor are very small, and no between-fit statistical significance can be established from this experiment.
  • The experimental evidence is mixed in strength. There is substantial evidence for protection against bad VCG fits, but less compelling evidence for large, reliable improvements over the original Anchor or competitive regularized alternatives.
  • (Problem with P) The regression approximation is asymptotic. The result in Appendix D.6 (leading to the approximation of P) assumes independent sampling, fixed population rank, regular moments, stable covariance inversion, and effectively vanishing ridge regularization. None of these automatically guarantees a good approximation when N=16N=16 and m=12m=12. Applying a first-order asymptotic formula in a near-interpolation regime is problematic. The authors recognize this limitation explicitly.
  • (Problem with NeffN_{\mathrm{eff}} in P^\hat{P}) If these particles carry highly correlated errors, the estimator's sampling variance does not decrease as though they were independent. This is a genuine concern, because RCPT puts NeffN_{\mathrm{eff}} in the denominator of its estimated penalty. It may underestimate uncertainty precisely when genealogical collapse makes the samples highly dependent. Again, the authors acknowledge this in Appendix H.4 and D.6. They explicitly say that conventional weight ESS does not generally estimate the covariance relevant to the regression estimator.
Questions for authors
  • Anchor is so close behind RCPT, and VCG is far behind? Why does VCG even fail at this; isn’t it introduce exactly to be better than Anchor on the task?
  • The results convincingly demonstrate RCPT's ability to prevent harmful VCG updates. However, particularly in the scaffold and CIFAR-10 experiments, the improvements over Anchor appear relatively modest, and the selected shrinkage factors are often small. To what extent does RCPT retain useful corrections that meaningfully improve upon Anchor, rather than predominantly reverting toward Anchor to avoid harmful updates?
  • The authors suggest stable grouped shrinkage rule as future work in Section 6; the alpha shrinkage rates would then effectively be inside the sum over particulates. Do the authors foresee any extra hurdle with this objective?
  • It seems that the theoretical results do not establish that the particular estimator used by RCPT is an accurate or optimal shrinkage estimator in the small-particle regime. Can the authors comment on that, or correct my misunderstanding if there is one?
  • Is the experimental regime artificially unfavorable to VCG? Many experiments fit a relatively high-dimensional correction using extremely few particles, often 12 coefficients from 16 particles. That's precisely where severe regression overfitting is expected.
  • The authors explicitly describe their structured ingredients as not quantitatively calibrated pointwise. And yet, RCPT still achieves lower average shrinkage-decision regret than direct LOO in their Checker experiment. It demonstrates that accurate estimation of the various quantities is not strictly necessary for an effective shrinkage decision. Biased approximations can sometimes still give better decisions because they have lower variability (Appendix D.5). A question to the authors then is: why does this particular biased, structured approximation give better shrinkage decisions than the alternatives, and how reliably does that advantage generalize?
4 comments
AuthorsOpenReview ↗

[1/3] Response to Reviewer Ytjg

Thank you for recognizing the failure-mode figure, the logical theory progression, the coherent method, and the low overhead. VCG’s population optimum cannot worsen residual variance, but we show that finite-particle estimation error can outweigh its benefit, sometimes substantially. RCPT controls how much of that fitted correction is retained, without additional model calls. New QM9 and scaffold results show gains over both Anchor and VCG: QM9 shows RCPT improving on a VCG baseline that already beats Anchor, while scaffold shows improved yields even at a larger fitting budget. Related concerns are grouped below. References follow the revision and intervals are nominal paired 95% intervals over whole fits, runs, or pockets.

1. CIFAR replication and the meaning of its comparisons (W1)

The image comparison now uses ten independent paired fitting-and-generation runs, rather than one fit. Each run generates 320 images per method. Intervals resample whole paired runs, capturing between-fit and generation variability.

RCPT has better means on all seven endpoints than VCG, Direct LOO, Binary LOO, and LOO-selected ridge sharing its fixed-guidance base (Table 5). Independent accuracy is 60.91%, versus VCG's 55.66% and Direct LOO's 59.69%. Against Direct LOO, the accuracy difference is +1.22 pp [0.09, 2.44]. Four of seven contrasts have favorable intervals. KID favors RCPT over all four fitted alternatives. Against selected ridge, KID and FID differences are −0.0240 [−0.0409, −0.0077] and −0.4608 [−0.7096, −0.2369].

Anchor (α=0\alpha=0) has 62.50% independent accuracy, versus 60.91% for RCPT. The paired difference is −1.59 pp [−2.03, −1.19]. Anchor is better on six endpoints and the FID difference is unresolved (+0.0005 [−0.2874, 0.2917]). The original DLSM configuration is a separate baseline with 53.50% independent accuracy. Thus the repeated runs confirm the benefit of retaining less of a fitted update on CIFAR while also supporting use of Anchor itself. At each of four additional fixed ridge strengths, the reported means favor RCPT on at least six of seven endpoints (Appendix L.5).

2. Useful correction beyond Anchor and competitive alternatives (W2, Q2)

Online QM9 N=36N=36 and frozen scaffold fit64 provide the generation evidence beyond both Anchor and full VCG. On QM9, eight paired runs give terminal-weighted MAE 0.7926 for RCPT versus 0.9403 for Anchor and 0.8238 for VCG. The paired differences are −0.1477 [−0.2827, −0.0038] and −0.0311 [−0.0476, −0.0138]. Terminal-weighted target success is 50.13% versus 29.18% and 45.60%, with gains +20.94 pp [6.00, 35.58] and +4.53 pp [2.25, 6.69]. Mean distinct successes tie VCG at 12.875 (Table 4).

Mean amplitude is 0.733 at N=36N=36. The deployment-log audit covers 24 runs at N=18,36,72N=18,36,72 and records no fallback or cap-boundary events (Table L.18). In the separate N=144N=144 endpoint comparison, RCPT retains amplitude 0.945, all 144 lineages, and 45.50 distinct successes, close to VCG's 45.25 and above Anchor's 0.75. Thus it can preserve useful adaptation rather than predominantly revert to Anchor. Across N=18,36,72,144N=18,36,72,144, mean amplitudes are 0.318, 0.733, 0.879, and 0.945. At N=18N=18, two RCPT runs lose ancestry and its VCG MAE contrast is unresolved. At N=72N=72, its Anchor MAE contrast is unresolved (Table L.1).

Frozen scaffold fit64 uses 64 fitting particles and 16 fresh outputs per pocket. Across 76 pockets, unique-useful yield is 27.1% for RCPT, 23.6% for Anchor, and 20.0% for VCG, with paired gains +3.45 pp [0.66, 6.17] and +7.07 pp [3.04, 11.10]. Both yield gains have paired intervals excluding zero. The validity gain over Anchor remains unresolved. Table 3 also includes ECG, and Appendix L.4 compares safeguard-matched alternatives.

The two molecular benchmarks are complementary: QM9 shows an interior correction improving on a VCG baseline that already beats Anchor, while scaffold demonstrates a gain in useful outputs after frozen fitting. The frozen gain occurs at the larger scaffold budget of 64 particles for 12 coefficients. Together with larger-budget retention, these results demonstrate repair, useful partial correction, and preservation of adaptation.

Strong simple alternatives remain. At QM9 N=144N=144, selected ridge has lower MAE. The ten-pocket scaffold fixed-shrinkage counterexample remains: α=0.5\alpha=0.5 yields 0.406 unique-useful output versus RCPT's 0.306. The separate four-seed QM9 ridge comparison also remains (Tables I.11–I.12). RCPT's favorable means over structured binary at N=36N=36 do not establish endpoint superiority. The differences are quantified in §8 below. These findings limit a universal ranking while preserving the demonstrated operating regimes.

Under B=0B=0, H,P>0H,P>0, the expected-risk oracle α0=H/(H+P)\alpha_0=H/(H+P) improves on Anchor by H2/(H+P)H^2/(H+P) and on full VCG by P2/(H+P)P^2/(H+P) (§4.2). That explains the possibility of useful partial correction. The endpoint comparisons test the implemented selector separately.

AuthorsOpenReview ↗

[2/3] Response to Reviewer Ytjg

3. Why population-optimal VCG can fail after fitting (Q1)

The population optimizer can choose zero correction, but its finite-sample estimate need not improve population risk. Theorem 1 gives the exact criterion:

R(λ^)−V(0)=P−H.\mathcal R(\widehat\lambda)-V(0)=P-H.

The full correction increases risk in expectation over fits when mean coefficient-estimation error exceeds the available headroom. At common-fit Checker N=16N=16, independent Risk/Anchor is 174.0 for VCG and 0.4931 for RCPT. At online QM9 N=144N=144, both methods instead retain useful guidance and all lineages. These illustrate different fitting-information regimes.

The contribution concerns the deployment consequence of estimation error: a nearly zero fitting residual can worsen the generative dynamics. Section 5.1 links the residual to unweighted transport error and local FK weight inflation, explaining why the regression failure matters to sampling.

4. Small-particle accuracy and the scope of the theory (W3, Q4)

Your understanding is correct: the theory does not establish that the implemented proxy is accurate or optimal at small particle counts. The exact identity, bias-aware expected-risk oracle, iid asymptotic approximation, and conditional decision bound are distinct statements.

Under Lemma D.1's independent-sampling and regularity conditions, the leading penalty is N−1tr⁡(ΩC†)N^{-1}\operatorname{tr}(\Omega C^\dagger). The structured rule approximates this with a deletion-residual scale and rank-over-information factor. The first-order expansion does not by itself justify accuracy at N=16,m=12N=16,m=12. Section 5 and Appendix D state that scope.

For a realized fitted correction ff, Theorem 3 uses q=Var⁡(f)q=\operatorname{Var}(f) and κ=Cov⁡(g0+f,f)\kappa=\operatorname{Cov}(g^0+f,f). If the empirical quadratic approximates them with errors eq,eκe_q,e_\kappa, selected-risk regret is at most 2eq+4eκ+εnum2e_q+4e_\kappa+\varepsilon_{\rm num}. The theorem does not assert those errors are small. Expected coefficient error PP and realized κ\kappa also differ, with Eκ=P+B\mathbb E\kappa=P+B.

These statements identify what a good decision requires. Empirical regret comparisons assess whether this implementation achieves it. They do not turn the conditional bound into an unconditional small-NN guarantee.

5. Dependence and weight ESS (W4)

The dependence concern is confirmed experimentally and remains a limitation. Holding nominal N=ESS=64N=\mathrm{ESS}=64 while reducing independent ancestors from 64 to 16 raises RCPT Risk/Anchor from 0.2639 to 16.4102. All 100 ensembles are harmful in the latter case, with mean amplitude 0.9903. Deleting one copy leaves relatives in the data, while equal weights preserve high ESS (Table L.11).

On actual RCPT-driven Checker histories, estimated-penalty/reference-error ratios are 0.1047, 0.2131, 0.4415, and 0.3149 at N=8,16,32,64N=8,16,32,64. At N=8N=8, RCPT reduces candidate risk from 510.9 to 6.042 but remains harmful on average. At N=64N=64, it reduces risk from 0.7093 to 0.3814, with no harmful integrated RCPT runs among ten. On VCG-driven histories, the ratios exceed one at N=8,16,32N=8,16,32 (Table L.12).

Appendix L.8 reports whole-run intervals and reference sensitivity, including QM9. The exact fixed-law identity permits dependent fitting data. The iid proxy motivation does not. High weight ESS cannot certify correction reliability, and observed online endpoint benefits do not establish uniform safety.

6. Is the fitting regime artificially unfavorable? (Q5)

Fitting 12 coefficients from 16 particles is vulnerable to overfitting. That regime alone would not establish broad utility. The revision includes reduced-rank controls and larger fitting budgets, including settings where full VCG works well.

The scaffold dictionary uses four expert fields with three bounded gates. QM9 uses gated reward-gradient families. These are expressive choices with a measurable information–cost tradeoff. Increasing scaffold fitting particles from 16 to 64 raises mean peak memory from 4.740 to 18.611 GB and time from 197.45 to 419.19 seconds per pocket (Table K.1).

In scaffold, median VCG Risk/Anchor falls from 2.1146 at N=16N=16 to 0.2215 at N=64N=64, where VCG has lower risk than RCPT in 73/76 pockets. RCPT's lower mean at that budget reflects protection against a severe tail. Table B.2 includes four-field VCG and the 16/32/64 endpoint sweep. The fit64 unique-useful gain extends beyond the smallest-budget setting. QM9's near-full retention at N=144N=144 supplies another informative regime. Thus the evidence includes a statistically resolved frozen-deployment benefit beyond the smallest fitting budget and an online regime where VCG is itself useful. The case for inexpensive calibration rests on these practical outcomes as well as the controlled failure diagnosis.

AuthorsOpenReview ↗

[3/3] Response to Reviewer Ytjg

7. Grouped amplitudes and particle sums (Q3)

The extension groups basis fields, and its amplitudes remain shared across particles at a given step. Write

v=v0+∑gαgug,ug=∑j∈Ggλ^jϕj.v=v^0+\sum_g\alpha_g u_g,\qquad u_g=\sum_{j\in G_g}\widehat\lambda_j\phi_j.

The risk quadratic contains cross-group terms αgαhCov⁡(Tpug,Tpuh)\alpha_g\alpha_h\operatorname{Cov}(\mathcal T_pu_g,\mathcal T_pu_h). Estimating these covariances and jointly selecting several amplitudes requires more information than selecting one scalar. Applying the scalar rule independently to each group ignores those interactions.

State-dependent amplitudes are a separate extension: Tp(αgug)\mathcal T_p(\alpha_gu_g) includes the product-rule term ∇αg⋅ug\nabla\alpha_g\cdot u_g. Appendix L.7 states the distinction, and Appendix F supplies the directional oracle and a finite-sample counterexample. These are hurdles to richer control, not a grouped method claimed to be validated here.

8. Why the structured approximation helps, and where it does not (Q6)

The measured explanation is smaller curvature-weighted decision error at Checker N=16N=16, not universally more accurate component estimates. On the same fitted directions, regret decomposes as follows (Table L.10):

ComponentDirect LOORCPT
Systematic amplitude error15.91880.4111
Dispersion11.54294.0157
Boundary0.49940.0736
Total27.96114.5003

The total difference is −23.4607 [−50.5370, −2.6957]. This is regret relative to fit-specific ray oracles, rather than classical bias–variance around one fixed amplitude. Jointly rescaling a quadratic by a positive constant leaves its minimizer unchanged, so component-scale errors can coexist with a good decision. The shared residual scale and complexity penalty can regularize noisy deletion loss. This audit does not establish a universal causal mechanism.

The relative risk and decision-regret advantages over Direct LOO and ridge depend on the fitting regime. At N=8N=8, dispersion falls but systematic error rises, and total-regret contrasts at N=8,32N=8,32 span zero. Additional Checker fits favor RCPT over Binary LOO, Direct LOO, and selected ridge at N=8,16N=8,16. At N=64N=64, Direct LOO and ridge have lower risk (Tables L.9, L.15). Scaffold common-state risk often favors Direct LOO. In the frozen Checker fitting-budget sweep, no tested budget shows a resolved corrected-SMC W2W_2 gain over both Anchor and VCG (Table L.19). This limits the terminal inference one can draw from the reference-risk gains.

The matched criterion-by-continuity factorial separates two effects. At N=16N=16, structured binary and RCPT have almost equal risk (difference −0.00016 [−0.0373, 0.0618]). At N=8,32N=8,32, continuity improves by −0.0834 [−0.1252, −0.0462] and −0.0418 [−0.0753, −0.0034]. All eight saved-online budget/driver comparisons favor continuous structured scaling over structured binary at nominal 95% (Tables L.16–L.17). These are counterfactual decisions on common saved fits.

On QM9 N=36N=36, RCPT's MAE is 0.79264 versus structured binary's 0.79659. Success is 50.13% versus 49.08%. Differences −0.00394 [−0.01270, 0.00526] and +1.05 pp [−0.59, 2.60] are unresolved, and distinct successes tie at 12.875. The evidence supports a useful structured decision rule in identified regimes, with strong alternatives retained in the comparison.

The theoretical and empirical evidence serve complementary roles: the analysis explains why a population guarantee can fail after fitting and quantifies the loss from the selected correction. The experiments show when reducing or retaining that correction improves useful generation with negligible added model cost. This supports the value of the decision framework and its concrete implementation without requiring uniformly accurate penalty estimates or a universal ranking over alternative selectors.

AuthorsOpenReview ↗

Gentle Follow-up on Our Response

Dear Reviewer Ytjg, thank you again for your thoughtful and constructive review. We would appreciate hearing whether our responses and the revised manuscript address the concerns you raised. If any points need further clarification, we would be happy to address them. Thank you for your time and consideration.

Comments on the paper

1 comment
AuthorsOpenReview ↗

Common response to the reviewers and Area Chair

We thank the reviewers for recognizing the finite-particle failure and for asking what calibration adds beyond Anchor and simpler controls. A correction that reduces risk at the population optimum can increase it when fitted on a finite particle set. We show how coefficient-estimation error can outweigh the available reduction in residual risk, and how that residual affects transport and local FK weight inflation. RCPT uses this analysis to choose how much of the fitted correction to retain, with no additional model calls. The molecular results show why this choice matters in generation. Revisions are marked in blue in the manuscript, including the new Appendix L.

Useful correction beyond both Anchor and VCG. On online QM9 at N=36N=36, eight paired runs give SMC-weighted MAE 0.7926 for RCPT versus 0.9403 for Anchor and 0.8238 for VCG. Weighted target success is 50.13% versus 29.18% and 45.60%. Both MAE and success improvements over each comparator have paired 95% intervals excluding zero (Table 4). RCPT and VCG tie at 12.88 mean distinct successes. VCG already improves on Anchor here, and retaining a fraction of its correction improves both weighted outcomes further.

Frozen scaffold fit64 raises unique-useful yield to 27.1%, versus Anchor's 23.6% and VCG's 20.0%. Across 76 pockets, paired gains are +3.45 percentage points [0.66, 6.17] and +7.07 [3.04, 11.10] (Table 3). This gain persists at the larger scaffold budget of 64 fitting particles for 12 coefficients. Together, the molecular results establish statistically resolved endpoint benefits in both frozen and online deployment. At larger QM9 budgets, near-full retention also shows that calibration need not discard useful guidance.

When is the unmodified proposal preferable? On CIFAR, Anchor is the shared proposal at α=0\alpha=0. Across ten paired runs it has better results than RCPT on six endpoints and nearly identical mean FID (78.5015 vs 78.5020). Original DLSM is a separate baseline. Among methods that deploy a fitted correction, RCPT improves all seven means over VCG, Direct LOO, Binary LOO and LOO-selected ridge. Against selected ridge, KID improves by 0.0240 [0.0077, 0.0409] and FID by 0.4608 [0.2369, 0.7096]. RCPT also has better means on at least six of seven endpoints at each of the four fixed ridge strengths (Table L.7). CIFAR shows the cost of deploying a noisy fit and how much calibration can recover. The molecular comparisons above show when retaining a fitted correction improves on Anchor itself. The CIFAR results use Eq. (H.4) with the base-score/classifier-gradient cross term corrected after submission.

Theory, reliability, and readability. The exact H/P identity explains the failure, the expected-risk oracle explains the opportunity for partial correction, and the conditional decision bound specifies what accurate selection requires. Dependence audits delimit the implemented proxy. They do not establish online safety. Section 3 introduces the sampling loop before the analysis, and Table L.14 reports every main-table Anchor contrast. Reported intervals are pointwise nominal 95% intervals over whole runs, fits, or pockets.

From OpenReview, CC BY 4.0. Authors as signed there; shown unedited.

Related papers

Loading the map…

Discussion (0)

Sign in to comment.