- Rating
- 2
- Confidence
- 3
- Informativeness
- 2
- Presentation
- 3
- Soundness
- 2
Read review
Paper summary
This paper proposes Inter-Trajectory Importance Sampling (ITIS), a multi-sample loss estimator designed to mitigate mode collapse in diffusion-based samplers for unnormalized target distributions. Rather than modifying the underlying reverse KL objective or introducing explicit particle repulsion, ITIS couples the loss contributions of independently generated trajectories through importance weights constructed from their transition kernels. The authors show that the resulting estimator preserves the expected objective and its gradient under appropriate conditions, while typically assigning smaller realized losses to batches with separated transition kernels.
Critical strengths
ITIS provides a flexible training modification that can be integrated into existing diffusion samplers without requiring additional sampling trajectories or introducing interactions at inference time. Its effectiveness is supported by improvements across DDS, DIS, and PIS on several benchmarks, suggesting practical applicability beyond a single sampling architecture.
Critical weaknesses
- Although ITIS employs analytically tractable importance weights, their variance can grow exponentially with the squared standardized separation between Gaussian transition kernels, potentially causing severe instability in high-dimensional settings or at small diffusion noise levels. Importantly, this is precisely the regime in which Proposition 1 establishes that the realized loss ratio approaches (1/K) in probability, meaning that the claimed preference for diverse trajectories is accompanied by increasingly rare but potentially enormous importance weights and gradient contributions. Since ITIS preserves the expected gradient of the original reverse KL objective, the manuscript does not sufficiently explain why these high-variance stochastic updates should systematically improve mode coverage rather than merely alter the optimizer's behavior. Moreover, the paper overlooks [1], which addresses a closely related problem through importance weighting and faces analogous concerns about importance-sampling variance and high-dimensional scalability, despite employing a different estimator. Without a more careful analysis of gradient variance, optimization stability, and the relationship to this prior work, the theoretical justification for the claimed principled mitigation of mode collapse remains insufficient.
- A potentially important issue is that the finite-batch ITIS gradient is not invariant to arbitrary multiplicative rescaling of the unnormalized target density. Specifically, replacing with , which represents exactly the same target distribution, shifts the terminal cost by but changes the ITIS stochastic gradient by , where is generally not constant for a realized batch. Although this additional term has zero expectation under the assumptions of Proposition 1, it can alter the stochastic-gradient variance and the resulting optimization dynamics, potentially affecting the claimed mode-coverage mechanism. This is particularly concerning because the method targets distributions with intractable normalization constants, for which the energy function is defined only up to an arbitrary additive constant. The authors should clarify how this invariance is handled in their implementation and demonstrate that the reported improvements are robust to such rescaling.
[1] Wang C, Zhang X, Cui K, et al. Importance weighted score matching for diffusion samplers with enhanced mode coverage[J]. arXiv preprint arXiv:2505.19431, 2025.
Questions for authors
- For Gaussian transition kernels with equal covariance , the importance-weight variance satisfies . How does the proposed method control this exponential variance growth, particularly in high-dimensional settings or at diffusion steps with small noise levels?
- Proposition 1 establishes that the realized loss ratio approaches as the transition kernels become increasingly separated, while the corresponding importance-weight variance diverges exponentially. How can the authors justify interpreting this typical loss reduction as a beneficial mode-coverage mechanism when it is accompanied by increasingly heavy-tailed stochastic estimates?
- Since ITIS and the conventional estimator have identical expected gradients, their different optimization outcomes must arise from differences in stochastic-gradient distributions and their interaction with the optimizer. Can the authors characterize the gradient variance or provide empirical evidence showing that the additional stochasticity systematically facilitates mode discovery rather than destabilizing training?
- The terminal cost is defined as , and the proposed ITIS estimator multiplies this cost by the inter-trajectory importance weights. Since the target density is only available up to an unknown normalization constant, does the implementation replace with its unnormalized counterpart ? If so, how is the omitted term handled after importance weighting?
- Consider replacing with , which leaves the normalized target distribution unchanged. The terminal ITIS loss then changes by , where , implying a change of in the stochastic gradient. Can the authors explain whether this dependence is present in their implementation and, if not, identify the mechanism that eliminates it?
- Could the authors provide empirical evidence that ITIS is robust to additive shifts of the target energy function, which leave the underlying target distribution unchanged? In particular, do such shifts affect the stochastic-gradient behavior, training stability, or reported mode-coverage improvements relative to the conventional single-trajectory estimator?
2 comments
Q3. We characterize the gradient variance in App. C.8.5 for the runs of Fig. 1. Here g is the raw gradient and u the applied Adam update divided by the learning rate. SNR is the norm of the mean divided by the square root of the total variance over 1000–2000 batches at a checkpoint, with Adam's state held fixed for u. Each SNR is the median over 16 checkpoints per seed, reported as mean ± SE over 3 seeds; norms are medians over all batches, checkpoints and seeds.
| Training loss | median ‖g‖ | median ‖u‖ | SNR of g | SNR of u | |
|---|---|---|---|---|---|
| d = 2 | L_Single, K = 1 | 10.8 | 55.1 | 0.144 ± 0.015 | 2.238 ± 0.015 |
| L_Multi, K = 400 | 30.3 | 35.7 | 0.040 ± 0.004 | 1.47 ± 0.14 | |
| d = 50 | L_Single, K = 1 | 6,560 | 182 | 5.33 ± 0.34 | 13.50 ± 0.29 |
| L_Multi, K = 400 | 37.7 | 49.2 | 0.0342 ± 0.0008 | 2.19 ± 0.24 |
The ITIS gradient has a lower SNR in both dimensions. The optimizer substantially narrows this gap in the applied update, although it remains larger in d = 50. Random reweighting does not produce the gain: shuffling the weights among the trajectories of a chunk, without gradient through them, keeps their distribution but removes their link to the trajectories' positions, and the coverage gain is lost (Table 10). The gain requires the gradient through the weights, whose direction depends on where the trajectories lie relative to each other (Table 10).
W2/Q4–Q6 Energy Offset. We thank the reviewer for this precise observation, which is correct. We studied it in detail and answer these points jointly. The full study is in the new App. C.8.8. Q4/Q5. Yes, we use the energy functions exactly as provided by the benchmarks, without any added constant, so the term is present in our implementation.
is the total importance weight at the final step, averaged over a chunk. It lies between , when the final transition kernels of the trajectories do not overlap, and , when they all overlap. The terminal ITIS term therefore contains the mean terminal cost as a coefficient on . If this coefficient is positive, minimizing the loss pushes the final transition kernels apart, and the offset shifts the coefficient by . As the reviewer notes, the term has zero expectation. In practice, however, typical batches have well below one, and the rare batches with very large weights that restore the zero mean are relatively suppressed by the optimizer (Adam). The standard remedy from RL, subtracting a running mean of the terminal cost, makes ITIS exactly invariant. However, it also sets the coefficient on to zero and thereby removes the associated push-apart gradient term, and ITIS then no longer learns with the original hyperparameters of Fig. 1 at K=400 (row below).
We therefore let each terminal cost enter the ITIS loss as . Here is a moving average of the mean terminal cost of previous batches, without gradient, and is a fixed margin. An offset shifts every and by the same amount, so the entire training run is independent of up to floating-point rounding. Since depends only on previous batches and is a constant, the expected gradient of Proposition 1 is unchanged. The margin sets the coefficient on explicitly. b+κ thus replaces the implicit dependence on the unknown normalization constant with an explicit hyperparameter that does not depend on the offset.
Q6. We test this on 2D GMM40 () with , batch size 2000 and the hyperparameters of Fig. 1, kept fixed without any retuning. Sinkhorn distance, mean over 3 seeds.
| a = −5 | a = 0 | a = +5 | |
|---|---|---|---|
| plain ITIS | 62.5 | 116.4 | 295.2 |
| b+κ, κ = 0 (plain centring) | 370.8 | 370.8 | 370.7 |
| b+κ, κ = 1 | 369.2 | 370.0 | 369.1 |
| b+κ, κ = 5 | 259.1 | 261.9 | 262.6 |
| b+κ, κ = 10 | 123.8 | 119.1 | 141.1 |
| b+κ, κ = 15 | 90.1 | 90.4 | 87.0 |
| b+κ, κ = 20 | 61.0 | 74.4 | 65.5 |
Fig. 5 in App. C.8.8 shows that plain ITIS is sensitive to the offset, while b+κ, with given by a running average, removes this dependence for every . The parameter is an additional, optional hyperparameter. We add a detailed discussion of this study to App. C.8.8.
Thank you for your detailed feedback and thoughtful questions. Please find our response below
W1:
We agree that ITIS is a higher-variance estimator of the same expected loss as the single-sample estimator. However, equality of expected losses does not imply equivalent optimization dynamics. The relevant question is how the structure of the estimator’s noise interacts with the optimizer. The analogy with stochastic versus full-batch gradient descent illustrates this distinction: stochasticity frequently is the essential ingredient in non-convex optimization. We use the standard DDS optimizer, unchanged from the original implementation (Vargas et al., 2023) and the benchmark of Blessing et al. (2024), and identical for the single-sample baseline. In our response to Q3 below we show that ITIS indeed has lower SNR in the raw gradient but that this variance is significantly reduced by the standard optimizers. For ITIS, the key feature is the skewed importance-weight distribution: typical samples receive small weights, whereas rare samples receive large weights. A typical sample lies on its own kernel’s side of the other kernel’s mean, so its weight decreases as the kernels separate. Consequently, typical batches yield r:=LMulti/LSingle<1, with gradient contributions that favor kernel separation and thus diversity. Equation 12 and the subsequent discussion identify the two contributions responsible for this behavior: the second term in the rhs of Eq. 12 is the gradient through the weights which pushes the trajectory into regions of diversity. The first terms of Eq. 12 is for typical batches just a scaled version of the single loss gradient but scaled down in weight space regions of high diversity. Table 10 tests these terms separately. Removing the gradient through the weights, so that each trajectory's cost is only reweighted by its own weight, loses the coverage gain. Shuffling these constant weights among the trajectories of a chunk, which keeps their distribution but removes their link to the trajectories' positions, performs worse still. Training with the gradient through the weights alone, without the reweighted single-sample term, diverges. The gain therefore requires the inter-trajectory term of Eq. (12) together with the reweighted single-sample gradient, and the reweighting alone, with own or permuted weights, does not produce the coverage gain.
IWSM/[1] is now discussed in Sec. 2.
Q1. We agree with the reviewer's expression for the weight variance. ITIS does not bound the variance of the weights themselves. What matters for training is how far a single batch can move the parameters, and three properties of the setup limit this. First, the weights are ratios of one-step transition kernels, not of path densities, so their variance does not accumulate over the diffusion steps. Second, we train with the standard DDS optimizer, unchanged from the original implementation which is based on Adam. App. C.8.5 shows that the updates exhibit a significantly higher SNR than the plain gradients. Third, small noise levels are not forced on the sampler, because the noise scale is tuned separately for each estimator; the advantage of ITIS persists when the number of diffusion steps is increased (App. C.8.7, Fig. 4). Explicit control of the weights is not needed in our experiments: weight clipping did not improve results on most benchmarks and the unclipped run performs best on GMM40 50D (Table 13). Regarding high dimensions, ITIS improves over the single-sample loss on GMM40 in d = 50 (Fig. 1) and d = 100 (App. C.8.3, Table 11), and on LJ-55 with d = 165 (Tab. 1).
Q2. We do not rely on the smaller loss value itself. A smaller loss alone would act like a smaller learning rate. Table 9 confirms that the gain is not a smaller effective step, since no scaling of the learning rate with the chunk size reproduces it. The gain instead requires the gradient through the importance weights. Without it, the coverage gain disappears (Table 10), as expected from our discussion of Eq. (12). That this mechanism yields genuine mode coverage is visible in the samples since ITIS places substantially more mass on the modes far from the prior, and far fewer modes remain underpopulated (Fig. 2, Fig. 3).