Inter-Trajectory Importance Sampling: A Principled Mitigation of Mode Collapse

Sampling from unnormalized distributions is a fundamental challenge in computational science. Diffusion samplers typically approach it by minimizing the reverse Kullback–Leibler divergence between path measures but remain susceptible to mode collapse. Conventional single-trajectory estimators evaluate trajectories independently, without directly assessing sample diversity. We introduce Inter-Trajectory Importance Sampling (ITIS), a multi-sample estimator that couples trajectory losses through path-measure importance weighting. ITIS remains unbiased, preserves the expected gradient and critical points, requires no additional forward passes or heuristic inter-particle forces, and yields i.i.d. samples at inference. At the finite-batch level, it assigns lower realized losses to diverse trajectory configurations than to collapsed ones. We characterize theoretically how this method mitigates mode collapse and we demonstrate its efficacy experimentally. As a drop-in loss replacement, ITIS improves established diffusion samplers on standard benchmarks and interacting-particle systems.