acceptodds
Under review as a conference paper at ICLR 2027

Three-Body Scattering for Generative Modeling

Abstract

An energy-distance field provides direct sample-level supervision for one-step generation, but sparse source interactions suffer from high noise. We propose Three-Body Scattering Modeling (TBSM), which learns a fake-to-real correction field via an online tracker that dynamically adapts to the evolving generated distribution. Under TBSM, each generated projectile is simultaneously attracted toward a real source and repelled from an independently generated source. At the exact energy-distance endpoint, the conditional expectation of this interaction coincides with the -Wasserstein gradient-flow velocity of . The online tracker approximates this expectation, while frozen-target regression transfers the estimated corrections into the generator, which is deployed standalone during inference. Theoretically, our analysis characterizes the regime where tracking provably reduces supervision error, establishing stationarity and convergence guarantees under explicit conditions. Computationally, each event conditions on a single real reference, requiring only sample interactions for a batch size of . Leveraging frozen image features, TBSM trains one-step generators on ImageNet-256, achieving an with pixel-space PixelDiT-XL and an with latent-space DiT-XL at . Finally, we provide a unified design map relating this learning paradigm to diffusion supervision, drift dynamics, and GAN-like objectives.

Primary area:
generative models
Keywords:
generative modeling

Reviews

Among:
Shape:
1234this paper, Bottom 10%
Figure 1. Mean review rating of the reviewed papers at ICLR 2027. This paper averages 1.67 over 3 reviews, above 10% of the others.
Reviewer_oJjjOpenReview ↗
Rating
2
Confidence
3
Informativeness
2
Presentation
2
Soundness
2
Read review
Paper summary

This paper proposes Three-Body Scattering Modeling (TBSM), a method for training one-step generative models through sample-level interactions. The method constructs a correction field by attracting generated samples toward real samples and repelling them from other generated samples. To reduce the noise in these interactions, the authors introduce an online tracker that learns a smoother correction field for generator training. The paper establishes a connection between the proposed interactions and the Wasserstein gradient flow of the energy distance. It also provides theoretical analyses of tracking errors and distributional identifiability under specific conditions. Experiments on ImageNet and text-to-image generation demonstrate competitive one-step generation performance across different model architectures.

Critical strengths
  1. Well-motivated method design. The paper identifies the high variance of sparse sample interactions as an important challenge in distribution-based generative modeling. The proposed online tracker provides a practical way to obtain more stable supervision while maintaining one-step inference. This design directly addresses the identified problem.
  2. Meaningful theoretical analysis. The paper establishes a connection between three-body interactions and the Wasserstein gradient flow of the energy distance. It also analyzes when tracking can reduce supervision errors and provides distributional identifiability results. These findings offer useful theoretical support for the proposed approach.
  3. Strong experimental performance. The method is evaluated on different generator architectures, including JiT, PixelDiT, and DiT. The results show competitive one-step generation quality on ImageNet, with additional experiments demonstrating applicability to text-to-image generation. The representation-space ablations further support the practical effectiveness of the method.
Critical weaknesses
  1. Limited theoretical justification for the practical training dynamics. The paper establishes a connection between three-body scattering and the Wasserstein gradient flow of the energy distance. However, the main gradient equivalence result in Theorem 1 applies to the exact energy-distance setting with λ=1\lambda=1. In practice, several of the reported experiments use λ<1\lambda<1, which changes the underlying distributional objective and may shift its optimum away from the target distribution. Furthermore, the practical method relies on learned trackers, corridor-based queries, and frozen feature representations. Although the paper provides separate theoretical analyses of tracking errors and full-corridor identifiability, these results do not directly establish the convergence or distributional consistency of the complete practical training procedure. This gap makes it difficult to determine how well the theoretical analysis explains the empirical improvements, particularly for the configurations that achieve the strongest results.
  2. Insufficient evaluation of the tracker under matched computational budgets. The online tracker is a central component of TBSM, and the ablation studies demonstrate clear improvements over instantaneous scattering supervision. However, the comparisons mainly match the number of generator updates rather than the total training cost. The tracked variants require additional network optimization and feature-space computations, which introduce extra computational overhead. More importantly, the paper does not sufficiently compare the learned tracker with simpler variance reduction strategies, such as averaging multiple independently sampled scattering vectors or using moving-average estimates. These alternatives could also reduce the noise in the supervision signal without requiring a separate learned correction network. A comparison under matched GPU time or computational budgets would help determine whether the improvements come specifically from learning the conditional correction field or from using additional training resources. This is important for assessing the practical benefits of the main methodological contribution.
Questions for authors
  1. The paper discusses the methodological differences between TBSM and Drift, particularly the use of sparse three-body interactions and an online tracker. However, the practical advantages of these differences remain unclear. In Table 2, Drift reports better FID than TBSM, although the results are obtained with different architectures and training settings. Could the authors provide a more direct comparison with Drift under matched generator architectures and computational budgets? In particular, it would be helpful to clarify whether TBSM offers advantages in training stability, convergence speed, computational efficiency, or generation quality, and which of these benefits can be attributed to the online tracker.
  2. Could the authors further clarify which theoretical properties remain valid when λ<1\lambda<1 and the tracker is trained using corridor-based queries? In particular, is it possible to characterize the stationary points or distributional bias of the resulting practical training dynamics? It would also be helpful to discuss how tracker approximation errors affect convergence when the generator distribution changes throughout training.
Reviewer_QKUZOpenReview ↗
Rating
2
Confidence
3
Informativeness
2
Presentation
3
Soundness
2
Read review
Paper summary

The paper proposes Three-Body Scattering Modeling (TBSM), that learns a correction field from interactions between a generated sample, a real sample, and another generated sample.The authors introduce an online tracker that estimates the average correction, which supervises the generator through frozen-target regression. Additionally, the paper connects the exact interaction to energy-distance descent. The authors show the effectiveness of the method by experiments on the tasks of one-step image generation.

Critical strengths
  • The paper is well-written with excellent illustrations and organization.
  • Figure 3 tests tracking at the energy-distance setting. FID improves from 10.31 to 4.71 under the same generator-update budget, showing that the learned correction can improve over individual noisy interactions.
  • The error decomposition separates source-sampling variance from tracker error. The tracker and feature encoders are used only during training, leaving generation to a single network evaluation.
Critical weaknesses
  • The main comparison does not match total training compute. The tracked method optimizes auxiliary networks, while the instant baseline omitted those updates. Because the interaction estimator itself is closely related to stochastic energy-distance estimation, the contribution depends on showing that tracking is preferable to spending the same budget on additional source samples or generator updates. The current comparison does not demonstrate this.
  • The benchmark table does not isolate an advantage over existing distribution-matching objectives. Most ImageNet runs start from pretrained diffusion or flow models, and the baselines differ in architecture and training procedure. Drifting and FD-loss report lower FIDs, although these are also not controlled comparisons. The results demonstrate competitive one-step conversion, but not really show the benefit of the proposed objective over alternatives unresolved.
Questions for authors
  • At matched training time, how does tracking compare with averaging more interaction samples or performing more instant-field updates?
  • With the same backbone and pretrained initialization, how does TBSM compare with an existing direct distribution-matching method?
Reviewer_cGNKOpenReview ↗
Rating
1
Confidence
3
Informativeness
2
Presentation
1
Soundness
1
Read review
Paper summary

The paper proposes Three-Body Scattering Modeling (TBSM), a method for training generative models to produce high-quality images in a single forward pass, rather than using the multiple sampling steps required by conventional diffusion models. The central idea is to treat generated samples as particles experiencing two competing interactions. Attraction, which causes a generated sample to be pulled toward a real training sample. Repulsion, which causes a sample to be pushed away from another independently generated sample. These interactions produce a correction vector that tells the generator how to change its output to better approximate the real data distribution. Because individual corrections are noisy, the authors introduce a second neural network, called a tracker, that learns to predict more stable corrections from previous training interactions. The generator is then trained toward these corrected targets, and the tracker is discarded at inference. The authors connect the method to energy distance, maximum mean discrepancy (MMD), and Wasserstein gradient flows. Experiments cover ImageNet generation and text-to-image generation, including a 20-billion-parameter pretrained model.

Critical strengths

The paper introduces an interesting approach to one-step generative modeling by combining stochastic three-body attraction, repulsion interactions with a learned tracker that estimates the expected correction field. The theoretical analysis establishes the relationship between the scattering updates and the energy-distance gradient field, while providing conditions under which tracking reduces estimation error. This provides a meaningful mathematical foundation for the proposed framework. Furthermore, the experimental results in Figure 3 support the effectiveness of tracking, with FID improving from 10.31 to 4.71 at the exact energy-distance endpoint 𝜆. The formulation is particularly appealing because it avoids requiring an adversarial discriminator (used in GANs) or diffusion-teacher supervision (used in Diffusion models) during TBSM training.

Critical weaknesses
  1. The contribution of the learned tracker is insufficiently isolated One of the central contributions of the paper is the introduction of a learned neural tracker to estimate the expected scattering correction and reduce the variance associated with instantaneous three-body interactions. The results in Figure 3 demonstrate substantial improvements when tracking is introduced, particularly at (\lambda=1), where FID improves from 10.31 to 4.71. While this is encouraging, the experiments do not sufficiently establish whether the improvements specifically arise from the proposed learned tracking mechanism or could be achieved through simpler variance-reduction approaches. For instance, averaging the scattering vectors obtained from multiple independently sampled source pairs would also reduce estimation variance without requiring an additional trainable network. A comparison against such approaches is important to justify the necessity and effectiveness of the tracker. Suggested experiment: Compare the proposed tracker against instantaneous scattering with varying numbers of sampled source pairs (e.g., 1, 2, 4, and 8), averaging the resulting correction vectors. Ideally, these comparisons should be performed under matched computational budgets.

  2. Computational efficiency is not sufficiently established through fair comparisons The paper emphasizes the computational efficiency of its formulation, particularly the (O(B)) source-interaction complexity and single-step inference. However, the proposed training framework introduces additional computational requirements through the learned tracker, multiple pretrained feature encoders, and their associated gradient computations. Although the experiments provide per-step timing information for different feature representations, the comparisons between instantaneous and tracked scattering primarily match the number of generator updates rather than total computational cost, as acknowledged by the authors. Consequently, it remains unclear whether the improvements obtained through tracking justify its additional computational overhead. Furthermore, the reported (O(B)) complexity concerns the number of sampled interactions and does not necessarily reflect the complete computational requirements of the training framework. Suggested experiment: Provide comparisons of total training time, GPU-hours, and peak memory consumption for tracked and instantaneous scattering. More importantly, compare the reconstruction/generation quality achieved under equivalent training-time or computational budgets, ideally using the same generator architecture, initialization, and feature representations. Reporting FID against training time or computational cost would be particularly informative for evaluating whether the proposed method offers a meaningful efficiency advantage.

  3. The effects of generated-sample repulsion and tracker-query interpolation are not independently evaluated An important aspect of the proposed formulation is the parameter (\lambda), which simultaneously controls the strength of the generated-source repulsion and the range of interpolation between generated and real samples used to query the tracker. Specifically, 𝜆 determines both the repulsive contribution in the scattering vector and the interpolation interval (𝝰 ~ U(0,1-𝜆)). Although Figure 3 and Table 2 investigate different values of 𝜆, changing this parameter modifies two distinct components of the algorithm simultaneously. Consequently, the experiments cannot determine whether the observed improvements arise from changes in generated-sample repulsion, tracker-query interpolation, or their interaction. This is particularly important because the paper provides different theoretical interpretations for the energy-distance endpoint and the full-corridor configuration. Suggested experiment: Introduce two independent parameters: one controlling the repulsion strength and another controlling the interpolation range used to query the tracker. A small factorial ablation comparing different combinations while keeping the tracker mixing coefficient 𝜌 fixed would allow the individual contributions of these mechanisms to be evaluated. I recognize that performing additional experiments at ImageNet scale or with large text-to-image models may be computationally expensive and time-consuming. Therefore, I believe smaller-scale experiments would be sufficient to address these concerns, provided they are carefully controlled and demonstrate the relevant behavior.

Questions for authors

Can the weaknesses be addressed?

From OpenReview, CC BY 4.0. Authors as signed there; shown unedited.

Related papers

Loading the map…

Discussion (0)

Sign in to comment.