acceptodds
Under review as a conference paper at ICLR 2027

Scaling Collider Event Generation with Residual-Quantized Tokens

Abstract

Full detector simulation and reconstruction of collider events are projected to become major bottlenecks at the High-Luminosity Large Hadron Collider, motivating the development of fast, ML-based surrogates. At the same time, LLMs have driven fast progress in generative discrete modeling: autoregressive transformers trained on tokenized data now represent the state of the art across a range of generative tasks. We extend the discrete modeling paradigm by introducing a particle-level generative model trained on residual-quantized full-event data. We demonstrate the ability of this model family to perform conditional generation from detector-stable particles; we study its scaling behavior across a range of dataset and model sizes, characterize the effects of repeated data exposure and demonstrate that token-level loss systematically predicts downstream physical fidelity. These results provide an empirical framework for scalable collider full-event generation based on residual-quantized representations.

Primary area:
applications to physical sciences (physics, chemistry, biology, etc.)
Keywords:
autoregressive modeling, scaling law, HEP, physics, detector simulation

Reviews

Among:
Shape:
1234this paper, Top 49%
Figure 1. Mean review rating of the reviewed papers at ICLR 2027. This paper averages 2.33 over 3 reviews, above 51% of the others.
Reviewer_gTJ1OpenReview ↗
Rating
1
Confidence
3
Informativeness
1
Presentation
2(+1)
Soundness
2
Read review
Paper summary

The paper applies residual vector quantization and autoregressive transformers to simulate collider event data. The authors argue that token prediction error improvements translate into improvements in event level and jet level residuals for accurate physics simulations. The authors also conduct experiments to suggest that token level scaling laws are a proxy for estimating physical performance.

Critical strengths

I think the idea of applying tricks from LLM architecture (RVQ, Autoregressive Transformers) to collider event generation is an interesting one. There is a need to simulate realistic collider event data and finding ways to leverage generative modeling is an interesting angle.

Critical weaknesses

This is an application paper, RVQ and autoregressive transformer models are standard and so there isn't any ML novelty. The connection between two pretty niche areas (LLM architecture and experimental particle physics) is unclear. Since this is the main contribution, I think the paper should spell out how applying these ideas lets practitioners do something new or useful. The writing is difficult to follow, and I'm not sure who the intended audience is for the paper. Even though it’s an application paper, the ideas from both ML and collider simulations should be self contained. I would suggest reviewing the existing generative models for collider event generation or analysis and adding a figure detailing the quantizer network.

Questions for authors

What exactly gets mapped to a fixed vocabulary, and how is that vocabulary defined? Section 2.1 says something about particles being defined by transverse momentum pT, pseudo-rapidity \eta, azimuthal angle \phi, its class and coordinates of its production vertex. These should all be defined for an ML audience. What is important about both top quark and anti-top quark pair production data vs QCD data? Are the inputs the same for all datasets? The paper says that the fidelity of the representation is limited by the codebook size K, how is K chosen and how is the number of residual quantization steps N optimized? How do tradeoffs between K and N impact downstream performance? There also seems to be no architectural novelty in the generative model itself, so comparing against other 3D point cloud generators is a class of important missing baselines.

3 comments

We thank the reviewer for the constructive criticism and the useful comments, in particular making it clear that some tools, features and metrics were not properly explained in the main text. We addressed this by explaining more in depth the motivation behind this work and we tried to explain more clearly what every feature or metric we referred to means.

Following, we address the main concerns and questions:

Methodology clarification

  • We added a simplified figure detailing the workings of the quantizer network, see Figure 2.
  • The latent vectors, outputs of the encoder network (separate for each particle), are mapped (by nearest-neighbor assignment) to the codebook, which is a lookup table assigning to each token in the vocabulary (basically a scalar ID from 0 to codebook_size-1) a latent vector. As a result, each particle mapped to a K⋅MK \cdot M tokens, resulting in the N⋅K⋅MN \cdot K \cdot M tokens for the whole event. Here, MM stands for the number of quantizers, KK is the codebook size, NN is the number of particles in the event.
  • We defined more clearly the kinematic variables used as inputs and added an illustration in the main text to give a clearer image of how the angular coordinates are defined by collider experiments. Please refer to Figure 1.
  • Both of ttˉt\bar{t} and QCD samples are available with high statistics. The former provides diverse and complex final-state topologies, while the latter extends the kinematic coverage, particularly into the high-pTp_T regime.

Additional ablations & comparisons

  • In a vanilla VQ-VAE the codebook size represents the number of possible configurations that the tokens can describe. As we add residual layers, each layer multiplies the possible number of configurations by its own codebook size. Thus the number of configurations that can be described by the tokenized representation balloons quickly when adding residual layers. The optimal codebook size and number of residual steps should be chosen by taking the following factors into consideration:

    1. The codebook collapse issue makes it so arbitrarily increasing the size of the codebook often doesn’t result in measurable improvements in reconstruction quality.
    2. For small autoregressive models with very big vocabularies (codebook size), the linear layer producing logits for each entry in the vocabulary can often grow to multiple times the size of the transformer backbone.
    3. Increasing the number of residual quantization steps increases the length of the modeled sequences, increasing quadratically the attention cost, and ~linearly the decoding time.

    We added quantizer ablations to the appendix (A.2.1, Table 7, Figure 9), showing that, for our dataset, the quantizer we selected performs similarly or better than others with different K and M parameters trained with the same setup. We report here a table of the SWDs between the original and reconstructed events for the ablations.

    M\times K & \text{Bits} & \text{Event} & \text{Jet} & \text{Particle} \\\\ \hline 3\times 8192 & 39 & 0.0158 & 0.0819 & 0.0476 \\\\ 4\times 4096 & 48 & 0.0111 & 0.0511 & 0.0348 \\\\ 5\times 776 & 48 & 0.0299 & 0.2955 & 0.0751 \\\\ 6\times 256 & 48 & 0.0936 & 0.2977 & 0.1042 \\\\ 4\times 8192\\ \text{(main)} & 52 & \mathbf{0.0073} & \mathbf{0.0382} & \mathbf{0.0275} \\\\ 5\times 1351 & 52 & 0.0351 & 0.0931 & 0.0476 \\\\ 6\times 406 & 52 & 0.0728 & 0.1059 & 0.0509 \\\\ 5\times 8192 & 65 & 0.0171 & 0.0637 & 0.0457 \\\\ 6\times 8192 & 78 & 0.1292 & 0.1017 & 0.0536 \end{array}$$
  • We added to the appendix a comparison between our 41M autoregressive model and a discrete DLM-style model using the same backbone with bidirectional attention, trained for the same number of steps and with the same exact training recipe and both using the same quantizer. We find that the autoregressive generator performs better even at a high number of steps for the DLM-style model.

  • We added comparisons to both Delphes, a widely used parametric simulation, and to Dreyer et al., 2026. Our model is competitive with both on most metrics, and while Parnassus is more performant in modeling the conditional response to the inputs at the particle and jet-levels, our model excels in modelling the geometric properties of the detector’s response and artifacts induced by the Particle Flow reconstruction algorithm that both use as a target, as well as improving on event-level features. Given better reconstruction from the tokenizer, which seems to be the main limiting factor in modeling quality, we expect our generator to at least reduce the gap in the features where it is being held back. Please refer to Section 3.2 and Figures 4 and 5.

Thanks for the clarifications and ablations. My main concern is that as an application paper, the value of the paper depends on demonstrating an application. The paper shows that the surrogate reproduces distributions, but not that it lets physicists do something new or useful. Does it produce the same physics conclusions as full simulation in some downstream analysis (for example a measurement or search sensitivity with uncertainties)? Without this, the paper is of limited interest from an ML perspective, and it's not clear to me what it enables for physicists.

The added timing uses non-matched published numbers on older hardware. The authors acknowledge this as an upper estimate, so it doesn't establish a strong cost advantage. The new comparison shows Parnassus a continuous generator outperforming the model on particle and jet level responses, so the benefit of discrete tokenization also isn't demonstrated.

We thank the reviewer for the continued engagement. We want to clarify that we do not claim the model is an analysis-ready replacement for the CMS full simulation today, but rather a proof of concept for residual-vector-quantized tokenization combined with autoregressive generation for fast full-event simulation. It shows that the approach works on CMS full-simulation data, and most importantly that it scales predictably and that improvements in token-level loss carry over to physically relevant metrics. To address the reviewer's question within the time constraints, we performed a limited, preliminary test on already-generated data of whether the model leads to the same analysis outcome as full simulation.

We use the same public CMS 2011 QCD and ttˉt\bar{t} events released with Parnassus, and compare the full simulation, Delphes, Parnassus and our model, starting from the same generator-level event.

1 - Signal acceptance

Different analyses use different selection criteria for analyzed events, we select a variety of published CMS papers and take the selection cuts from them. For different criteria we compare the acceptance ε=Nselected/Ntotal\varepsilon = N_{\text{selected}} / N_{\text{total}} with the one derived from full simulation. We report the ratio to the full simulation εmodel/εfull\varepsilon_{\text{model}} / \varepsilon_{\text{full}}.

\text{Selection} & \text{Sample} & \text{Delphes} & \text{Parnassus} & \text{AR 41M} & \text{AR 1B} \\\\ \hline \text{Dijet, } m_{jj}>0.9\text{ TeV [1]} & \text{QCD} & 1.103 \pm 0.003 & 0.993 \pm 0.003 & 0.978 \pm 0.003 & \mathbf{0.999 \pm 0.003} \\\\ H_T>1\text{ TeV [2]} & \text{QCD} & 1.275 \pm 0.004 & 0.988 \pm 0.004 & 0.984 \pm 0.004 & \mathbf{0.997 \pm 0.003} \\\\ \text{1 lepton} + \geq 4\text{ jets [3]} & t\bar{t} & 0.936 \pm 0.012 & 0.993 \pm 0.013 & 0.923 \pm 0.014 & \mathbf{1.001 \pm 0.012} \\\\ \geq 6\text{ jets [4]} & t\bar{t} & 0.929 \pm 0.027 & \mathbf{1.013 \pm 0.028} & 0.863 \pm 0.028 & 0.977 \pm 0.027 \\\\ \end{array}$$ Uncertainties are statistical, estimated by 200 bootstrap resamplings of the events. 2 - Impact on measurement Top quarks are produced mainly in $t\bar{t}$ pairs and decay almost always to a $W$ boson and a $b$ quark; when one $W$ decays to two quarks and the other to an electron or muon and a neutrino, the event has one lepton and four jets [3]. CMS uses these events to measure the $t\bar{t}$ production cross section (how often top-quark pairs are produced) as $\sigma = (N - B) / (\varepsilon \cdot L)$ [5], with $N$ the number of selected events, $B$ the expected background and $L$ the integrated luminosity. The acceptance $\varepsilon$ of this selection is taken from simulation. $N$, $B$ (in this setting) and $L$ do not depend on the simulator, so replacing full simulation by a model changes the measured cross section exactly by $\sigma_{\text{model}} / \sigma_{\text{full}} = \varepsilon_{\text{full}} / \varepsilon_{\text{model}}$: $$\begin{array}{l|c} \text{Acceptance from} & \sigma_{\text{model}} / \sigma_{\text{full}} \\\\ \hline \text{AR 1B} & \mathbf{0.999 \pm 0.012} \\\\ \text{Parnassus} & 1.007 \pm 0.013 \\\\ \text{Delphes} & 1.068 \pm 0.014 \\\\ \text{AR 41M} & 1.083 \pm 0.016 \\\\ \end{array}$$ With our 1B model the measured cross section is unchanged within our statistical precision, a shift of about 1% of the systematic uncertainty CMS quotes for this measurement (+8.2 / −9.2%, [3]). We want to stress that this analysis is done without energy-corrected jets, and leave calibration studies for further work. [1] arXiv:1210.2387 [2] arXiv:1208.2931 [3] arXiv:1212.6682 [4] arXiv:1302.0508 [5] arXiv:1208.2671
Reviewer_5HsFOpenReview ↗
Rating
3
Confidence
2
Informativeness
3
Presentation
3
Soundness
3
Read review
Paper summary

The paper proposes a generative model for high-energy physics detector simulation, uses a residual-quantized variational autoencoder (RQ-VAE) to represent particle-level collider events as discrete tokens, followed by a decoder-only transformer conditioned on truth-level particles to generate reconstructed events. The authors investigate scaling behaviour across models ranging from 10M - 1B parameters, varying training dataset size and repeated data exposure, and evaluate the fidelity of generated events at particle, jet, and event levels.

Critical strengths

-- The extension of residual-quantized autoregressive modelling to full collider-event simulation is a potentially important contribution. Complete particle-level event modelling, rather than individual jets or detector components, addresses a practically important problem and provides a flexible representation for downstream physics analyses.

-- The authors explored a wide range (48) of experimental configurations. This is a substantial investigation, and the explicit consideration of data repetition is particularly relevant because generating training examples using conventional detector simulation is computationally expensive.

-- Probably the paper's most interesting result is the demonstration in Fig. 6 of a consistent relationship between next-token prediction loss and sliced Wasserstein distance across particle, jet, and event-level observables.

-- The authors test on physical processes excluded from training, including Higgs decays to four leptons and higher-energy QCD events. The relatively good agreement between generated and simulated distributions suggests that the model learns aspects of the conditional detector response that will generalize.

Critical weaknesses

-- The paper lacks a direct quantitative comparison with competing full-event simulation approaches (eg. the flow-matching approach of Dreyer et al. (2026)). As a result, it is difficult to determine whether residual-quantized autoregressive modelling offers any practical advantage in generation quality, computational efficiency, or scalability. The proposed approach works, but we don't know if it is competitive with existing alternatives.

-- The approach combines established components (residual vector quantization, transformer-based autoregressive modelling, conditional generation, and existing scaling-law formulations). The contribution is primarily their application and empirical characterization in a new domain. While I believe this constitutes a worthwhile contribution, particularly for scientific machine learning, the paper offers relatively little architectural or algorithmic innovation. In my opinion this isn't a critical flaw for an applications paper.

-- The motivation is to replace expensive conventional detector simulations with faster learned surrogates. However, autoregressive generation can be computationally expensive, particularly for events with many particles and multiple tokens per particle. The paper does not establish the actual inference-time speedup relative to conventional simulation or competing ML-based simulators.

-- The authors do not systematically investigate how the number of quantization levels, codebook size, or reconstruction fidelity affect downstream generation. Specifically, does increasing tokenizer capacity produces greater benefits than increasing autoregressive model size?

Questions for authors

none

1 comment

We thank the reviewer for the valuable comments and the constructive criticism. Following is a response to the cited critical weaknesses:

The paper lacks a direct quantitative comparison with competing full-event simulation approaches (eg. the flow-matching approach of Dreyer et al. (2026)). As a result, it is difficult to determine whether residual-quantized autoregressive modelling offers any practical advantage in generation quality, computational efficiency, or scalability. The proposed approach works, but we don't know if it is competitive with existing alternatives.

We added comparisons to both Delphes, a widely used parametric simulation, and to Dreyer et al., 2026. Our model is competitive with both on most metrics, and while Parnassus is more performant in modeling the conditional response to the inputs at the particle and jet-levels, our model excels in modelling the geometric properties of the detector’s response and artifacts induced by the Particle Flow reconstruction algorithm that both use as a target, as well as improving on event-level features. Given better reconstruction from the tokenizer, which seems to be the main limiting factor in modeling quality, we expect our generator to at least reduce the gap in the features where it is being held back. Please refer to Section 3.2 and Figures 4 and 5.

The motivation is to replace expensive conventional detector simulations with faster learned surrogates. However, autoregressive generation can be computationally expensive, particularly for events with many particles and multiple tokens per particle. The paper does not establish the actual inference-time speedup relative to conventional simulation or competing ML-based simulators.

We added the full-simulation walltime to the timing table in the appendix. We used the publicly available figure that was taken in the closest simulation environment to the one that produced our samples, but it is not identical. The timing figure is also not up-to-date with the most recent hardware so it should be taken as a upper estimate. We report the timing results here as well:

\textbf{Model} & \text{A100 } B & \text{A100 evt/s} & \text{H200 } B & \text{H200 evt/s} & \text{CPU evt/s} \\\\ \hline \text{10M} & 1{,}024 & 126.5 & 2{,}048 & 265.1 & - \\\\ \text{22M} & 512 & 15.1 & 1{,}024 & 26.8 & - \\\\ \text{41M} & 256 & 62.5 & 512 & 132.5 & - \\\\ \text{97M} & 256 & 38.2 & 256 & 72.7 & - \\\\ \text{187M} & 128 & 22.0 & 256 & 48.8 & - \\\\ \text{1B} & 32 & 4.4 & 64 & 12.0 & - \\\\ \hline \text{Parnassus} & 1{,}000 & 73.5 & - & - & - \\\\ \text{Delphes} & - & - & - & - & 89.3 \\\\ \text{Full simulation} & - & - & - & - & 0.0110 \\\\ \end{array}$$ > The authors do not systematically investigate how the number of quantization levels, codebook size, or reconstruction fidelity affect downstream generation. Specifically, does increasing tokenizer capacity produces greater benefits than increasing autoregressive model size? This comparison is an interesting one, while we did run a small number of ablations that do suggest some relations, we do not think the findings are strong enough to make any particular claim. The main difficulty in testing this is that different tokenizers achieve different reconstruction quality, and this has a massive effect on the downstream quality of the generated samples. Furthermore when different quantizer networks are trained they often achieve similar average loss but the picture might look different when looking per-reconstructed-feature, meaning they might improve where another network lacks at the expense of a separate reconstructed feature. We link to this comment a table of the SWDs between the distributions produced by generators trained with the exact same training setup (albeit the compute it takes to train them differs) on different quantizers. The 5x8192 quantizer’s first layer experienced codebook collapse during training, so even though the reconstruction performance rivalled that of the other models (VQ ablations were added in appendix), modeling over the sequences it produced proved challenging for the generative model. $$\begin{array}{c|l|ccc} \textbf{Quantizer} & \textbf{Model} & \textbf{Event} & \textbf{Jet} & \textbf{Particle} \\\\ \hline 3\times 8192 & \text{VQ-Only} & 0.016 & 0.082 & 0.048 \\\\ & \text{AR 97M} & 0.113 & 0.262 & 0.105 \\\\ \hline 4\times 8192 & \text{VQ-Only} & 0.007 & 0.038 & 0.028 \\\\ & \text{AR 41M} & 0.174 & 0.276 & 0.113 \\\\ & \text{AR 97M} & 0.100 & 0.193 & 0.080 \\\\ \hline 5\times 8192 & \text{VQ-Only} & 0.017 & 0.064 & 0.046 \\\\ & \text{AR 41M} & 1.168 & 1.148 & 0.424 \\\\ \end{array}$$ Further additions made include a more detailed introduction and explanation of the features and metrics used, and in the appendix quantizer ablations and a comparison to a discrete diffusion DLM-style model trained on the same tokenized data.
Reviewer_v2jvOpenReview ↗
Rating
3(+1)
Confidence
2
Informativeness
2
Presentation
2
Soundness
2
Read review
Paper summary

This paper proposes a residual-quantized autoregressive framework for conditional full-event generation in high-energy physics. The method employs an RQ-VAE to tokenize particle-level information and an autoregressive transformer to generate reconstructed particles conditioned on truth-level events. The paper further investigates the scaling behavior across different model sizes, dataset sizes, and data repetition settings.

Critical strengths
  • Applying generative models to full-event simulation in high-energy physics is an important and meaningful research direction, with potential benefits for reducing the computational costs of traditional detector simulation.

  • The paper provides a systematic investigation of scaling behavior across different model sizes, dataset sizes, and data repetition settings. These experiments offer useful insights into the scaling properties of autoregressive models for scientific data generation.

  • The paper reveals an interesting relationship between token-level prediction loss and downstream physical fidelity. This finding is insightful and potentially useful for guiding model development and compute allocation without requiring extensive generation-based evaluations.

Critical weaknesses
  • Although the proposed method demonstrates promising generation performance, its advantages over the CMS PFlow reference are not evident. Noticeable discrepancies remain in several physical observables, and further analysis would help better establish the generation quality and practical effectiveness of the proposed method.

  • Although reducing computational costs is a primary motivation of this work, the paper does not provide direct comparisons of generation speed with traditional simulation methods. Such comparisons would help better demonstrate the practical value of the proposed method.

  • The paper lacks direct experimental comparisons with Dreyer et al., 2026, which addresses the same conditional full-event generation task using flow-based generative models. Including this important baseline would help better establish the competitiveness of the proposed method and clarify the advantages of autoregressive generation over existing approaches.

Questions for authors

Please refer to the weaknesses part. The major concern is the lacked comparison with exsiting methods.

2 comments

We thank the reviewer for the constructive criticism and the nice comments. Following is a response to the cited weaknesses/questions:

Although the proposed method demonstrates promising generation performance, its advantages over the CMS PFlow reference are not evident. Noticeable discrepancies remain in several physical observables, and further analysis would help better establish the generation quality and practical effectiveness of the proposed method.

The histograms we label CMS Pflow refer to the full Monte Carlo simulation that we use as a training target. We do agree that some of the physical observables present discrepancies relative to the target, and to other fast-simulation methods (see the following), though this seems to be a consequence of the reconstruction quality at the tokenizer level, rather than explicit mismodeling from the generator. We expect the generator’s performance to improve given a more performant tokenizer.

Although reducing computational costs is a primary motivation of this work, the paper does not provide direct comparisons of generation speed with traditional simulation methods. Such comparisons would help better demonstrate the practical value of the proposed method.

We added the full-simulation walltime to the timing table in the appendix. We used the publicly available figure that was taken in the closest simulation environment to the one that produced our samples, but it is not identical. The timing figure is also not up-to-date with the most recent hardware so it should be taken as a upper estimate. We report the timing results here as well:

\textbf{Model} & \text{A100 } B & \text{A100 evt/s} & \text{H200 } B & \text{H200 evt/s} & \text{CPU evt/s} \\\\ \hline \text{10M} & 1{,}024 & 126.5 & 2{,}048 & 265.1 & - \\\\ \text{22M} & 512 & 15.1 & 1{,}024 & 26.8 & - \\\\ \text{41M} & 256 & 62.5 & 512 & 132.5 & - \\\\ \text{97M} & 256 & 38.2 & 256 & 72.7 & - \\\\ \text{187M} & 128 & 22.0 & 256 & 48.8 & - \\\\ \text{1B} & 32 & 4.4 & 64 & 12.0 & - \\\\ \hline \text{Parnassus} & 1{,}000 & 73.5 & - & - & - \\\\ \text{Delphes} & - & - & - & - & 89.3 \\\\ \text{Full simulation} & - & - & - & - & 0.0110 \\\\ \end{array}$$ > The paper lacks direct experimental comparisons with Dreyer et al., 2026, which addresses the same conditional full-event generation task using flow-based generative models. Including this important baseline would help better establish the competitiveness of the proposed method and clarify the advantages of autoregressive generation over existing approaches. We added comparisons to both Delphes, a widely used parametric simulation, and to Dreyer et al., 2026. Our model is competitive with both on most metrics, and while Parnassus is more performant in modeling the conditional response to the inputs at the particle and jet-levels, our model excels in modelling the geometric properties of the detector’s response and artifacts induced by the Particle Flow reconstruction algorithm that both use as a target, as well as improving on event-level features. Given better reconstruction from the tokenizer, which seems to be the main limiting factor in modeling quality, we expect our generator to at least reduce the gap in the features where it is being held back. Please refer to **Section 3.2** and **Figures 4 and 5**. Further additions made include a more detailed introduction and explanation of the features and metrics used, and in the appendix quantizer ablations and a comparison to a discrete diffusion DLM-style model trained on the same tokenized data.

Dear authors,

I would like to thank for the explaination and the addtional analysis. It shows that the proposed method outperforms the previous work in some cases. Therefore I would like to raise my score. But I would also like to ask the AC to take consideration that I am not familar to this physics field.

The reviewer

From OpenReview, CC BY 4.0. Authors as signed there; shown unedited.

Related papers

Loading the map…

Discussion (0)

Sign in to comment.