acceptodds
Under review as a conference paper at ICLR 2027

DecayFM: A Physics-Structured Foundation Model for Data-Efficient Rare Particle Decay Identification

Abstract

Identifying particle decay modes from reconstructed detector measurements is essential to high-energy physics, but building separate classifiers for many channels is costly, particularly when downstream training samples are limited. This setting presents two challenges for transferable representation learning: heterogeneous physical measurements contribute differently across decay modes, and highly imbalanced training frequencies can bias learning toward common channels. We introduce DecayFM, a physics-structured foundation model for data-efficient particle decay identification. DecayFM separately encodes kinematic, energy-deposition, and particle-identification features, integrates them through a particle-set Transformer, and applies paired low-rank adapters whose contributions are weighted by a learned head–tail router. A two-stage training procedure separates shared representation learning from frequency-aware adaptation. We construct the ParticleDecayMode (PDM) dataset series using BESIII simulation, comprising approximately 8.3 million decay events for large-scale pretraining and downstream evaluation, including a benchmark of 200 rare decay modes. On rare-mode classification tasks, pretrained DecayFM improves accuracy over the reported from-scratch baseline by up to 11.61 percentage points. In the evaluated few-shot settings, parameter-efficient adaptation with 10-100 training events per class exceeds the accuracy of the corresponding baseline trained with 1,000 events per class. These results support physics-structured pretraining as a promising approach to reducing downstream training-data requirements for rare decay identification.

Primary area:
applications to physical sciences (physics, chemistry, biology, etc.)
Keywords:
particle physics, foundation model, scientific machine learning

Reviews

Among:
Shape:
1234this paper, Bottom 10%
Figure 1. Mean review rating of the reviewed papers at ICLR 2027. This paper averages 1.67 over 3 reviews, above 10% of the others.
Reviewer_7b3oOpenReview ↗
Rating
1
Confidence
4
Informativeness
1
Presentation
3
Soundness
1
Read review
Paper summary

The paper presents the DecayFM model for classifying J/psi decay modes in high energy physics. DecayFM embeds kinematic, energy, and particle features separately, combines them with a transformer block, and finally uses a router + classification head. Results on classification, transfer learning, and few-shot learning are presented on a new rare decays dataset.

Critical strengths
  • Interesting application to rare decays in high energy physics.
  • Introduces a new, decently-sized dataset (which I hope the authors will release publicly).
  • The paper is mostly well-written and the method and design choices are easily understood.
Critical weaknesses

Unfortunately, I have significant concerns regarding the experimental rigour, novelty, and significance of this work.

Soundness

  • Overall the methodology is weakly motivated and not supported by empirical results and ablation studies. The authors discuss their reasoning for choices such as separate embeddings for subsets of features and the long-tail-aware router, but these are all based on assumptions that are never tested. It is important to test e.g. replacing the separate embeddings by a joint embedding, or removing the long-tail-aware router, etc. Some of the decisions such as separate embeddings may sound nice on paper but could in fact unnecessarily restrict the expressivity of the network while not reducing parameters.
  • Many of the previous works mentioned such as MPM and JetCLR are self-supervised learning methods, so it is not clear why they would be good benchmarks for a supervised learning method such as DecayFM.
  • The paper focuses entirely on supervised training on simulations with no discussion of the downstream use cases and how such a model would be calibrated for application to real data.
  • Missing a lot of training details such as model sizes (both for DecayFM and benchmarks), training time, LoRA ranks (which should be ablated as well), etc.
  • No error bars over multiple seeds.

Informativeness

  • The model architecture is a basic combination of existing pieces and not novel. This could perhaps be compensated for by rigorous ablation studies, but those are not provided either. Thus, there are no clear conceptual or empirical insights gained from this work.
  • It is expected that a pretrained model exposed to more training should outperform a model trained from scratch on a much smaller dataset. This has been demonstrated in myriad domains, included particle physics as well, so I’m not sure what the novelty is in the experimental results.
  • The long-tail-aware router could be an interesting component, but it needs both ablation studies as well as engagement with prior work on routers, such as for MoE models, and empirical analysis of whether the router is actually working as expected.
  • Overall, as the subfield of classification algorithms in particle physics is fairly mature now, it would be far more informative to start with an existing high-performing baseline such as ParT and test your new additions as modifications to the baseline one-by-one. By implementing many different things at the same time from scratch in DecayFM, it is hard to disentangle which aspects are responsible for any gains over existing baselines.

Presentation

  • Overall, it is missing a high-level motivation for why we should be interested in rare J/psi decays specifically in the first place. I am not aware of previous publications in this particular subfield of particle physics at ICLR or similar venues, which is why a clear motivation is all the more important.
Questions for authors

I list my points of confusion and weaknesses for the authors to address above. Some additional questions:

  1. Can you support each of your design decisions with careful ablation studies?
  2. A lot of the design decisions are applicable to other particle physics datasets as well such as boosted jets at the LHC; can you thus also test your model on established datasets such as JetClass to more clearly benchmark it against existing baselines?
  3. Why focus specifically on J/psi decays? It seems an overstatement to term this a “foundation model” without demonstrating generalization to other particles.
  4. Will you release the dataset publicly?
Reviewer_TzZGOpenReview ↗
Rating
2
Confidence
4
Informativeness
1
Presentation
3
Soundness
1
Read review
Paper summary

The paper proposes a training strategy for classifying particle decays from their observed stable remnants. At it's core it's a classification task on permutation-invariant set-like inputs.

The main contribution is the proposal of an routing model that has separate modules for high-frequency classes and "tail" classes.

Critical strengths

1/ the model seems to work and there are indications that the inductive bias might help, however no uncertainties are given so it's hard to judge and the inductive bias benefit may vanish at scale

Critical weaknesses

1/ The scale or scope is not at the level of a foundation model, e.g. other models / datasets cited here are trained on hundreds of millions or tens of billions samples and cover multiple tasks. Here, the focus is on decay classification only.

2/ no uncertainties are provided in tables and therefore the differences between the methods are hard to judge

Questions for authors

1/ what is the relevance of the complex embedding structure. Have you tested having a vanilla backbone but still maintain the router for head/tail?

2/ can this model be used for any other task other than decay classification?

3/ for a foundation model I would expect much larger datases, have you explored this?

4/ how are the hyper parameters / capacities of the baselines (number of parameters) chosen?

Reviewer_NxXROpenReview ↗
Rating
2
Confidence
4
Informativeness
1
Presentation
3
Soundness
1
Read review
Paper summary

This paper proposed DecayFM, a supervised pre-train model with in-domain downstream test of BESIII decay simulated data with physically grounded features. This is not a self-supervised pretrained model and did not show scaling law, in terms of model parameters, data size, and compute powers with their generalizability on down-stream analysis. However, they showed LoRA based fine tuning and head-tail routers to cope with unbalanced distribution along with the benchmark suit release (ParticleDecayMode (PDM) benchmark suite).

Critical strengths
  • This paper addresses a significant, real-world operational challenge in hadron spectroscopy and rare-decay searches, moving beyond inclusive jet tagging to identify hundreds of exclusive decay channels.

  • The authors created the 8.3M-event PDM dataset series provides a much-needed standardized testbed for evaluating representation transfer on reconstructed detector objects.

  • When provided with 50 to 100 events per class, the supervised pretrained model transfers effectively to rare decay modes, outperforming a baseline trained from scratch with 1,000 samples per class.

Critical weaknesses
  • Not a foundation model, strictly speaking. Rather, the paper brands a supervised multi-class classifier as a foundation model. Stage 1 pretraining is not self-supervised (e.g., masked constituent modeling or contrastive learning); it is standard 150-way supervised classification using cross-entropy on BESIII collision events. The downstream evaluation performs the exact same task in the same detector environment. This constitutes conventional in-domain supervised transfer learning rather than a task-agnostic foundation model. Also it misses scaling study (data, model, and compute).

  • I don't like few shot claim either. The headline sample-efficiency gains require 50 to 100 samples per class (Table 4). Across 200 classes, this amounts to 10,000–20,000 labeled events, which falls well outside the standard machine learning definition of few-shot learning. Under true few-shot conditions (K∈{1,5}K \in \{1, 5\}), the model struggles significantly—achieving only 6.77% (1-shot) and 10.56% (5-shot) accuracy on the 200-class task.

  • The empirical evaluation lacks essential ablation studies: No comparison between the 3-subspace physical feature split and a standard unified projection. No comparison between the dual-router LoRA mechanism and standard, single-branch LoRA.

Questions for authors
  • What is the downstream classification performance on PDM-rare-200 when replacing the 3-subspace encoder with a standard single-layer MLP, and when replacing the head/tail routed adapter with a single standard LoRA adapter?

  • On PDM-rare-200, what is the empirical mean and variance of the predicted routing probability pp? Does it collapse near zero across all test instances?

  • LoRA Failure on In-Domain Tasks: What causes the noticeable drop in accuracy for Decay head + LoRA on 100- and 150-class in-domain classification in Table 1?

  • Class Disjointness: Are the 200 rare modes in PDM-rare-200 strictly disjoint in their intermediate resonance topologies from the 150 modes in PDM-150-Pretrain, or is there substantial topological overlap?

From OpenReview, CC BY 4.0. Authors as signed there; shown unedited.

Related papers

Loading the map…

Discussion (0)

Sign in to comment.