acceptodds
Under review as a conference paper at ICLR 2027

Learning Physics, Not Energy: A Multimodal Benchmark for Neutrino Detectors

Abstract

The experimental search for neutrinoless double-beta decay (NLDBD) would answer one of the most important questions in physics: Why is there more matter than antimatter in our universe? A key challenge in NLDBD searches is identifying NLDBD signal events from many unwanted background events. Deep learning classifiers have shown strong performance on this task, but they often exhibit energy shortcut learning: instead of learning event topology, which reflects the underlying physical interactions, they exploit event energy as an easier shortcut to make classification decision. In this work, we conducted a comprehensive multimodal benchmark for NLDBD event classification. This work makes three contributions: First, we coordinated the public release of four NLDBD datasets spanning three modalities, including time series, point clouds, and sparse 2D images. Second, we designed two complementary metrics to quantitatively characterize topology-based classification power and the severity of energy shortcut learning. Third, we designed 13 models to benchmark across four datasets, including both native-representation models and a unified Transformer backbone adapted through different tokenization and positional encoding. Our results identify the Region + MLP Transformer as the strongest overall model across all datasets, establishing a baseline architecture and evaluation framework toward multimodal foundation models for NLDBD experiments.

Primary area:
datasets and benchmarks
Keywords:
Physics, Multimodal Dataset, Shortcut Learning, AI4Science, Transformer, Neutrino

Reviews

Among:
Shape:
1234this paper, Bottom 27%
Figure 1. Mean review rating of the reviewed papers at ICLR 2027. This paper averages 2.00 over 3 reviews, above 27% of the others.
Reviewer_vovfOpenReview ↗
Rating
2
Confidence
2
Informativeness
2
Presentation
2
Soundness
2
Read review
Paper summary

This paper introduces a benchmark for signal-like versus background-like event classification across four neutrinoless double-beta-decay detector datasets spanning waveforms, point clouds, and sparse images. It proposes an energy-weighted AUC and a histogram-based energy-independence score, then compares four native-representation models with nine variants of a shared Transformer backbone. The principal empirical claim is that Region + MLP is the most consistently strong configuration across modalities, while the broader resource contribution is a coordinated release and common evaluation framework.

Critical strengths
  • The coordinated release of datasets from distinct detector technologies is potentially valuable to both scientific ML and experimental-physics communities.
  • The paper is unusually transparent about what its two energy-aware metrics do and do not establish. I appreciate the honesty.
Critical weaknesses
  • Every model is trained with one seed, many reported differences are only 0.001–0.01 AUC, and NEXT permits portions of the same source file in different splits. It would be helpful to include statistical analysis, e.g., multiple seeds.
  • If could be wrong, but counting “Top-3” finishes across four heterogeneous proxies seems to be an arbitrary rank statistic, not evidence of a detector-general representation.
Questions for authors
  • What experiment directly demonstrates that the classifiers use event topology rather than another non-topological correlate, given that neither AUCw nor the independence score identifies the predictive feature?
  • How do the rankings change when models are trained on energy-matched samples and evaluated within narrow energy strata, alongside the current pooled, post-hoc weighted AUC?
  • Are the 0.001–0.01 AUCw differences in Table 1 stable over independent training seeds, and what are the resulting confidence intervals and Top-3 membership frequencies?
  • Why is “Top-3 count” an appropriate aggregate statistic when it discards effect size, uncertainty, and the very different difficulty and sample size of each dataset?
  • Does Region+MLP’s Top-3 pattern persist when native architectures and Transformer variants are chosen with comparable, documented validation-only selection procedures and search budgets?
3 comments

We thank the reviewer for the specific and actionable questions. Please also see our general response.

W1 / Q3. Single seed and small differences.

All 13 models are now trained with 3 seeds on all 4 datasets (natives with 6). Seed-to-seed spread (SD) in AUCw is below 0.005 for most models. It is larger for Entity+MLP and the RoPE Summary/Entity variants on MJD (0.02–0.06). The test sets are large (117k–590k events), so bootstrap CIs over test events are much smaller than this. Seed spread is therefore the uncertainty that matters for rankings. In practice:

  • Differences between tokenizations (Region vs Summary vs Entity, ≳ 0.005) hold across seeds.
  • Differences within the Region family do not. Region+MLP and Region+Fourier are tied.
  • On SuperNEMO, every model lies within 0.578–0.592 AUCw, so ranks there are not meaningful. We now state that in the paper.

NEXT split. Our wording in Appendix A.3 overstated the issue. No event is ever divided between splits. Of 14,973 source files, none is shared between train and test. Two files are shared between validation and test, contributing 39 of 116,549 test events. Removing them changes inclusive AUC, AUCw and I by at most 0.0001 for every model and seed. We will correct the appendix.

W2 / Q4. Top-3 count.

We agree it discards effect size and uncertainty, and we have dropped it. Across seeds:

ModelMean AUCw [95% CI]Avg rank
Region+Fourier0.8798 [0.8787, 0.8807]4.25
Region+MLP0.8797 [0.8785, 0.8809]2.50
Region+RoPE0.8780 [0.8774, 0.8787]4.25
GINE (best native)0.8755 [0.8750, 0.8760]4.50

The full table covers all 13 models. We now describe Region+MLP as tied for the best mean and the most consistent across datasets, not as the strongest model.

Q1. Topology vs. non-topological correlate.

We scrambled the arrangement of each event and kept every sample value, and therefore its energy and amplitude distribution:

  • MJD: shuffling the time order of the waveform drops AUCw from 0.93–0.98 to 0.49–0.54 for all 10 models (Transformers and natives). The classifiers use the pulse shape, not a scalar summary of it.
  • EXO-200: shuffling channels drops Region+MLP from 0.970 to 0.845. Shuffling time blocks drops Entity+Fourier from 0.947 to 0.585. Each scramble keeps part of the structure, so the drop is partial, not a collapse to 0.5.
  • NEXT: we retrained Region+MLP (seeds 42, 1) and Region+Fourier (seeds 1, 2) from scratch on scrambled voxels. Moving each voxel to a random point in the event's bounding box removes the track but keeps voxel energies, count and extent: AUCw falls from 0.981–0.988 to 0.922–0.925 in all four runs. Shuffling energies among voxels (end blobs moved) or setting them to the event mean gives 0.942–0.958. The track is worth 0.057–0.065 AUCw, the blob positions 0.025–0.039 (paired-bootstrap CI ±0.001). The rest comes from what the scramble keeps (voxel energies, count, extent), similar to EXO-200 the drop is partial. Only scramling the test events of the trained model gives 0.51–0.86, which mixes lost information with unfamiliar input.

Energy alone gives the same AUCw before and after every scramble, as expected. On NEXT, I is also unchanged (0.969–0.974).

Q2. Narrow energy windows and energy-matched training.

(i) In a 100 keV window at Qββ, energy alone gives AUC = 0.50. The Region and Summary Transformers keep high AUC there: MJD 0.96–0.99, EXO-200 0.93–0.96, NEXT 0.98–0.99. The ranking inside the window is close to the overall one. On EXO-200 the three Region models are the top three, on MJD Region+RoPE and Region+Fourier lead, and on NEXT Region+MLP is the best Transformer.

(ii) We retrained Region+MLP and Summary+MLP on energy-matched training sets (one seed). On MJD the order is unchanged (0.965 vs 0.941). On EXO-200 Region+MLP's advantage disappears (0.953 vs 0.955, within seed noise).

Q5. Comparable selection procedures.

We re-selected the NEXT native candidates using validation data only. Validation and test rankings agree exactly within every architecture group (Spearman ρ = 1.0), so the test set did not change any choice. However, our stated rule ("best of each group") was not what we did: only GINE is its group's validation winner. The validation winners in the other groups (PointMLP 0.994, sparse 3D ResNet 0.991, CNN–GNN hybrid 0.991) all score above Region+MLP on NEXT (0.987). We will report them and correct the text. The native models also keep their own training settings (Table 9), so the comparison is not at an equal tuning budget. We therefore no longer claim that Region+MLP beats tuned modality-specific models on each dataset. Our claim is narrower: one shared backbone is competitive across four detector types.

I thank the authors for their thoughtful responses, which have addressed many of my initial questions. I can increase my score but I do believe the contributions of this paper are a better fit for a domain-specific journal than an AI conference.

We thank the reviewer for engaging with our responses, and for considering a higher score. We appreciated the flaws pointed out in the paper, and we spent considerable amount of time running experiments, learning from them, and updating our paper.

We originally submitted to the ICLR datasets and benchmarks area because we think these questions go beyond physics: both metrics apply to any nuisance variable measured for every event, not just energy (e.g. mass sculpting in collider jet tagging). We hoped the benchmark would be useful to the broader ML community. That being said, we will definitely keep in mind a more domain-specific journal for future work.

Thanks again for your time and constructive questions and responses.

Reviewer_1hHoOpenReview ↗
Rating
2
Confidence
3
Informativeness
2
Presentation
2
Soundness
2
Read review
Paper summary

This paper presents a comprehensive multimodal benchmark for neutrinoless double-beta decay event classification. The authors also design two families of neural architectures, a native-representation approach and a unified transformer approach, and report results for both on the proposed benchmark.

Critical strengths

The paper contributes a domain-specific benchmark that should benefit the neutrino physics community by lowering the barrier to applying modern machine learning methods to event classification. Such a benchmark is timely, given the growing interest in AI-driven analysis within neutrino science.

Critical weaknesses

The methodological novelty on the machine learning side is limited. Both architecture families appear to adapt established designs rather than introduce techniques motivated by the specific structure of the data.

Questions for authors

N/A

1 comment

We thank the reviewer for the positive assessment and the comments. Please also see our general response for the new experiments and two corrections to Table 1.

Limited ML novelty.

We agree that we do not propose a new architecture. The contribution is a benchmark and resource: datasets, an evaluation protocol for a known nuisance variable, and baselines across data types. We hope that this fits the scope of the ICLR datasets and benchmarks area.

The main ML finding now holds across seeds. One small Transformer (~110K parameters) uses the same backbone on all four detectors and changes only the tokenizer and positional encoding. Over 3 seeds, its Region-token variants tie for the best mean AUCw (Region+Fourier 0.8798, Region+MLP 0.8797; difference −0.0001, CI ±0.0016). The best modality-specific model is GINE at 0.8755, and its CI does not overlap theirs. The tokenizers follow the structure of each dataset (Table 11); for example, region tokens group hits at the scale of the detector's energy deposits. Averaged over datasets, Region tokens beat Summary tokens, which beat Entity tokens.

Reviewer_MuDUOpenReview ↗
Rating
2
Confidence
3
Informativeness
2
Presentation
3
Soundness
3
Read review
Paper summary

This paper presents a study of the phenomenon of short-cut-learning in the context of the task of neutrinoless double-beta decay (NLDBD) detection. The claimed contributions are a multimodal benchmark across different NLDBD detectors and a study of the short-cut learning phenomenon in this application domain.

Critical strengths
  • Interesting and important application domain
  • Short cut learning is an important challenge in many learning problems. Defining useful metrics for it's detection and evaluation would be a clear contribution.
  • Paper is very well-written, easy to understand and a pleasure to read.
Critical weaknesses
  • The paper strikes me as having a significant structural problem. It is motivated as a benchmark paper, but the datasets that define the benchmark all already exist as published works. So the benchmark seems like a consolidation of existing work and would constitute a relatively weak contribution on that basis. It is also motivated as an applications paper, with the application being the detection of neutrinoless double-beta decay (NLDBD), however the paper offers relatively little in the way of advancing the state-of-the-art (SOTA) in this task (or even establishing what the SOTA is). The primary contributions of the paper centre on a study of the phenomenon of short-cut-learning (albeit in the content of NLDBD detection), but as such it has some significant weaknesses (3 elaborated below).

  • (1) Short-cut-learning is an important phenomenon that most ML researchers or practitioners have encountered at some point. However this paper approaches this general phenomenon from a very specific and narrow perspective of the NLDBD detection task, where -- critically -- there isn't really a short cut learning problem. By this I mean, there is no evidence presented that the phenomenon leads to sub-optimal performance in any sense. It seems like the energy (the feature identified as the short-cut feature seems like a genuine high-performing feature. This paper seems to be setting it self up to study a problem that doesn't exist.

  • (2) The central contributions are the definition of two metrics for measuring short-cut learning, but they are both deeply embedded in the specific application domain: (a) the Energy Independence Score, and (b) the Energy-weighted AUC. Both of these are likely of limited utility to others encountering actual problems with short-cut learning as they are application specific and require the full identification (including labels) of the feature causing the problem. This is not the typical short-cut learning situation.

  • (3) But perhaps most critically, these two metrics were only investigated in the context of a set of different detector models (including "native" single-modality representation approaches and a unified Transformer approach) on the NLDBD detection task where short-cut learning doesn't seem to be a problem. So we have no evaluation of these metrics in a context where they would be useful, i.e. where short-cut learning is a problem and needs to be detected. While on the topic, the comparison of the different modelling approaches is interesting, and could constitute the basis of a nice applied paper (on the NLDBD detection task), but the focus would need to widen and not focus solely on the evaluation via the proposed metrics..

  • Finally the paper presents a rather simplified perspective on short-cut learning. I recommend the authors dig a bit deeper. For example work such as Pezeshki et al (2020) "Gradient Starvation: A Learning Proclivity in Neural Networks" considers the same phenomenon mechanistically. This is just a suggestion.

Questions for authors

none.

1 comment

We thank the reviewer for the detailed review. Please also see our general response for the new experiments and two corrections to Table 1.

Benchmark only consolidates existing work; no SOTA.

We would like to clarify that data curation is part of contribution of this work. Three of the four datasets (NEXT, SuperNEMO, EXO-200) were released publicly for the first time by the author’s team in 2026;  Only MJD dataset was public in 2023 before this paper. The benchmark also adds common task definitions, splits, loaders and one evaluation protocol across three data types. The purpose of this manuscript is to provide a systematic benchmarking study over the dataset as a baseline for the future work.

(1) Energy is a genuine feature, so there is no shortcut problem.

In a 0νββ search the decision is made inside a narrow window around Qββ. There, signal and background have the same energy, so energy carries no information. However, the classifier's signal efficiency is calibrated on events at other energies (in MJD and EXO-200, the 208Tl double-escape peak at ~1593 keV, not Qββ). If the score depends on energy, the efficiency measured at the peak is wrong at Qββ. That is a bias in the half-life, and a change in AUC would not reveal it.

We measured this directly. We fixed a cut at 90% signal efficiency on the double-escape peak and measured the efficiency at Qββ:

Region+MLP efficiency, peak → QββRegion+MLP Δε [95% CI]Δε, other modelsRegion+MLP Δε, energy-matched training
MJD90% → 85%−0.048 [−0.086, −0.024]0 to −0.07−0.060
EXO-20090% → 78%−0.124 [−0.163, −0.083]−0.02 to −0.16−0.036

These models have small gaps between inclusive AUC and AUCw (≤ 0.03). Yet their efficiency is off by 5–12 points at Qββ. On EXO-200, retraining on an energy-matched training set cuts the shift from −0.124 to −0.036. On MJD it does not, so part of the MJD shift is not caused by energy in the training set. We agree the paper did not make this argument. The revision adds it, with the efficiency-vs-energy curves for every model.

(2) The metrics are application-specific and need the shortcut feature to be known.

Both metrics need a nuisance variable that is measured for every event. In physics this is the usual case: energy, position and pileup are all recorded. Collider jet tagging has the same problem with jet mass ("mass sculpting") [Dolen et al. 2016; Louppe et al. 2017; Kasieczka & Shih 2020]. AUCw is a covariate-standardized AUC (Liu & Zhou 2013). I measures how stable the score is within each class as the covariate changes. Neither depends on anything specific to energy. We agree they cannot find unknown shortcuts, and the revision states this scope.

(3) The metrics are never tested where a shortcut actually occurs.

We added this test. On MJD and EXO-200 we subsampled the training set so that energy alone separates the classes with AUC 0.50 (none), 0.60–0.68, and 0.74–0.84 (strong). The test set is the same for all runs. For Region+MLP, going from no correlation to strong correlation gives:

AUCwIΔε (peak → Qββ)
MJD0.965 → 0.9380.797 → 0.757−0.060 → −0.006
EXO-2000.953 → 0.9390.795 → 0.663−0.036 → +0.075

On EXO-200, AUCw barely moves while I drops by 0.13 and the efficiency shift changes sign. Summary+MLP behaves the same way. As a positive control we turned off amplitude normalization on MJD, so raw pulse height (a direct energy proxy) reaches the model. I then grows from 0.040 to 0.111. This shows that I and the efficiency shift detect the shortcut, and AUC alone largely misses it.

Gradient starvation.

Thank you for your suggestion, we will keep this reference in mind for future work.

Comments on the paper

3 comments

Dear authors and reviewers,

The author-reviewer discussion window will close tomorrow on November 18.

@authors: If not done already, please respond to the reviewers' comments before the discussion window closes. I would advise short and concise responses over lengthy explanations.

@reviewers: Please check the authors' responses and provide any additional comments before the discussion window closes. Consider updating your initial review if necessary, and in particular your score if your opinion has changed. You may increase your score if you feel the authors' responses warrant it, maintain your score if your opinion remains the same, or even decrease your score if you feel the other reviews or the authors' responses reveal new shortcomings. At the very least, please acknowledge the authors' responses.

Thank you for your attention and cooperation.

The AC

AuthorsOpenReview ↗

General Comments

We thank all reviewers for their careful reading and constructive comments. Since submission we have run the following experiments. Unless stated otherwise, every number in our responses comes from a single evaluator applied to all models, with training seeds 42 (as in the paper), 1 and 2.

  1. Multiple seeds. All 13 models on all 4 datasets, 3 training seeds each (the 4 native models have 6 seeds on each dataset), all scored with one evaluator. We report the mean, the spread across seeds and 95% CIs.
  2. Top-3 count replaced. In its place we report mean AUCw with a CI, the gap to the best model on each dataset, and the average rank.
  3. Controlled shortcut test. We retrained models on training sets where we set how strongly energy predicts the label, from no correlation up to a strong one.
  4. Narrow energy windows. We now report AUC inside a 100 keV window at Qββ, where energy alone gives AUC = 0.50.
  5. Topology ablation. We scrambled the time or space arrangement of each event while keeping its energy and sample values, on MJD, EXO-200 and NEXT.
  6. Validation-only re-selection of the NEXT native candidates.

Re-running everything with one evaluator also turned up two errors in the submitted Table 1. We correct them here:

  • The NEXT RoPE rows were failed runs (e.g. Region+RoPE 0.512). Rerun under the paper protocol, Region+RoPE reaches 0.977 on NEXT and ranks third overall. Entity+RoPE still fails on NEXT (~0.52) in every seed.
  • The SuperNEMO Transformer I values (0.974–0.977) came from a coarser 8-bin version of the metric. With the standard 5 keV bins they are 0.921–0.922, the same as the native models. We withdraw the claim that the Transformer has better energy independence on SuperNEMO, however it is competitive.

Details are in our individual responses to each reviewer, and all of the above will be included in the revision.

Hello Reviewers,

We have uploaded a revised paper incorporating all the analyses discussed in our responses. In Section 6.2 and Appendix C now include energy-only baselines, controlled energy-shortcut experiments, and calibration-transfer tests. These analyses reveal residual energy dependence, including shifts in signal efficiency between the calibration energy and the decay’s Q-value. We have updated the abstract and conclusions to reflect these findings and clarified the interpretation and limitations of the two energy-aware metrics.

Thank you for your feedback, which helped us strengthen the analysis on our paper and address concerns during review.

From OpenReview, CC BY 4.0. Authors as signed there; shown unedited.

Related papers

Loading the map…

Discussion (0)

Sign in to comment.