acceptodds
Under review as a conference paper at ICLR 2027

RINO: Renormalization Group Distillation with No Labels

Abstract

Supervised machine learning in high-energy physics (HEP) requires training on labeled simulations of particle collisions, which are known to be unfaithful to the underlying physics. This leads to significant inaccuracies and biases in the transfer of trained models to real data, limiting the power of deep learning methods. Self-supervised learning (SSL) is a promising alternative for pretraining directly on real, unlabeled data. However, existing techniques largely operate with inductive biases inspired by domains such as images and natural language. We propose RINO (**R**enormalization Group D**I**stillation with **NO** Labels), which, for the first time, exploits the fundamentally multi-scale renormalization group structure of high-energy *jet* formation as a physically grounded source of SSL views. We define teacher-student representations across energy scales using different levels of hierarchical clustering, aligned via DINO-style self-distillation. To mirror the background-dominated nature of real collider data, RINO pretrains exclusively on jets from quantum chromodynamics (QCD) processes and is evaluated on out-of-distribution (OOD) transfer to rare signal classes. In our controlled data-efficiency study, RINO matches fully supervised training on JetNet after finetuning on only 1% of the labeled data. With the full labeled dataset, RINO improves OOD classification accuracy on top-quark jets by 5.9% and, without retraining, improves AUC on the unseen Higgs boson decay channel by 4.1% over the strongest SSL baseline. Scale-structure diagnostics reveal that DINO-style self-distillation permits our backbone to retain scale-dependent structure. Ablation studies show that this accounts for an 8.6% OOD accuracy gap with respect to alternative SSL paradigms such as SimCLR. Code is available at https://anonymous.4open.science/r/RINO-95DC.

Primary area:
applications to physical sciences (physics, chemistry, biology, etc.)
Keywords:
self-supervised learning, self-distillation, multi-scale representation learning, out-of-distribution generalization, high-energy physics, jet tagging, top tagging, renormalization group

Reviews

Among:
Shape:
1234this paper, Top 30%
Figure 1. Mean review rating of the reviewed papers at ICLR 2027. This paper averages 2.67 over 3 reviews, above 70% of the others.
Reviewer_1hwGOpenReview ↗
Rating
3(+1)
Confidence
1(−1)
Informativeness
2
Presentation
2
Soundness
2
Read review
Paper summary

RINO is a self-supervised jet-representation method that applies DINO/iBOT-style teacher-student training across views obtained by stopping exclusive kT clustering at different numbers of subjets. It pretrains on QCD jets from JETCLASS, finetunes on labeled JETNET top-versus-QCD data, and evaluates on JETCLASS top jets and other unseen signal topologies. The paper reports improved OOD top-tagging, label efficiency, cross-topology transfer, and smaller representation-space domain discrepancies than several controlled-backbone SSL baselines.

Critical strengths
  • Using physically ordered kT coarse-graining as the view construction is intuitive.
  • The paper includes useful controlled comparisons, like Tables 1 and 3, Section 4.5.
Critical weaknesses
  • I noticed RINO’s teacher settings were selected using JETCLASS OOD top-tagging accuracy, the headline metric. Can the authors identify a separate tuning split or report the tuned result on an untouched test set.
  • Evaluation events are disjoint from pretraining, but all SSL methods pretrain on unlabeled JETCLASS QCD. I would like to ask the authors to qualify the OOD claim as transfer with access to target-domain background data.
  • Baselines inherit published pretraining settings while RINO receives target-based tuning; reported deviations cover fine-tuning seeds. It would be more appropriate to assess the comparison under comparable tuning opportunities and independent pretraining runs.
Questions for authors
  • Exactly which JETCLASS events and partitions are used for QCD pretraining, hyperparameter selection, probes, and final evaluation; are any final-evaluation QCD events present during pretraining?
  • Can all headline results be recomputed on an untouched target test set after teacher temperature and EMA momentum are selected without using JETCLASS target labels?
  • Why is the protocol described as target-label-free OOD evaluation when Appendix A.17 explicitly uses target-domain top-tagging accuracy to select RINO hyperparameters?
  • What happens when every baseline receives the same target-validation access and hyperparameter-search budget as RINO?
  • How many independently pretrained backbones were used per method, and how much variance remains when SSL pretraining, not only downstream finetuning, is repeated?
  • For the ablation controls the clustering objective and nominal view scales. Can matched-count or information-retention controls further distinguish energy-ordered physical structure from differences in what each clustering algorithm preserves?
5 comments
AuthorsOpenReview ↗

Reply to Reviewer 1hwG (Part 1 of 3)

We thank the reviewer for the close reading of our protocol and the constructive questions. We have run new experiments to address them, and respond below, grouped by topic. Notably, the teacher-setting selection on JetClass is not needed: RINO retrained with DINOv2's default teacher setting [1] achieves an accuracy of 0.843±0.0210.843 \pm 0.021 over three independent pretraining runs, against 0.840±0.0150.840 \pm 0.015 for the tuned setting. We share the key results first. The complete tables with the other entries will be updated in the revised PDF.

1. Partitions, and Overlap Between Pretraining and Evaluation

Exactly which JETCLASS events and partitions are used for QCD pretraining, hyperparameter selection, probes, and final evaluation; are any final-evaluation QCD events present during pretraining?

Partitions and overlap. None of the final-evaluation QCD jets is used to pretrain or finetune the models. The partitions are files of the official JetClass splits (train_100M, val_5M, test_20M): every SSL method pretrains on the 10M unlabeled QCD jets of train_100M, with the 500k QCD jets of val_5M as the pretraining validation set, and all finetuned taggers are evaluated on top and QCD jets from test_20M, which holds 2M of each. To reduce the computational burden, the kk-NN and linear probes of the pretrained backbones use a random subsample of 200k of the test-split samples, fit and scored by cross-validation. A.4 gives the pretraining size and states the disjointness, but we will revise A.4 to explicitly state the splits, the pretraining validation set, and the probe subsample.

Hyperparameter selection. The reported model's teacher settings are indeed selected on the JetClass accuracy of this evaluation split in the current submission (A.17), which we agree should not enter any decision. As we show in Section 3, this selection is not needed: with DINOv2's default teacher setting, set before any of these runs is evaluated, RINO yields 0.843±0.0210.843 \pm 0.021 on the same test split, against 0.840±0.0150.840 \pm 0.015 for the tuned setting. Their means differ by 0.3 percentage points, well within the spread between pretraining runs of either one.

2. Access to Target-Domain Background

Evaluation events are disjoint from pretraining, but all SSL methods pretrain on unlabeled JETCLASS QCD. I would like to ask the authors to qualify the OOD claim as transfer with access to target-domain background data.

We agree and will qualify the claim accordingly in Sec. 1 and Sec. 4.1 of the revised PDF: the OOD result is a transfer in which pretraining has access to unlabeled background (QCD) jets from the target domain. This matches how the method is used in practice: the tagger is deployed on collider data, which are unlabeled, and self-supervised pretraining can use them because it needs no labels. In our benchmark, the unlabeled JetClass QCD jets are a proxy for QCD-dominated real data (Sec. 4.1). However, no JetClass label enters the pretraining or finetuning stage; the signal class never enters pretraining; and none of the 2M evaluation QCD jets appears in pretraining (1 above).

We also test RINO without this access. The model is pretrained instead on Aspen Open Jets [2], real jets from the CMS 2016 Open Data with no JetClass jets at all, and then finetuned on JetNet. RINO's accuracy is 0.813±0.0210.813 \pm 0.021 on JetClass top vs. QCD over independent pretraining runs, against 0.840±0.0150.840 \pm 0.015 when pretrained on JetClass QCD (±\pm over pretraining runs in both). Without JetClass jets in pretraining, RINO's accuracy is about 3 percentage points lower, but it still transfers.


[1] M. Oquab, et al. DINOv2: Learning Robust Visual Features without Supervision. Trans. Mach. Learn. Res., 2024. arXiv:2304.07193.

[2] O. Amram, et al. Aspen Open Jets: unlocking LHC data for foundation models in particle physics. Mach. Learn.: Sci. Technol., 6:030601, 2025. arXiv:2412.10504.

AuthorsOpenReview ↗

Reply to Reviewer 1hwG (Part 2 of 3)

3. Selection on the Headline Metric, and Independent Pretraining Runs

I noticed RINO's teacher settings were selected using JETCLASS OOD top-tagging accuracy, the headline metric. Can the authors identify a separate tuning split or report the tuned result on an untouched test set.

Can all headline results be recomputed on an untouched target test set after teacher temperature and EMA momentum are selected without using JETCLASS target labels?

Why is the protocol described as target-label-free OOD evaluation when Appendix A.17 explicitly uses target-domain top-tagging accuracy to select RINO hyperparameters?

How many independently pretrained backbones were used per method, and how much variance remains when SSL pretraining, not only downstream finetuning, is repeated?

We would like to demonstrate that the teacher-setting tuning is not necessary. RINO retrained with DINOv2's default teacher setting yields 0.843±0.0210.843 \pm 0.021 over three independent pretraining runs, against 0.840±0.0150.840 \pm 0.015 for the tuned setting. JetClass labels are therefore not needed to select the teacher settings, and the main results no longer use any teacher setting selected on JetClass.

Why we remove the selection instead of moving it. In the paper's protocol, JetClass is a proxy for real collider data (Sec. 4.1, A.13), so it should be treated as an unlabeled dataset, with its labels entering only for evaluation. First, a separate JetClass tuning split would come from the same distribution as the test set, so selecting on it would still rely on labels that do not exist in practice. Second, real data carry no labels on which one can select a set of hyperparameters in hindsight. Therefore, we do not select the teacher settings at all: the teacher temperature and the EMA momentum follow DINOv2's default teacher setting (EMA momentum on a cosine schedule from 0.994 to 1, with teacher temperature warmed up linearly from 0.04 to 0.07), fixed before any of these runs is evaluated, with everything else unchanged. No teacher-setting choice then uses the JetClass test split.

The target-label-free protocol. RINO's training is target-label-free: no JetClass label enters pretraining or finetuning. We agree that JetClass labels should not also have been used to choose the teacher settings (A.17); with DINOv2's default teacher setting they are not. The caption of the deployment-workflow figure (Fig. 7 in the submission, Fig. 8 in the revised PDF) will say that "target-label-free" refers to the training stages.

The headline metric, recomputed. Table 1 uses one pretrained backbone per SSL method; its finetuned-head ±\pm covers only that backbone's 10 finetuning seeds (Sec. 4.1). The table below gives Table 1's MLP-head accuracy (finetuned on JetNet, evaluated on JetClass top vs. QCD) over repeated, independent pretraining runs for RINO and five baselines (at least three runs per method; at least five finetuning seeds per run). Each entry is the mean over pretraining runs. The first ±\pm is the standard deviation between pretraining runs, and the second is the spread between finetuning seeds within a run (the standard deviation over each run's finetuning seeds, combined over the runs as a root mean square).

MethodJetClass top vs. QCD accuracy (±\pm pretraining ±\pm finetuning)
JetCLR0.719±0.125±0.0260.719 \pm 0.125 \pm 0.026
OmniJet-α\alpha0.810±0.017±0.0070.810 \pm 0.017 \pm 0.007
MPMv10.777±0.073±0.0390.777 \pm 0.073 \pm 0.039
MPMv20.792±0.036±0.0230.792 \pm 0.036 \pm 0.023
JetCLR-scale0.812±0.021±0.0180.812 \pm 0.021 \pm 0.018
RINO (DINOv2's default teacher setting)0.843±0.021±0.0090.843 \pm 0.021 \pm 0.009
RINO (tuned setting, A.17)0.840±0.015±0.0040.840 \pm 0.015 \pm 0.004

Two observations support the statement above:

  1. The two teacher settings agree: their means differ by 0.3 percentage points, well within the run-to-run spread of either (1.5 to 2.1 percentage points).
  2. The printed 0.847 (Table 1) lies within one standard deviation of the tuned setting's multi-run mean.

With either teacher setting and the MLP head, RINO remains above every baseline's multi-run mean.

The revised PDF will update Tables 1 and 2 in full over independent pretraining runs.

AuthorsOpenReview ↗

Reply to Reviewer 1hwG (Part 3 of 3)

4. Equal Tuning Access

What happens when every baseline receives the same target-validation access and hyperparameter-search budget as RINO?

An equal-budget search for every baseline does not fit this discussion window, so we answer from two sides.

RINO without the selection. With DINOv2's default teacher setting, RINO's teacher settings involve no choice on JetClass, and RINO remains above every baseline's multi-run mean with the MLP head (3 above).

Baselines with the selection. The appendix already reports checks of this kind, which also serve as extra information for future work. The finetuning weight decay and dropout of the supervised model, MPMv1, MPMv2 and RINO are varied in A.7 (Table 10): Table 1 uses the shared default, and the "Best" column is selected on JetClass accuracy. In Table 10, RINO's worst setting (0.843) exceeds the best setting of the other three methods (MPMv2, 0.810). One choice does enter Table 1: OmniJet-α\alpha is read out by mean pooling over all tokens, which outperforms the last-token alternative (A.17). This is in line with the default classification head of the OmniJet-α\alpha code, which also pools over all tokens (by a sum). We also vary a main pretraining hyperparameter of OmniJet-α\alpha: tokenizer codebooks of 256, 2,048 and 8,192 codes (8,192 is the codebook size of the original OmniJet-α\alpha model) yield 0.8220.822, 0.8190.819 and 0.8180.818 (one pretraining run each). These are inside the range of the individual runs with our 512-code tokenizer (0.7880.788 to 0.8290.829; mean 0.810±0.0170.810 \pm 0.017, table in 3), so we see no dependence on the codebook size, and all stay below RINO's mean with either teacher setting.

5. Matched-Count and Information-Retention Controls

Can matched-count or information-retention controls further distinguish energy-ordered physical structure from differences in what each clustering algorithm preserves?

Information retention. We run an information-retention control study with the classifier of the new experiment in our reply to Reviewer ANRi (Section 1.2), with the same network and training settings: a small permutation-invariant network that sees one view alone, from each subjet's transverse momentum fraction and direction. For each clustering and each NN, it separates top from QCD jets (JetClass test jets with at least 17 constituents, 100k per class; test AUC over three seeds, standard deviation at most 0.0060.006). The clustering only builds the pretraining views, so the control is on those views.

Subjets NN2346816
kTk_\mathrm{T}0.9140.9630.9610.9590.9610.963
Cambridge/Aachen0.7740.8830.9270.9550.9610.960
anti-kTk_\mathrm{T}0.6390.7380.8040.8710.9030.946

(All particles: 0.969.) The clusterings differ in what their coarse views preserve: at two to four subjets, kTk_\mathrm{T} views carry more of what separates top from QCD jets than Cambridge/Aachen or anti-kTk_\mathrm{T} views. Cambridge/Aachen views catch up from N=6N = 6 on, and anti-kTk_\mathrm{T} views only approach them at N=16N = 16. Two subjets cannot resolve the three prongs of a top-quark decay, which likely explains why N=2N = 2 is lower even for kTk_\mathrm{T}. However, the kTk_\mathrm{T} algorithm still outperforms the other two at N=2N = 2.

Matched count. The views are compared at the same NN throughout. In about 90% of jets with more than 16 constituents, all three algorithms keep every particle. We repeat the control on these jets alone:

Subjets NN2346816
kTk_\mathrm{T}0.9150.9620.9600.9590.9590.963
Cambridge/Aachen0.7890.8920.9310.9570.9630.964
anti-kTk_\mathrm{T}0.6510.7480.8110.8780.9060.946

Every entry is within 0.020.02 of the table above, so the differences are in how the particles are grouped, not in which are kept.

What this shows. The order at coarse NN is the order of the transfer accuracies in Sec. 4.5 (0.8390.839, 0.7960.796, and 0.6930.693 for kTk_\mathrm{T}, Cambridge/Aachen, and anti-kTk_\mathrm{T} respectively; for kTk_\mathrm{T}, 0.843±0.0210.843 \pm 0.021 across multi-seed pretraining runs with DINOv2's default teacher setting, now in the revised PDF), and the experiment in our reply to Reviewer ANRi (Section 1.2) finds the same clustering to carry the least trace of hadronization at coarse NN. Therefore, the control does distinguish the energy-ordered clustering: its coarse views keep most of the top vs. QCD separation and carry the least hadronization and detector detail.

We would like to thank the reviewer again for the constructive questions, which help make the protocol and its limits explicit.

Reviewer_YdPpOpenReview ↗
Rating
3(+1)
Confidence
4
Informativeness
2
Presentation
2
Soundness
2
Read review
Paper summary

This paper proposes RINO, a physics-informed self-supervised learning framework for jet representation learning. RINO constructs multi-scale views of particle jets using k_T clustering and aligns them through DINO-style teacher-student self-distillation, combined with iBOT and KoLeo regularization. The model is pretrained on QCD jets from JetClass, finetuned on JetNet, and evaluated on out-of-distribution jet classification tasks. Experiments show improvements in OOD top tagging, label efficiency, and generalization to unseen Higgs decay topologies. The paper also presents ablation studies and representation analyses to investigate the contributions of multi-scale views and self-distillation.

Critical strengths

S1. The proposed multi-scale view construction is well motivated. RINO exploits the hierarchical structure of QCD jet formation and uses k_T clustering to construct physically meaningful views at different resolutions. This provides a reasonable alternative to conventional data augmentations and introduces a useful inductive bias for self-supervised jet representation learning.

S2. The experimental results are encouraging across different evaluation settings. RINO achieves the highest reported OOD top-tagging accuracy in Table 1, with improvements over existing SSL baselines. The data-efficiency results in Figure 2 and the cross-topology evaluations in Table 2 further demonstrate its potential for learning transferable representations.

Critical weaknesses

W1. The use of target-domain labels for hyperparameter selection raises concerns about the fairness of the OOD evaluation. The paper emphasizes transfer without target-domain labels in Sections 3.1 and 4.1. However, Appendix A.1 states that the final teacher schedule and temperature were selected using OOD accuracy on JetClass. Appendix A.17 further indicates that RINO's teacher hyperparameters were selected using target-domain accuracy, while baseline hyperparameters were inherited from their original implementations. Although the untuned RINO model already outperforms the baselines, this selection procedure weakens the claim of strictly label-free target-domain evaluation and introduces a potential tuning advantage. Reporting the main results with hyperparameters selected without target-domain labels would make the comparison more convincing.

W2. The comparison between self-distillation and contrastive learning does not fully isolate the contribution of the training objective. In Section 5.1, the performance gap between RINO and JetCLR-scale is used to support the advantage of self-distillation. However, Appendix A.17 shows that JetCLR-scale uses two sampled scale views with an auxiliary MAE reconstruction objective, while RINO uses multi-view DINO alignment together with iBOT and KoLeo regularization. Therefore, the two methods differ in several aspects beyond the alignment objective. A more controlled comparison using identical views, view sampling, and auxiliary objectives would help determine whether the improvement comes specifically from self-distillation.

W3. The explanation that scale-aware representations improve OOD generalization is not sufficiently supported. Section 5.1 and Appendix A.9 show that RINO preserves scale-dependent structure in its backbone, while JetCLR-scale produces more scale-invariant representations. The authors associate this difference with the 8.6% OOD accuracy gap. However, the observed cosine-similarity patterns do not establish that preserving scale information is responsible for the performance improvement. The methods also differ in their training objectives, making this interpretation difficult to isolate. Additional experiments that explicitly control the degree of scale invariance would provide stronger evidence for this claim.

W4. The contribution of the physics-informed scale hierarchy is not fully isolated. Section 3.2 constructs multi-scale views using fixed subjet multiplicities rather than explicit physical energy scales. Since the same number of subjets may correspond to different physical resolutions across jets, it is unclear how much of the improvement comes from the proposed physics-informed hierarchy rather than hierarchical coarse-graining more generally. Although the clustering comparison in Section 4.5 supports the choice of k_T, an additional comparison using fixed physical resolution thresholds would help clarify this distinction.

Questions for authors

Q1. How were RINO's teacher hyperparameters selected using JetClass OOD accuracy? Could the authors report the main results using a selection procedure that does not access target-domain labels?

Q2. Can the authors provide a more controlled comparison between self-distillation and contrastive learning, using identical view sampling and auxiliary objectives, to better isolate the contribution of DINO-style alignment?

Q3. Is there direct evidence that preserving scale-dependent information in the backbone leads to better OOD generalization? Could the authors explicitly control cross-scale invariance and evaluate its impact on downstream performance?

Q4. How sensitive is RINO to the definition of clustering scales? Would using fixed physical k_T thresholds instead of subjet multiplicities produce similar improvements?

5 comments
AuthorsOpenReview ↗

Reply to Reviewer YdPp (Part 1 of 3)

We thank the reviewer for the detailed review and constructive feedback, and for recognizing the multi-scale view construction as well motivated. We have run several new experiments to address the weaknesses the reviewer raises, and respond to each below. Notably, RINO retrained with DINOv2's default teacher setting [1], i.e., without the teacher-setting selection on JetClass, achieves an accuracy of 0.843±0.0210.843 \pm 0.021 over three independent pretraining runs, still above every baseline with the same MLP head. We share the results that are complete first, and will follow up in this thread as the remaining runs finish.

1. Teacher Settings Selected with Target-Domain Labels

Appendix A.1 states that the final teacher schedule and temperature were selected using OOD accuracy on JetClass. [...] Reporting the main results with hyperparameters selected without target-domain labels would make the comparison more convincing.

Removing the selection. Following this suggestion, the main results no longer use any teacher setting selected on JetClass. To show that RINO does not need this tuning, we retrain it with DINOv2's default teacher setting (EMA momentum on a cosine schedule from 0.994 to 1, with teacher temperature warmed up linearly from 0.04 to 0.07), fixed before any of these runs is evaluated. With it, RINO achieves an accuracy of 0.843±0.0210.843 \pm 0.021 (pretraining) ±0.009\pm 0.009 (finetuning) on JetClass top vs. QCD over three independent pretraining runs, above every re-run baseline's multi-run mean with the MLP head architecture.

Detailed results. We run every method over at least three independent pretraining runs. Each pretraining run is finetuned with at least five seeds, and its accuracy is the average over them. Each entry below is the mean of these per-run accuracies, with two uncertainties:

  • The first ±\pm is the spread between pretraining runs: the standard deviation of the per-run accuracies.
  • The second ±\pm is the spread between finetuning seeds within one pretraining run: the standard deviation over finetuning seeds for each run, combined over the pretraining runs as a root mean square.

Every row is finetuned on JetNet with an MLP head and evaluated on JetClass in the same way.

MethodJetClass top vs. QCD accuracy (±\pm pretraining ±\pm finetuning)
JetCLR0.719±0.125±0.0260.719 \pm 0.125 \pm 0.026
OmniJet-α\alpha0.810±0.017±0.0070.810 \pm 0.017 \pm 0.007
MPMv10.777±0.073±0.0390.777 \pm 0.073 \pm 0.039
MPMv20.792±0.036±0.0230.792 \pm 0.036 \pm 0.023
JetCLR-scale0.812±0.021±0.0180.812 \pm 0.021 \pm 0.018
RINO (DINOv2's default teacher setting)0.843±0.021±0.0090.843 \pm 0.021 \pm 0.009
RINO (tuned setting, A.17)0.840±0.015±0.0040.840 \pm 0.015 \pm 0.004

We draw two conclusions:

  • With the MLP head, RINO remains the best method. With either teacher setting it is about 3 percentage points above the best baseline (0.8430.843 and 0.8400.840 against 0.8120.812), and it is among the most stable between finetuning seeds (0.0090.009 and 0.0040.004, against 0.0070.007 to 0.0390.039 for the baselines).

  • The teacher-setting tuning is not necessary. RINO with DINOv2's default teacher setting and with the tuned setting agree within their uncertainties, 0.843±0.0210.843 \pm 0.021 (pretraining) ±0.009\pm 0.009 (finetuning) against 0.840±0.0150.840 \pm 0.015 (pretraining) ±0.004\pm 0.004 (finetuning), so we argue that JetClass labels are not needed to select the teacher settings. The printed 0.847 (Table 1) lies within one standard deviation of the tuned setting's mean.

The revised PDF will update Tables 1 and 2 in full over independent pretraining runs.


[1] M. Oquab, et al. DINOv2: Learning Robust Visual Features without Supervision. Trans. Mach. Learn. Res., 2024. arXiv:2304.07193.

AuthorsOpenReview ↗

Reply to Reviewer YdPp (Part 2 of 3)

2. Self-Distillation Versus Contrastive Alignment

A more controlled comparison using identical views, view sampling, and auxiliary objectives would help determine whether the improvement comes specifically from self-distillation.

JetCLR-scale in the paper. JetCLR-scale was designed to make it easier for the model to learn invariance: it keeps JetCLR's two-view contrastive design and replaces the augmented pair with two kTk_\mathrm{T} views of the same jet, drawn from two of three scale tiers (coarse {2,3,4}\lbrace 2, 3, 4 \rbrace, intermediate {6,8}\lbrace 6, 8 \rbrace, fine {16,uncl.}\lbrace 16, \mathrm{uncl.} \rbrace) so that the pair always spans different scales (A.17). Nevertheless, we agree that the paper's comparison in Sec. 4.5 is not fully controlled.

A matched control. Following the reviewer's suggestion, we also train JetCLR with exactly RINO's views and KoLeo regularization for a fairer comparison. The run uses the same kTk_\mathrm{T} views and view sampling as RINO (teacher scales {6,8,16}\lbrace 6, 8, 16 \rbrace plus three student-only views sampled per batch from {2,3,4,uncl.}\lbrace 2, 3, 4, \mathrm{uncl.} \rbrace), the same backbone and projection-head architecture, optimizer, learning-rate schedule, batch size, number of epochs and training jets, with the KoLeo loss added as a regularization term. Since JetCLR has no teacher (unlike DINO or iBOT), a masked-particle reconstruction term with iBOT's mask ratio of 25% is used to replace the iBOT objective. Both rows are finetuned on JetNet and evaluated on JetClass in the same way (entries as in 1 above).

PretrainingJetClass top vs. QCD accuracy
RINO (DINOv2's default teacher setting)0.843±0.0210.843 \pm 0.021 (pretraining) ±0.009\pm 0.009 (finetuning)
JetCLR with RINO's views0.808±0.0110.808 \pm 0.011 (one run; ±\pm over finetuning seeds)

RINO remains higher, by 3.5 percentage points, with the views and the view sampling matched and the KoLeo loss added. This supports the paper's reading that the gap reflects the training objective and not only the views (Sec. 4.5). The control is one pretraining run, and the revised PDF will report it with that scope.

3. Scale Structure and OOD Generalization

Is there direct evidence that preserving scale-dependent information in the backbone leads to better OOD generalization? Could the authors explicitly control cross-scale invariance and evaluate its impact on downstream performance?

To answer this directly, we are running exactly the control the reviewer suggests: RINO's recipe with one added term, a penalty that pushes the backbone's representation to be the same across kTk_\mathrm{T} clustering scales, with everything else unchanged. We will follow up in this thread with its cross-scale cosines and its OOD accuracy.

AuthorsOpenReview ↗

Reply to Reviewer YdPp (Part 3 of 3)

4. The Physics-Informed Hierarchy, and Fixed Physical Thresholds

How sensitive is RINO to the definition of clustering scales? Would using fixed physical k_T thresholds instead of subjet multiplicities produce similar improvements?

We answer from two perspectives: the physical scales of the fixed-NN views, which can be read directly from the views, and RINO trained with views at a fixed resolution threshold ycuty_\mathrm{cut}.

The physical scales of the fixed-NN views. Each NN-subjet view stops the jet's exclusive kTk_\mathrm{T} clustering at its own merge scale qNq_N, corresponding to a resolution yN=qN2/pT,jet2y_N = q_N^2 / p_{\mathrm{T,jet}}^2. This is the ycuty_\mathrm{cut} variable of kTk_\mathrm{T} clustering [2], and we follow its definition for subjets in hadron-collider jets [3], with the merge scale as defined in FastJet [4]. The median [interquartile range] of qNq_N and yNy_N over JetClass QCD jets with more than 16 constituents is (Table 8 in A.5; the revised PDF will add the yNy_N column):

NNqNq_N [GeV]yNy_N
29.91 [6.07, 17.09]2.6 [0.96,7.6]×10−42.6~[0.96, 7.6] \times 10^{-4}
35.69 [3.69, 9.01]8.5 [3.6,21]×10−58.5~[3.6, 21] \times 10^{-5}
43.89 [2.57, 5.99]4.0 [1.7,9.4]×10−54.0~[1.7, 9.4] \times 10^{-5}
62.25 [1.48, 3.39]1.3 [0.56,3.0]×10−51.3~[0.56, 3.0] \times 10^{-5}
81.49 [0.95, 2.26]5.8 [2.4,13]×10−65.8~[2.4, 13] \times 10^{-6}
160.46 [0.24, 0.76]5.5 [1.4,15]×10−75.5~[1.4, 15] \times 10^{-7}

The fixed-NN ladder is therefore a ladder of physical scales, from about 10 GeV at N=2N = 2 to below 1 GeV at N=16N = 16, and it keeps this order within each jet: qNq_N decreases strictly from N=2N = 2 to N=16N = 16 for 99.63% of these QCD jets and 99.78% of top jets. The reviewer is right that one NN still means different resolutions in different jets, as the interquartile ranges show; the second perspective tests whether this matters.

Training with a fixed resolution threshold. We train RINO with views defined at a fixed ycuty_\mathrm{cut} instead of a fixed NN. This run yields 0.837±0.0080.837 \pm 0.008 (±\pm over finetuning seeds), in agreement with the fixed-NN views (e.g., 0.843±0.0210.843 \pm 0.021 over three independent pretraining runs with DINOv2's default teacher setting; table in 1 above), so a fixed resolution threshold gives a similar improvement.

We would like to thank the reviewer again for the constructive questions, each of which leads to a new run or a more precise claim.


[2] S. Catani, et al. New clustering algorithm for multijet cross sections in e+e−e^+e^- annihilation. Phys. Lett. B, 269:432, 1991.

[3] J. R. Forshaw and M. H. Seymour. Subjet rates in hadron collider jets. JHEP, 9909:009, 1999. arXiv:hep-ph/9908307.

[4] M. Cacciari, et al. FastJet user manual. Eur. Phys. J. C, 72:1896, 2012. arXiv:1111.6097.

Reviewer_ANRiOpenReview ↗
Rating
2
Confidence
3
Informativeness
2
Presentation
2
Soundness
2
Read review
Paper summary

This paper proposes a self-supervised learning framework for jet representations that draws on the renormalization-group structure of jet formation. Different jet-clustering scenarios are treated as different views of the same jet, and the model is trained with a DINO-style self-distillation objective so that representations are consistent across these views. The method is evaluated on the JetNet benchmark and compared with existing self-supervised approaches for jets.

Critical strengths

Physically motivated ideation. Using different jet-clustering histories as views ties the augmentations to the multi-scale structure of jet formation rather than to generic perturbations, which is a natural and principled choice for jet SSL.

Critical weaknesses

Weak connection to RG flow. The renormalization-group framing is mostly motivational. The paper does not show that the clustering-based views implement an RG transformation in any formal sense, such as a fixed point, scale invariance, or a flow that relevant quantities follow.

Questions for authors

In Table 1, why is the “supervised” performance so weak? What underlying model architecture is used here?

3 comments
AuthorsOpenReview ↗

Reply to Reviewer ANRi (Part 1 of 3)

We thank the reviewer for recognizing the physical motivation of the clustering-history views as "a natural and principled choice for jet SSL." We would like to first address the connection to the RG flow, and then the supervised model in Table 1.

1. The Connection to the RG Flow

The paper does not show that the clustering-based views implement an RG transformation in any formal sense, such as a fixed point, scale invariance, or a flow that relevant quantities follow.

We agree that the submission calls the views "effectively the RG flow" and "a heuristic analogy" without saying what is formal:

  • A fixed point: no. We do not claim any fixed points for the views or for the network.
  • Scale invariance: no. RINO does not force its backbone representation to be scale invariant; indeed, it keeps scale-dependent structure (Sec. 5.1, A.9).
  • A flow that relevant quantities follow: yes, in the sense of 1.1 below.

Below we state which RG properties the kTk_\mathrm{T} views have (1.1), and add an experiment that measures what each view keeps and discards (1.2).

1.1 The RG Properties of the kTk_\mathrm{T} Views

  1. The views come from an iterative coarse-graining whose resolution is a momentum scale. Exclusive kTk_\mathrm{T} clustering repeatedly merges the pair of objects with the smallest kTk_\mathrm{T} distance (Eq. 1). The view at resolution qq is where the smallest distance first exceeds q2q^2. Continuing from NN subjets to M<NM < N gives the same view as clustering directly to MM. What is specific to kTk_\mathrm{T} is that the stopping variable is a momentum scale: each view has its own merge scale qNq_N, with medians from 9.9 GeV at N=2N = 2 down to 0.46 GeV at N=16N = 16 (A.5, Table 8). For simplicity, we define views using the number of objects NN in the clustering history. Nevertheless, views defined at a fixed resolution ycut=q2/pT,jet2y_\mathrm{cut} = q^2 / p_{\mathrm{T,jet}}^2 give a consistent result: one fixed-resolution run achieves an accuracy of 0.837±0.0080.837 \pm 0.008 (±\pm over finetuning seeds), within the run-to-run spread of the fixed-NN views (0.843±0.0210.843 \pm 0.021 over three independent pretraining runs with DINOv2's default teacher setting [8]).

  2. In the soft-collinear limit, the scale is the evolution scale of the parton shower, which is an RG evolution. The shower evolves a jet from the hard scale down to a cutoff of O(1 GeV)\mathcal{O}(1~\mathrm{GeV}) [3] by solving DGLAP-type evolution equations [1, 2]. There, the merge scale q=dijq = \sqrt{d_{ij}} (Eq. 1) is the relative transverse momentum of the splitting divided by RR: approximately the variable that orders the Pythia shower and sets its running coupling [3]. The kTk_\mathrm{T} clustering history therefore serves as an approximate branching history: CKKW matching [4] takes the scales of its running couplings and Sudakov factors from kTk_\mathrm{T} clustering histories.

  3. Quantities defined on the views follow evolution equations in that scale. At perturbative resolutions, the kTk_\mathrm{T} jet rates and the mean number of kTk_\mathrm{T} jets [5, 6], and the kTk_\mathrm{T} subjet rates inside hadron-collider jets [7], depend calculably on the resolution parameter, and coherent-branching evolution equations resum the corresponding logarithms. The finest views reach the O(1 GeV)\mathcal{O}(1~\mathrm{GeV}) end of the shower, and these are the views in which the experiment of 1.2 below finds the strongest trace of hadronization.


[1] Yu. L. Dokshitzer, et al. Basics of Perturbative QCD. Éditions Frontières, 1991.

[2] Z. Nagy and D. E. Soper. What is a parton shower? Phys. Rev. D, 98:014034, 2018. arXiv:1705.08093.

[3] C. Bierlich, et al. A comprehensive guide to the physics and usage of PYTHIA 8.3. SciPost Phys. Codebases, 8, 2022. arXiv:2203.11601.

[4] S. Catani, et al. QCD matrix elements + parton showers. JHEP, 0111:063, 2001. arXiv:hep-ph/0109231.

[5] S. Catani, et al. New clustering algorithm for multijet cross sections in e+e−e^+e^- annihilation. Phys. Lett. B, 269:432, 1991.

[6] S. Catani, et al. Average number of jets in e+e−e^+e^- annihilation. Nucl. Phys. B, 377:445, 1992.

[7] J. R. Forshaw and M. H. Seymour. Subjet rates in hadron collider jets. JHEP, 9909:009, 1999. arXiv:hep-ph/9908307.

[8] M. Oquab, et al. DINOv2: Learning Robust Visual Features without Supervision. Trans. Mach. Learn. Res., 2024. arXiv:2304.07193.

AuthorsOpenReview ↗

Reply to Reviewer ANRi (Part 2 of 3)

1. The Connection to the RG Flow (Continued)

1.2 A New Experiment: What the Views Keep and What They Discard

An RG-like coarse-graining at resolution qq should keep structure above qq and lose sensitivity to physics below it. This also matters experimentally: hadronization (at a scale of 1 GeV) and the detector response are leading sources of differences between generators, and between simulation and data. In simulation, we can change this small-scale physics and check whether a view still notices.

Setup

  • Samples. We regenerate 195k QCD jets with the JetClass recipe (MadGraph, Pythia 8 and Delphes with the JetClass cards and selection; 500<pT<1000500 < p_\mathrm{T} < 1000 GeV). We keep every jet at three stages: before hadronization, after hadronization, and after the detector simulation.
  • Test. For each view NN, a small permutation-invariant network tries to tell two samples apart using that view alone, from each subjet's transverse momentum fraction and direction.
  • Measure. We report the area under the ROC curve (AUC). An AUC of 0.5 means the classifier finds no difference, and 1.0 means it separates the samples perfectly. AUCs are averaged over 3 seeds (std ≤0.007\le 0.007), on jets with at least 17 particles so that every view is complete.

Results. Top vs. QCD uses JetClass test jets with the same classifier. The other two tests compare two stages of the same regenerated jets.

AUC ofclusteringN=2N = 2346816all particles
top vs. QCDkTk_\mathrm{T} (RINO)0.910.960.960.960.960.960.97
Cambridge/Aachen0.770.880.930.950.960.960.97
anti-kTk_\mathrm{T}0.640.740.800.870.900.950.97
before vs. after hadronizationkTk_\mathrm{T} (RINO)0.510.530.560.620.670.831.00*
Cambridge/Aachen0.710.790.840.920.961.001.00*
anti-kTk_\mathrm{T}0.900.960.980.991.001.001.00*
hadron vs. detector levelkTk_\mathrm{T} (RINO)0.560.590.610.640.680.811.00*
Cambridge/Aachen0.820.910.950.991.001.001.00*
anti-kTk_\mathrm{T}0.940.980.991.001.001.001.00*

*Trivial: the number of particles differs between the two stages.

There are two main observations.

  1. For the kTk_\mathrm{T} algorithm, top vs. QCD is already close to the unclustered jet from N=3N = 3 (0.96 against 0.97), while hadronization becomes essentially undetectable to this probe at coarse NN (0.51 at N=2N = 2), and much of the detector-level effect with it (0.56). Even at N=16N = 16 both remain only partly visible (0.83 and 0.81). In comparison, the other two clusterings do worse on both measures: their coarse views keep less of the top vs. QCD separation (0.77 and 0.64 at N=2N = 2) and much more of hadronization and the detector (0.71 to 0.94 at N=2N = 2), and both effects are fully separable by N=16N = 16.

  2. The kTk_\mathrm{T} algorithm merges the pair with the smallest relative transverse momentum first (in the soft-collinear limit), so stopping at NN subjets means stopping at a momentum scale (item 2 in 1.1 above). Cambridge/Aachen merges by angle and anti-kTk_\mathrm{T} around the hardest particles, so their coarse views often end with single soft particles, whose momenta hadronization and the detector set. Importantly, the two controlled steps and the transfer result agree: the views that carry the least hadronization and detector detail are the views that transfer best. The same ordering appears in the transfer accuracies of Table 12 (Sec. 4.5): RINO pretrained on kTk_\mathrm{T}, Cambridge/Aachen and anti-kTk_\mathrm{T} views reaches 0.839±0.0050.839 \pm 0.005 (0.843±0.0210.843 \pm 0.021 across multi-seed pretraining runs with DINOv2's default teacher setting, now in the revised PDF), 0.796±0.0120.796 \pm 0.012 and 0.693±0.0260.693 \pm 0.026. The detector step probes one component of the JetNet (simplified detector description) to JetClass (Delphes) shift.

In the revised PDF, we will state the connection to the RG flow more precisely, along the lines of 1.1, and add this experiment in a new appendix section.

AuthorsOpenReview ↗

Reply to Reviewer ANRi (Part 3 of 3)

2. The Supervised Model in Table 1

In Table 1, why is the "supervised" performance so weak? What underlying model architecture is used here?

Architecture. The supervised model uses the jet transformer encoder that every encoder-based method in Table 1 shares, trained from scratch on JetNet labels: 88 layers, hidden dimension 256256, 1616 attention heads, feed-forward dimension 10241024, SwiGLU, pre-residual RMSNorm and LayerScale, 8.48.4M trainable parameters, and the same seven per-particle input features (A.3, Table 6; Table 19). None of the five weight-decay and dropout settings in A.7 improves on its default (Table 10).

Why it is weak. The gap is an out-of-distribution (OOD) effect, from finetuning to deployment: every model in Table 1 is trained or finetuned on JetNet labels and deployed on JetClass, a different simulation.

  • In distribution, there is no gap. On held-out JetNet jets the supervised model reaches 0.9060.906 at 100% of the labels, against RINO's 0.9090.909, so we claim no in-distribution advantage.
  • It is not classical overfitting. Its held-out JetNet accuracy stays flat or rises with the label fraction (A.14). This suggests that the supervised model specializes to JetNet-specific features that do not transfer well to top tagging on JetClass.
  • A difference in training data. All the SSL rows also pretrain on unlabeled JetClass QCD jets (Sec. 5.3), while the supervised model receives no such pretraining by definition.

We would like to thank the reviewer again for the constructive questions, which help make the RG link precise.

Comments on the paper

3 comments

Dear authors and reviewers,

The author-reviewer discussion window will close tomorrow on November 18.

@authors: If not done already, please respond to the reviewers' comments before the discussion window closes. I would advise short and concise responses over lengthy explanations.

@reviewers: Please check the authors' responses and provide any additional comments before the discussion window closes. Consider updating your initial review if necessary, and in particular your score if your opinion has changed. You may increase your score if you feel the authors' responses warrant it, maintain your score if your opinion remains the same, or even decrease your score if you feel the other reviews or the authors' responses reveal new shortcomings. At the very least, please acknowledge the authors' responses.

Thank you for your attention and cooperation.

The AC

AuthorsOpenReview ↗

Summary of Changes in Revised Version

We have uploaded a revised version of the paper in response to the reviewers' comments. Changed text is shown in blue, and the main text is now 10 pages. The main revisions are the following:

  • Tables 1 and 2 (Reviewers YdPp and 1hwG): Every SSL method is reported over independent pretraining runs, with the spread between pretraining runs and between finetuning seeds given separately (Sec. 4.1). RINO in these tables uses DINOv2's default teacher setting (A.1, A.17).
  • Matched-views control (Reviewer YdPp): JetCLR trained with RINO's views and view sampling (Sec. 4.5, Fig. 3, Sec. 5.1).
  • Scale structure (Reviewer YdPp): The control with the cross-scale invariance term (Sec. 5.1, A.9, new Fig. 7). The scale structure is stated as an observation about the trained models, not as the cause of the accuracy gap between RINO and JetCLR-scale.
  • RG connection (Reviewers ANRi and 1hwG): Revised wording (Sec. 1, Fig. 1, Sec. 2) and a new simulation study of what the kTk_\mathrm{T} views keep and discard (A.15, Table 17, Fig. 10).
  • Physical scales of the kTk_\mathrm{T} views (Reviewer YdPp): Table 8 adds the dimensionless resolution yNy_N of each view (A.5).
  • OOD protocol (Reviewer 1hwG): The access to unlabeled target-domain QCD jets in pretraining is stated in Sec. 1 and Sec. 4.1; A.4 lists the data partitions; and the caption of the deployment-workflow figure (now Fig. 8) states explicitly where the training labels and the JetClass labels enter.

The abstract, Secs. 4.2–5.1, the label-efficiency figures (Figs. 2 and 9), the Limitations, and the Conclusion are updated to these results. The AI use statement has a paragraph on the discussion phase.

We thank the reviewers and the Area Chair for their time and constructive feedback.

AuthorsOpenReview ↗

Final Remarks

Dear Reviewers and Area Chair,

We thank all the reviewers for their thoughtful comments. We appreciate the positive feedback that our multi-scale view construction is "well motivated" (YdPp), "intuitive" (1hwG), and "a natural and principled choice for jet SSL" (ANRi), and that it "introduces a useful inductive bias for self-supervised jet representation learning" (YdPp). We are also pleased to hear that the "experimental results are encouraging across different evaluation settings" (YdPp) and that the paper "includes useful controlled comparisons" (1hwG).

The reviewers' constructive suggestions have strengthened the paper. We addressed them in our individual replies and in the revised manuscript, where the changes are shown in blue. In summary:

  • We now report every SSL method in Tables 1 and 2 over independent pretraining runs, with the spread between pretraining runs and between finetuning seeds given separately (Sec. 4.1). RINO in these tables uses DINOv2's default teacher setting, so no teacher setting is selected on JetClass (A.1 and A.17), and its multi-seed mean accuracy remains the highest on out-of-distribution (OOD) top tagging. In addition, we argue that tuning this setting on JetClass is not needed, since a tuned setting gives the same accuracy within uncertainties (A.17).
  • We add a matched control in which JetCLR, trained with exactly RINO's views and view sampling, remains below RINO (Sec. 4.5 and Fig. 3). This suggests that the training objective also contributes to RINO's gain, although the larger improvement comes from the multi-scale kTk_\mathrm{T} views (Sec. 5.1).
  • We add a control that makes the backbone nearly scale-invariant, and find that the OOD accuracy stays within the spread between pretraining runs (Sec. 5.1, A.9, and Fig. 7). We therefore report the scale structure as an observation, not as the cause of the accuracy gap.
  • We state the connection to the RG flow more precisely (Sec. 1 and Sec. 2). We also add a simulation study of what the views keep and discard, and find that the coarse kTk_\mathrm{T} views keep most of the top vs. QCD separation while being largely insensitive to hadronization and detector effects, more so than Cambridge/Aachen and anti-kTk_\mathrm{T} views (A.15, Table 17, and Fig. 10). A matched-count control shows that the differences in top vs. QCD separation between the clusterings remain when all three keep every particle, so they come from how the particles are grouped, not from which are kept (our reply to Reviewer 1hwG).
  • We add the resolution yNy_N of each view to Table 8, and find that views at a fixed ycuty_\mathrm{cut} reach an accuracy in agreement with the fixed-NN views (our reply to Reviewer YdPp).
  • We qualify the OOD results as a transfer in which pretraining has access to unlabeled target-domain QCD jets (Sec. 1 and Sec. 4.1), explicitly list the data partitions (A.4), and state that no JetClass label enters the two training stages (caption of Fig. 8).
  • We clarify that the supervised model uses the same encoder as the encoder-based SSL methods (A.3 and Table 19) and that its lower accuracy on JetClass is an OOD effect: on held-out JetNet jets, it matches RINO (our reply to Reviewer ANRi).

We hope our responses and the revised manuscript have addressed your concerns. We remain available for further questions during the remainder of the discussion period.

Best regards,

Authors

From OpenReview, CC BY 4.0. Authors as signed there; shown unedited.

Related papers

Loading the map…

Discussion (0)

Sign in to comment.