- Rating
- 2
- Confidence
- 2
- Informativeness
- 2
- Presentation
- 3
- Soundness
- 2
Read review
Paper summary
This paper investigates whether Joint-Embedding Predictive Architectures (JEPAs) can serve as efficient and reliable surrogate models for physical dynamical systems, particularly longitudinal particle beam dynamics. The authors focus on three questions: whether imposing symplectic structure improves prediction, how JEPA compares with reconstruction-based models in accuracy and computational cost, and how these models perform under observation noise and out-of-distribution data. The evaluation results show that imposing symplectic structure improves the geometric properties of latent transitions, but does not always improve prediction accuracy. JEPA achieves faster training and better robustness to additive Gaussian noise, while reconstruction-based models generally achieve better prediction accuracy under clean observations.
Critical strengths
-
I appreciate that the authors do not simply show that enforcing symplecticity improves prediction. Instead, they compare multiple architectures and investigate why some structure-preserving models perform better than others. For example, Figure 2 shows that HNN and SRNN satisfy the geometric constraints but still perform worse than the unconstrained baseline. I think this is an interesting observation, as it suggests that imposing the correct physical structure on the predictor is not sufficient if the learned latent coordinates are not suitable.
-
The comparison between JEPA and reconstruction-based models is interesting. Although the autoencoder achieves better accuracy on clean data, JEPA is much less affected by additive Gaussian noise. I also appreciate that the authors investigate Poisson noise and OOD data, where the conclusions are different. These experiments help explain both the advantages and limitations of the JEPA objective rather than presenting it as consistently superior.
Critical weaknesses
-
My main concern is that the paper directly imposes symplectic constraints on the 64-dimensional latent space based on the symplectic property of the original two-dimensional particle dynamics. However, it is unclear whether this property should still hold after an arbitrary nonlinear encoding of the phase-space density. Symplecticity is preserved under canonical coordinate transformations, but the CNN encoder is not constrained to learn such transformations. Therefore, even if a latent predictor satisfies the symplectic conditions in Figure 2, this does not necessarily mean that it preserves the actual physical structure of the beam dynamics. I think this is important because one of the main contributions is to examine whether the physical geometry can be restored through structural constraints. The authors should better justify the connection between the original physical symplecticity and the imposed latent constraints.
-
In Section 3, the authors assume that the beam dynamics follow a deterministic and autonomous one-turn map, where the current state uniquely determines the next state. However, Appendix B shows that the RF voltages are sampled independently over 60–100 kV across simulations. Since these machine settings affect the synchrotron dynamics, different simulations may have different transition maps even when their current beam distributions are similar. Meanwhile, the learned predictors only take the encoded density frame as input, without explicitly including the machine settings. I wonder whether this introduces ambiguity into the learning problem, since the model is effectively asked to approximate multiple dynamical systems using a single transition function. This may also affect the comparison between different predictor architectures, particularly the linear and Hamiltonian models. I think the authors should clarify this assumption and investigate whether conditioning on machine parameters changes the conclusions.
-
One concern is the choice of VISReg as the default regularizer. In Appendix C, Table 4, the authors show that VICReg performs considerably better than VISReg. For example, VICReg reduces the mean rollout MSE ratio from 0.16 to 0.08 and the final-step MSE ratio from 0.69 to 0.33. Interestingly, this result is already very close to the Linear-orth Autoencoder in Table 1, which achieves 0.07 and 0.31, respectively. However, the main comparisons are still conducted using VISReg. The authors explain their preference for VISReg, but I am not fully convinced that this is sufficient, since the paper aims to evaluate the performance trade-offs between JEPA and reconstruction-based models. I think using a weaker JEPA configuration may affect the fairness of the comparison, and the authors should at least report the main accuracy and noise-robustness results using VICReg as well.
-
The paper emphasizes long-horizon prediction as an important motivation, especially for physical simulations that need to track beam dynamics over thousands of turns. However, Section 5 states that each simulation contains only 250 turns, corresponding to about 0.88 of one nominal synchrotron period. Therefore, even the longest evaluation does not cover a complete oscillation cycle. I wonder whether the reported performance can be maintained over multiple synchrotron periods, where small errors in oscillation frequency and phase may accumulate significantly. This is particularly relevant to the comparison of symplectic and orthogonal predictors, since preserving geometric properties does not necessarily guarantee accurate long-term trajectories. I think experiments over several complete oscillation periods would make the conclusions more convincing.
Questions for authors
-
How does the imposed latent symplectic structure relate to the original physical coordinates? Can the authors provide evidence that satisfying the latent geometric conditions also improves preservation of physical quantities, such as beam emittance or synchrotron tune?
-
How are the variations in RF settings handled by the learned models? Are these settings implicitly recoverable from the input frames? If not, could the authors compare models trained under fixed machine settings or explicitly conditioned on RF parameters?
-
Why is VISReg selected as the default regularizer when VICReg performs significantly better in Appendix C? In Table 4, VICReg reduces the mean rollout MSE ratio from 0.16 to 0.08 and the turn-249 ratio from 0.69 to 0.33. I wonder whether the main conclusions about JEPA's prediction accuracy and noise robustness would change if VICReg were used in the comparison.
-
How general is the claimed noise robustness of JEPA? The Gaussian noise experiment shows a clear advantage, but Appendix D reports that under Poisson noise, the JEPA models experience a larger relative increase in first-turn prediction error than the autoencoder models. Could the authors clarify whether JEPA's advantage is mainly due to filtering noise in the empty background, rather than better robustness to noise affecting the actual beam distribution?
4 comments
Follow up on rebuttal
Dear reviewer,
We appreciate your time and constructive feedback on our submission. Since this is almost the last day of the discussion round, we wanted to follow up to see if our rebuttal addressed your concerns. Please let us know if we can provide any further clarification. Even if the rebuttal didn't address your concerns, we would still love to know your thoughts, since it will help us strengthen our paper.
I thank the authors for their rebuttal. I found some of the revisions in the updated manuscript help clarify my previous concerns. For example, the additional comparison between VISReg and VICReg helps explain the choice of regularizer: it seems that VICReg performs better for Linear-orth but does not work as well for the other predictor architectures, which makes the authors' choice more reasonable.
However, I think two issues still deserve further investigation, i.e., whether the imposed latent symplecticity can actually help preserve physical invariants, and how the prediction results would change if the model were explicitly conditioned on the varying RF settings.
That being said, I think many of my concerns have been reasonably touched. I will reconsider my rating accordingly.
Thank you!
Dear reviewer,
Thank you so much for taking the time to read our revisions and for your thoughtful follow-up. We are glad that the expanded VISReg vs. VICReg comparison and our other updates helped clarify your concerns. We appreciate your time and constructive feedback throughout the process, your engagement in the discussion, and your reconsideration of your ratings!
Yes, HNN, SRNN, and SympNets partition the 64-dimensional latent vector into canonical coordinates and enforce symplecticity on the latent transition map. However, because the convolutional encoder is unconstrained, the mapping from physical phase space to latent space is not guaranteed to be a canonical transformation. Consequently, preserving symplectic volume in latent space does not automatically guarantee conservation of physical invariants, which is precisely why we observe JEPA struggling with OOD despite low latent symplectic error. Designing structure-preserving encoders that explicitly align latent variables with physical invariants or canonical variables is an open research question beyond the scope of this study, as highlighted in the Discussion section.
We thank the reviewer for highlighting this distinction. Upstream machine conditions influence the injected distribution, while the Recycler RF settings govern its subsequent evolution. In our simulations, these settings remain constant within each run, so the dynamics are autonomous conditional on the settings. However, sampling RF voltages across runs produces a family of transition maps. Our ongoing efforts toward more complex beam scenarios, including time-dependent RF programs and slip-stacking, motivate incorporating machine settings explicitly into JEPA.
Please refer to our comment at the top for our rebuttal on the rationale for choosing VISReg over VICReg.
We originally chose the 250-turn record to match the experimental oscilloscope acquisition window. We have now checked its physical informativeness using a controlled BLonD amplitude scan. These BLonD simulations save turn-by-turn coordinates for every macroparticle. A nonlinear phase-advance diagnostic using only the first 250 coordinate states recovers individual synchrotron tunes within 0.000502% of independent long-track references through 2.5 rad amplitude, resolving the amplitude-dependent frequency variation. Thus, a complete oscillation is not required for accurate frequency identification under these conditions. This supports evaluating whether JEPA captures the relevant frequencies within the recorded window.
Please refer to our comment at the top for our rebuttal on noise modeling. You are spot-on that the difference comes down to additive Gaussian noise perturbing the empty background versus Poisson noise perturbing only the occupied beam distribution. As noted above, we have narrowed our claims across the paper to specify robustness to additive noise.