- Rating
- 2
- Confidence
- 3
- Informativeness
- 1
- Presentation
- 2
- Soundness
- 2
Read review
Paper summary
This paper introduces JEPA-Anything, a framework that reuses a common latent-state prediction procedure across independently parameterized domain models. Each domain retains its own input representation, encoder, and model parameters. Its core mechanism, Orthogonal Predictive Factorization, learns projections that transform target embeddings into coordinate blocks, predicts these blocks through a shared trunk with factor-specific output heads, and reconstructs a complete state through pseudoinverse synthesis. The training objective combines predictive regression with orthogonality and variance regularization.
Critical strengths
-
Broad empirical coverage with substantial improvements on some tasks.
-
The paper is very readable overall.
Critical weaknesses
-
Limited Novelty in the Core Method The core idea lacks significant novelty. The method mainly adds learnable projections, partitioned outputs, and regularization to an existing JEPA. Since the model uses a shared backbone and linear heads, these partitioned heads can simply be combined into a single dense layer. A full-rank projection does not change the predictor's capabilities, and exact orthogonality makes the prediction error equivalent to that in the original space. Therefore, this method is mostly just a reparameterization and regularization of an existing predictor, not a fundamentally new mechanism. While these changes might affect training, the paper lacks deep analysis or new insights to prove this is a strong methodological contribution.
-
Limited Contribution of the Cross-Domain Framework The contribution of the cross-domain framework is limited. Each domain still uses its own inputs, encoders, targets, and parameters; only the algorithmic steps are reused. JEPA already handles context encoding, target encoding, and latent prediction. While reusing a method across domains is reasonable, the paper does not clearly show what new technical ability this interface adds. Applying the same module to multiple domains shows it is broadly applicable, but this is not a new methodological innovation. The paper should more clearly separate the contribution of the unified framework from the specific changes made by OPF.
-
Insufficient Experimental Proof of the Mechanism The experiments do not clearly prove that the improvements come from the proposed partitioned allocation of predictive capacity. Although ablation studies are provided, the effects of projection learning, output partitioning, and regularization are not well separated. Since partitioned linear heads are mathematically similar to a single dense layer, the paper should explain more clearly how partitioning changes the training process. Adding a control experiment with a fixed random orthogonal projection from the start (while keeping the variance regularization) would be very helpful. Also, performance drops after factor masking only show that the coordinates are being used, not that predictive information is allocated better. In the CITRIS analysis, the confidence interval compared to random rotations includes zero. Thus, the evidence shows the overall training design works, but it does not strongly support the proposed explanation of why it works.
-
Theoretical Analysis Does Not Fully Explain the Improvements The theoretical analysis does not fully explain the predictive improvements. The results on orthogonality mainly limit coordinate-error amplification. They do not analyze how the encoders, predictors, and projections optimize together, or how errors accumulate during training. The post-hoc QR experiment shows that conditioning can improve without changing the prediction function, and the sensitivity analysis shows the lowest error actually occurs without orthogonality regularization. This does not mean orthogonality is useless, but it shows that better geometric structure does not always lead to better prediction. The paper needs to clearly explain when these benefits happen and how better geometry leads to better task performance during training.
Questions for authors
Please refer to Weaknesses.