- Rating
- 1
- Confidence
- 3
- Informativeness
- 1
- Presentation
- 2
- Soundness
- 1
Read review
Paper summary
This paper identify a "problem" for jepa based model like LeWM. These models map a current frame embedding and action to the embedding of a future frame, and velocity information is missing in this paradigm. The authors introduce RateIdent, a three-stage diagnostic protocal to probe the problem, and TI-JEPA, a a lightweight fix that splits the latent into a pose code and an explicit finite-difference motion code, and predict both infomation together.
Critical strengths
None
Critical weaknesses
W1:(main issue) The authors claim that "single-frame targets are the wrong object to predict when velocity matters". However, if historical information is included in prediction, namely, if model have a "context window" for last few frames, velocity and other dynamics can be learned by predicting single frames. That is because these information can be encoded in context window and learned by model. As most world models, including LeWM(mainly used in paper), actually have context window, the claim is not valid. This is a fundamental flaw in the paper.
Even the authors' experiment faithfull reflect this issue. LeWM has 3 frames of history, and in table 2, baseline k = 3 have the ability to predict velocity, even better than TI-JEPA in InertiaBall and Pendulum env. Why are authors comparing with "single frame" LeWM, or memoryless LeWM? This not a practical case. And TI-JEPA also use past 3 frames information, the comparison for single frame baseline is not fair.
W2: Assume the authors' method is somehow helpful, can add some velocity information and surpass memory-having baselines. The authors claim that "TI-JEPA’s kill-experiment branch separation exceeds the memory-having baseline’s by roughly 38×". I read through chapter 7.6, and here are my questions: Why the rest of paper use official LeWM checkpoint, but here a trained-from-scratch ckpt is used? How is the performance of this trained-from-scratch ckpt on other tasks? How is the performance of offical ckpt? And branch separation is a strange metric, in my understanding, it meansures the end pos of arm. TI-JEPA achieves 2.166, but what does greater than 1 means? Does greater means better? Why for rollout position MSE, the baseline is better? All these lack further analysis.
Unfortunately, unless the authors can provide explanation and more convincing evidence, this paper do not meet the standard of acceptance.
Questions for authors
See weakness.