- Rating
- 3
- Confidence
- 3
- Informativeness
- 2
- Presentation
- 3
- Soundness
- 3
Read review
Paper summary
This paper improves LLM-JEPA by introducing an experience replay path. The study primarily uses Llama-3.2-1B-Instruct and evaluates the model across five datasets. Through experimental settings including matched training compute, token-matched control, and current-batch control, combined with three replay selection strategies and five random seeds, the paper analyzes error correction capability, retention of correct predictions, regex structure, and expression length. Results show that ER-JEPA outperforms LLM-JEPA in multiple aspects and reduces overfitting and accuracy degradation during training.
Critical strengths
The research question is valuable, the methodology is simple, and the experimental design is relatively solid. In particular, the paper goes beyond a direct accuracy comparison by including matched-compute, token-matched, and current-batch controls, which help support the claim that historical replay provides additional supervision beyond extra token exposure or an additional loss term.
Critical weaknesses
- The paper does not report the value of the replay budget , nor does it present results for different values of . This omission affects both reproducibility and the understanding of the replay mechanism and its training cost.
- The novelty of the method is incremental, with the main modification being the integration of experience replay into LLM-JEPA. Experience replay is a mature mechanism, and the paper does not systematically compare ER-JEPA with stronger replay, data-resampling, or anti-forgetting baselines under matched compute.
Questions for authors
- What is the value range for the replay budget ? Is there any ablation study on ?
- What is the design rationale behind Equation (18)? Intuitively, the more times a sample is replayed, the more important it may be to the model. However, under the formula, a higher replay count makes the sample more likely to be discarded. Why is this design chosen, and why not simply discard samples based on the JEPA error directly?
3 comments
Response to Reviewer Xv4h – W2 & Q2
W2. Comparisons with replay, resampling, and anti-forgetting baselines
Thank you for raising this concern. ER-JEPA is not a trivial combination of experience replay and LLM-JEPA. In LLM-JEPA, representation alignment converges early, yet the model still makes generation errors and loses previously correct predictions (Figure 3). ER-JEPA mitigates this gap between representation alignment and prediction accuracy. Please see our theoretical proof in Appendix B (Proposition B.1) and the parameter trajectories and test loss landscape in Appendix Figure 11 of the revised manuscript. We hope these analyses help clarify our mechanism. The results show that re-evaluating stored examples changes the gradient update direction and guides optimization to a lower-loss region while preserving alignment.
Moreover, ER-JEPA is neither an anti-forgetting algorithm nor a data-resampling heuristic. First, our manuscript never mentions the word "forgetting." Second, we do not compare ER-JEPA directly with continual-learning replay or anti-forgetting baselines because these methods are designed for continual learning and primarily address catastrophic forgetting. In contrast, our study investigates how to improve existing open-source LLMs' learning and understanding of core semantics while preserving their text-generation capability. These are distinct research problems, and a direct empirical comparison would therefore not be scientifically meaningful. Within our stationary setting, Section 3.3 (Figures 6 and 7) already shows that all three ER-JEPA replay variants outperform token-matched resampling and current-batch reuse under matched budgets, while Appendix Table 3 presents our stepwise module ablation, making it clear why both the token-matched control and the current-batch control are needed.
Finally, ER-JEPA generalizes across multiple models and tasks with higher accuracy and faster convergence. As shown in Figure 1, Table 1, and Section 3.2 of the revised manuscript, ER-JEPA consistently improves performance across four model families and five tasks. Under matched compute, ER-JEPA achieves higher accuracy and reaches SFT performance much faster (Figures 4 and 5).
Thank you again for your kind comments, which are valuable for improving our manuscript.
Q2. What is the design rationale behind Equation (18)? Why not simply discard samples based on the JEPA error directly?
Thank you for your insightful question and we will discuss more about it below.
We design Equation (18) to prevent already replayed or noisy samples from permanently occupying finite memory slots when the buffer reaches capacity. In , records how much replay supervision sample has already received rather than its intrinsic importance. Discounting by ensures healthy memory turnover. Once a sample has been replayed several times, its learnable signal has largely been absorbed, or its remaining error reflects noise. Lowering its retention score frees finite slots for newly encountered samples () and avoids overfitting a small set of stubborn examples. Pairing error-driven memory with decay is also a common design in modern memory architectures [1, 2]. For example, Titans [1] and Nested Learning [2] store high-surprise inputs while applying decay mechanisms so older traces gradually make room for new information.
As for discarding samples based on raw JEPA error alone, noisy or ambiguous outliers can retain a persistently high alignment error even after repeated replay updates. Without discounting by , such stubborn examples would permanently occupy finite memory slots as grows, starving newly encountered learnable samples () of replay opportunities and risking overfitting to a small subset of outliers. Depreciating by directly prevents this slot monopolization by ensuring that repeatedly replayed entries gradually yield room to fresh samples. In our main SYNTH runs, capacity () exceeds total writes (), so eviction is inactive. Under small capacities where eviction is active ( to in Appendix F.2), accuracy remains stable from to .
Thank you again for your time and valuable feedback, which has helped us improve the clarity and technical depth of our work.
References
[1] Behrouz, A., Zhong, P., and Mirrokni, V. Titans: Learning to Memorize at Test Time. NeurIPS, 2025.
[2] Behrouz, A., Razaviyayn, M., Zhong, P., and Mirrokni, V. Nested Learning: The Illusion of Deep Learning Architectures. NeurIPS, 2025.
We have highlighted the relevant revisions and the figures presenting our new results in blue in the revised manuscript. If you have any further questions, feel free to communicate with us.
Response to Reviewer Xv4h – W1 & Q1
Thank you for taking the time to review our paper and for your encouraging comments. We address your questions and comments below, and would appreciate your consideration in raising your score in light of these clarifications and new results.
W1 and Q1. Replay budget details, training cost, and ablation study on R
Thank you for these comments. We added more details about the replay budget R, its training cost, and an ablation study on R in Appendix F.2 (Figure 22 and Table 10) of the revised manuscript.
First, the per-GPU replay budget R ranges from 0 (standard LLM-JEPA without replay) up to B or 2B, where B is the fresh micro-batch size per GPU. All main experiments use R=16 examples per GPU per micro-batch, matching B=16 (R/B=1.0), with replay weight β=1.0 and memory capacity M=100,000. Early in training when the active memory holds fewer than R items, we set Rt=min(R,∣Mt∣). Response Table 1 reports the five-seed training compute and token profile at the matched compute budget (240.31 PFLOPs) on NL-RX-SYNTH with
Llama-3.2-1B-Instruct.Response Table 1. Replay budget and matched-compute training cost (240.31 PFLOPs) on NL-RX-SYNTH with
Llama-3.2-1B-Instructover five seeds.At matched PFLOPs, ER-JEPA (R=16) improves exact-match accuracy by +14.90 percentage points over LLM-JEPA (R=0). After training, we discard the episodic memory and replay pathway, so ER-JEPA adds zero inference cost.
Additionally, we conducted a controlled ablation study over R∈0,4,8,16,32 on Spider text-to-SQL generation using
Qwen3-1.7B(Appendix F.2, Figure 22 and Table 10), with a fresh micro-batch size of B=32 per GPU across 6 epochs. Response Table 2 summarizes the accuracy and training time across different values of R.Response Table 2. Ablation study on the replay budget R per GPU on Spider with
Qwen3-1.7B(B=32 per GPU, 6 epochs).As shown in Response Table 2 and Appendix F.2 (Figure 22 and Table 10), increasing the replay budget R generally improves execution accuracy while requiring more training time. A very small budget (R=4) gives limited late-stage gains due to sparse replay coverage, whereas R≥8 steadily raises accuracy and reaches the R=0 baseline level in less training time (up to a 1.42× speedup at R=32, 24.6 vs. 34.9 min). However, moving from R=16 (0.5B) to R=32 (B) brings only a modest additional accuracy gain while adding noticeably more training time (47.7 vs. 42.4 min). Therefore, setting R between 0.5B and B provides a practical balance between accuracy gains and training cost.