acceptodds
Under review as a conference paper at ICLR 2027

Learning from the Past: Experience Replay Strengthens Joint-Embedding Predictive Learning in Language Models

Abstract

Large language models (LLMs) excel at token-level generation but may learn undesirable abstract semantics and lack comprehensive perception. LLM-JEPA mitigates this by aligning different views of the same underlying knowledge via a joint-embedding predictive architecture (JEPA). However, strong alignment does not necessarily lead to accurate, stable predictions. To address this, we propose ER-JEPA, which adds an episodic replay path to LLM-JEPA. ER-JEPA stores training pairs in a memory. At each step, it stores and retrieves relevant data to provide additional supervision. This enables learning from both the current batch and stored training pairs, providing additional supervision for token prediction and representation alignment. Experiments across multiple datasets (NL-RX, GSM8K, Spider,and NQ-Open) demonstrate that ER-JEPA consistently outperforms LLM-JEPA.

Primary area:
foundation or frontier models, including LLMs
Keywords:
Large language models, JEPA, Continual Learning, Representation Learning

Reviews

Among:
Shape:
1234this paper, Top 49%
Figure 1. Mean review rating of the reviewed papers at ICLR 2027. This paper averages 2.33 over 3 reviews, above 51% of the others.
Reviewer_Xv4hOpenReview ↗
Rating
3
Confidence
3
Informativeness
2
Presentation
3
Soundness
3
Read review
Paper summary

This paper improves LLM-JEPA by introducing an experience replay path. The study primarily uses Llama-3.2-1B-Instruct and evaluates the model across five datasets. Through experimental settings including matched training compute, token-matched control, and current-batch control, combined with three replay selection strategies and five random seeds, the paper analyzes error correction capability, retention of correct predictions, regex structure, and expression length. Results show that ER-JEPA outperforms LLM-JEPA in multiple aspects and reduces overfitting and accuracy degradation during training.

Critical strengths

The research question is valuable, the methodology is simple, and the experimental design is relatively solid. In particular, the paper goes beyond a direct accuracy comparison by including matched-compute, token-matched, and current-batch controls, which help support the claim that historical replay provides additional supervision beyond extra token exposure or an additional loss term.

Critical weaknesses
  1. The paper does not report the value of the replay budget RR, nor does it present results for different values of RR. This omission affects both reproducibility and the understanding of the replay mechanism and its training cost.
  2. The novelty of the method is incremental, with the main modification being the integration of experience replay into LLM-JEPA. Experience replay is a mature mechanism, and the paper does not systematically compare ER-JEPA with stronger replay, data-resampling, or anti-forgetting baselines under matched compute.
Questions for authors
  1. What is the value range for the replay budget RR? Is there any ablation study on RR?
  2. What is the design rationale behind Equation (18)? Intuitively, the more times a sample is replayed, the more important it may be to the model. However, under the formula, a higher replay count makes the sample more likely to be discarded. Why is this design chosen, and why not simply discard samples based on the JEPA error directly?
3 comments
AuthorsOpenReview ↗

Response to Reviewer Xv4h – W1 & Q1

Thank you for taking the time to review our paper and for your encouraging comments. We address your questions and comments below, and would appreciate your consideration in raising your score in light of these clarifications and new results.

W1 and Q1. Replay budget details, training cost, and ablation study on RR

Thank you for these comments. We added more details about the replay budget RR, its training cost, and an ablation study on RR in Appendix F.2 (Figure 22 and Table 10) of the revised manuscript.

First, the per-GPU replay budget RR ranges from 00 (standard LLM-JEPA without replay) up to BB or 2B2B, where BB is the fresh micro-batch size per GPU. All main experiments use R=16R = 16 examples per GPU per micro-batch, matching B=16B = 16 (R/B=1.0R / B = 1.0), with replay weight β=1.0\beta = 1.0 and memory capacity M=100,000M = 100{,}000. Early in training when the active memory holds fewer than RR items, we set Rt=min⁡(R,∣Mt∣)R_t = \min(R, |\mathcal{M}_t|). Response Table 1 reports the five-seed training compute and token profile at the matched compute budget (240.31240.31 PFLOPs) on NL-RX-SYNTH with Llama-3.2-1B-Instruct.

Response Table 1. Replay budget and matched-compute training cost (240.31240.31 PFLOPs) on NL-RX-SYNTH with Llama-3.2-1B-Instruct over five seeds.

MethodBudget RR / GPUPFLOPsFresh / Replay Tokens (M)Exact Match (%)
Baseline (SFT)00240.31240.313.473.47 / 0.000.0053.63±4.9053.63 \pm 4.90
LLM-JEPA00240.31240.312.652.65 / 0.000.0068.75±4.9268.75 \pm 4.92
ER-JEPA (Content Replay)1616240.19240.192.202.20 / 2.482.4883.65±1.48\mathbf{83.65 \pm 1.48}

At matched PFLOPs, ER-JEPA (R=16R = 16) improves exact-match accuracy by +14.90+14.90 percentage points over LLM-JEPA (R=0R = 0). After training, we discard the episodic memory and replay pathway, so ER-JEPA adds zero inference cost.

Additionally, we conducted a controlled ablation study over R∈0,4,8,16,32R \in \\{0, 4, 8, 16, 32\\} on Spider text-to-SQL generation using Qwen3-1.7B (Appendix F.2, Figure 22 and Table 10), with a fresh micro-batch size of B=32B = 32 per GPU across 66 epochs. Response Table 2 summarizes the accuracy and training time across different values of RR.

Response Table 2. Ablation study on the replay budget RR per GPU on Spider with Qwen3-1.7B (B=32B = 32 per GPU, 66 epochs).

Replay Budget RR / GPURatio R/BR / BPeak Exec Acc (%) ↑\uparrowFinal Exec Acc (%) ↑\uparrowSpeedup to 61.961.9\\%Training Time (min)
R=0R = 0 (LLM-JEPA)0.0000.00061.9261.9261.9261.921.00×1.00\times34.934.9
R=4R = 40.1250.12562.4062.4060.4060.401.13×1.13\times39.039.0
R=8R = 80.2500.25063.1763.1763.1763.171.09×1.09\times39.339.3
R=16R = 160.5000.50065.6565.6565.1065.101.28×1.28\times42.442.4
R=32R = 321.0001.00066.0066.0065.3865.381.42×1.42\times47.747.7

As shown in Response Table 2 and Appendix F.2 (Figure 22 and Table 10), increasing the replay budget RR generally improves execution accuracy while requiring more training time. A very small budget (R=4R = 4) gives limited late-stage gains due to sparse replay coverage, whereas R≥8R \ge 8 steadily raises accuracy and reaches the R=0R = 0 baseline level in less training time (up to a 1.42×1.42\times speedup at R=32R = 32, 24.624.6 vs. 34.934.9 min). However, moving from R=16R = 16 (0.5B0.5B) to R=32R = 32 (BB) brings only a modest additional accuracy gain while adding noticeably more training time (47.747.7 vs. 42.442.4 min). Therefore, setting RR between 0.5B0.5B and BB provides a practical balance between accuracy gains and training cost.

AuthorsOpenReview ↗

Response to Reviewer Xv4h – W2 & Q2

W2. Comparisons with replay, resampling, and anti-forgetting baselines

Thank you for raising this concern. ER-JEPA is not a trivial combination of experience replay and LLM-JEPA. In LLM-JEPA, representation alignment converges early, yet the model still makes generation errors and loses previously correct predictions (Figure 3). ER-JEPA mitigates this gap between representation alignment and prediction accuracy. Please see our theoretical proof in Appendix B (Proposition B.1) and the parameter trajectories and test loss landscape in Appendix Figure 11 of the revised manuscript. We hope these analyses help clarify our mechanism. The results show that re-evaluating stored examples changes the gradient update direction and guides optimization to a lower-loss region while preserving alignment.

Moreover, ER-JEPA is neither an anti-forgetting algorithm nor a data-resampling heuristic. First, our manuscript never mentions the word "forgetting." Second, we do not compare ER-JEPA directly with continual-learning replay or anti-forgetting baselines because these methods are designed for continual learning and primarily address catastrophic forgetting. In contrast, our study investigates how to improve existing open-source LLMs' learning and understanding of core semantics while preserving their text-generation capability. These are distinct research problems, and a direct empirical comparison would therefore not be scientifically meaningful. Within our stationary setting, Section 3.3 (Figures 6 and 7) already shows that all three ER-JEPA replay variants outperform token-matched resampling and current-batch reuse under matched budgets, while Appendix Table 3 presents our stepwise module ablation, making it clear why both the token-matched control and the current-batch control are needed.

Finally, ER-JEPA generalizes across multiple models and tasks with higher accuracy and faster convergence. As shown in Figure 1, Table 1, and Section 3.2 of the revised manuscript, ER-JEPA consistently improves performance across four model families and five tasks. Under matched compute, ER-JEPA achieves higher accuracy and reaches SFT performance much faster (Figures 4 and 5).

Thank you again for your kind comments, which are valuable for improving our manuscript.

Q2. What is the design rationale behind Equation (18)? Why not simply discard samples based on the JEPA error directly?

Thank you for your insightful question and we will discuss more about it below.

We design Equation (18) to prevent already replayed or noisy samples from permanently occupying finite memory slots when the buffer reaches capacity. In qn=σn/(1+rn)q_n = \sigma_n / (1 + r_n), rnr_n records how much replay supervision sample nn has already received rather than its intrinsic importance. Discounting σn\sigma_n by 1+rn1 + r_n ensures healthy memory turnover. Once a sample has been replayed several times, its learnable signal has largely been absorbed, or its remaining error reflects noise. Lowering its retention score frees finite slots for newly encountered samples (rn=0r_n = 0) and avoids overfitting a small set of stubborn examples. Pairing error-driven memory with decay is also a common design in modern memory architectures [1, 2]. For example, Titans [1] and Nested Learning [2] store high-surprise inputs while applying decay mechanisms so older traces gradually make room for new information.

As for discarding samples based on raw JEPA error σn\sigma_n alone, noisy or ambiguous outliers can retain a persistently high alignment error σn\sigma_n even after repeated replay updates. Without discounting by 1+rn1 + r_n, such stubborn examples would permanently occupy finite memory slots as rnr_n grows, starving newly encountered learnable samples (rn=0r_n = 0) of replay opportunities and risking overfitting to a small subset of outliers. Depreciating σn\sigma_n by 1+rn1 + r_n directly prevents this slot monopolization by ensuring that repeatedly replayed entries gradually yield room to fresh samples. In our main SYNTH runs, capacity (M=100,000M = 100{,}000) exceeds total writes (13,31213{,}312), so eviction is inactive. Under small capacities where eviction is active (M=10M = 10 to 10410^4 in Appendix F.2), accuracy remains stable from 84.2284.22\\% to 84.9884.98\\%.

Thank you again for your time and valuable feedback, which has helped us improve the clarity and technical depth of our work.

References

[1] Behrouz, A., Zhong, P., and Mirrokni, V. Titans: Learning to Memorize at Test Time. NeurIPS, 2025.

[2] Behrouz, A., Razaviyayn, M., Zhong, P., and Mirrokni, V. Nested Learning: The Illusion of Deep Learning Architectures. NeurIPS, 2025.

We have highlighted the relevant revisions and the figures presenting our new results in blue in the revised manuscript. If you have any further questions, feel free to communicate with us.

Reviewer_v4HUOpenReview ↗
Rating
2
Confidence
2
Informativeness
2
Presentation
1
Soundness
1
Read review
Paper summary

This paper studies joint-embedding predictive architectures (JEPAs) for language models in combination with experience replay. The authors argue that strong representation alignment, as in LLM-JEPA, does not always lead to accurate predictions. To address this, they propose ER-JEPA, which adds an episodic replay path to the existing LLM-JEPA pipeline: past training pairs are stored in a memory and revisited during later updates to provide additional supervision. The method is evaluated with a Llama-3.2-1B backbone on four benchmarks.

Critical strengths
  • The proposed method shows strong performance improvements over the LLM-JEPA baseline.
Critical weaknesses
  • Presentation and clarity. The paper is not well structured and is difficult to follow.
  1. Figure 2 is referenced (Introduction) before Figure 1, which is not referenced until Section 3.2.

  2. Several equations are not well defined, and some notations are not explained.

  3. Important details are deferred to the appendix, which makes the main paper hard to understand on its own. This includes the memory storage, retrieval, and update rules and the replay selection policies

  4. Eq. (4) appears to be incorrect.

  • Unclear motivation. The paper argues that LLM-JEPA can still make errors even after its alignment loss converges. However, ER-JEPA uses the same LLM-JEPA objective and differs mainly in which samples are fed to it, namely by revisiting stored examples. It is unclear how this addresses the limitation of LLM-JEPA identified in the paper. The authors should explain why replaying past examples resolves the gap between representation alignment and prediction accuracy.

  • Single backbone. All experiments use Llama-3.2-1B. It is unclear whether the method generalizes to other model families.

  • Scalability. How does the proposed method perform on larger models?

Questions for authors
  • Is the improvement specific to JEPA? Could the authors report results of applying the same episodic replay to other objectives (e.g., SFT without the JEPA loss)?

  • Does ER-JEPA generalize to other model families?

6 comments
AuthorsOpenReview ↗

Response to Reviewer v4HU – W1

Thank you for taking time to review our paper. We address your questions and comments below, and would appreciate your consideration in raising your score in light of these clarifications and new results.

W1. Presentation, figure ordering, notations, main-text completeness, and Equation (4)

1. Figure reference ordering (Figure 1 vs. Figure 2)

Presenting empirical results in Figure 1 before introducing the method in Figure 2 is a common choice in recent large language model research. In our manuscript, Figure 1 summarizes ER-JEPA's empirical gains across tasks immediately after the abstract. Figure 2 then compares next-token prediction, LLM-JEPA, and ER-JEPA in the Introduction.

For example, LLM-JEPA [1] (ICLR 2026) summarizes fine-tuning results across datasets and model families in Figure 1, then illustrates how JEPA pairs natural-language descriptions with code in Figure 2. DeepSeek-R1 [2] (Nature, 2025) shows how DeepSeek-R1-Zero's AIME accuracy and response length evolve during reinforcement learning in Figure 1, then presents the multistage training pipeline for DeepSeek-R1 in Figure 2. Likewise, Large Language Diffusion Models [3] (NeurIPS 2025) presents zero- and few-shot benchmark results in Figure 1 and diagrams LLaDA's pre-training, supervised fine-tuning, and sampling in Figure 2. Other recent papers on language models, vision-language models, and image generation also follow the same convention [4–8].

2. Complete notation definitions and self-contained main text

We carefully checked the equations and parameter definitions in the paper and added more details. Please see the blue-highlighted text in Section 2 of the revised manuscript. If any specific symbol or equation still feels unclear, please let us know and we will gladly refine it.

As for the memory rules and replay selection policies in the appendix, our main focus is the episodic replay pathway itself rather than a specific selection strategy. Indeed, our experiments show that three standard strategies (content-based, uniform, and hard replay) all consistently outperform LLM-JEPA. Therefore, we placed these detailed rules in the appendix to keep the main text concise and focused.

3. Eq. (4) appears to be incorrect

Equation (4) in our original manuscript is reasonable and directly matches our implementation. In Section 2.1, Equation (4) originally wrote the overall step-tt training objective as:

LER-JEPA(Bt)=LLLM-JEPA-new(Bt)+β,LLLM-JEPA-replay(Bt).\mathcal{L}_{\mathrm{ER\text{-}JEPA}}(\mathcal{B}_t) = \mathcal{L}_{\mathrm{LLM\text{-}JEPA\text{-}new}}(\mathcal{B}_t) + \beta\\,\mathcal{L}_{\mathrm{LLM\text{-}JEPA\text{-}replay}}(\mathcal{B}_t).

We would like to explain how Equation (4) is computed. At step tt, LLLM-JEPA-new(Bt)\mathcal{L}_{\mathrm{LLM\text{-}JEPA\text{-}new}}(\mathcal{B}_t) (Equation (3)) evaluates the token-normalized language-modeling loss and sample-averaged JEPA cosine distance loss on the BB fresh examples in Bt\mathcal{B}_t. In parallel, LLLM-JEPA-replay(Bt)\mathcal{L}_{\mathrm{LLM\text{-}JEPA\text{-}replay}}(\mathcal{B}_t) (Equation (6) in Section 2.2) retrieves Rt≤RR_t \le R stored pairs (τ~iu,τ~ia)(\widetilde{\tau}_i^u, \widetilde{\tau}_i^a) from episodic memory, recomputes their representations under the current shared parameters (θ,ϕ)(\theta, \phi) (Equation (5)), and applies the same two loss terms with per-branch normalization (taking the value 00 when Rt=0R_t = 0). Equation (4) then combines both branches via β≥0\beta \ge 0 (loss = l_new + beta * l_replay) to update (θ,ϕ)(\theta, \phi) in a single backward pass.

Moreover, the reason why Equation (4) was originally written in this form is to mirror the dual-pathway design in Figure 2(c) (recovering standard LLM-JEPA when β=0\beta = 0) while keeping the high-level objective in Section 2.1 concise before introducing replay details in Section 2.2. Specifically, Bt\mathcal{B}_t was written as the argument of the replay term because our default content-based replay queries episodic memory using cue representations from Bt\mathcal{B}_t at step tt, and summing the two separately normalized terms ensures that β\beta cleanly controls replay strength regardless of sequence-length differences or early warm-up sizes (Rt<RR_t < R).

At the same time, to further improve readability and clarity, we edited Equations (4) and (6) in Section 2 of the revised manuscript (see the blue-highlighted text) to write LER-JEPA(Bt,Rt)=LLLM-JEPA-new(Bt)+β,LLLM-JEPA-replay(Rt)\mathcal{L}_{\mathrm{ER\text{-}JEPA}}(\mathcal{B}_t, \mathcal{R}_t) = \mathcal{L}_{\mathrm{LLM\text{-}JEPA\text{-}new}}(\mathcal{B}_t) + \beta\\,\mathcal{L}_{\mathrm{LLM\text{-}JEPA\text{-}replay}}(\mathcal{R}_t), better distinguishing the current mini-batch Bt\mathcal{B}_t from the selected replay batch Rt=(τ~iu,τ~ia)i=1Rt\mathcal{R}_t = \\{(\widetilde{\tau}_i^u, \widetilde{\tau}_i^a)\\}_{i=1}^{R_t}.

AuthorsOpenReview ↗

Response to Reviewer v4HU – W2

W2. Motivation of ER-JEPA and how episodic replay bridges representation alignment and prediction accuracy

Episodic replay bridges representation alignment and prediction accuracy by re-evaluating stored examples to change the gradient update direction, which guides optimization to a lower-loss region while preserving alignment. We address your comments and provide more details below.

In our original manuscript, Appendix B (Proposition B.1) proves that introducing the episodic replay term changes the gradient update direction along current-batch blind directions and reduces loss on past examples. Unlike LLM-JEPA, which optimizes only the current-batch loss LBL_{\mathcal{B}}, ER-JEPA optimizes a different objective LB+βLRL_{\mathcal{B}} + \beta L_{\mathcal{R}} by adding a replay loss LRL_{\mathcal{R}} over stored examples. In LLM-JEPA, once alignment loss converges early in training, single-batch updates have zero gradient along directions to which the current batch is insensitive (PgB=0P g_{\mathcal{B}} = 0). Moving along these blind directions can disturb token predictions learned on earlier examples, even while alignment remains strong. As proved in Proposition B.1, re-evaluating stored examples Rt\mathcal{R}_t under the current parameters ω0\omega_0 produces a replay gradient gRg_{\mathcal{R}} with an orthogonal component d=−PgRd = -P g_{\mathcal{R}}. Adding βgR\beta g_{\mathcal{R}} therefore redirects the update from ωC=ω0−ηgB\omega_{\mathrm{C}} = \omega_0 - \eta g_{\mathcal{B}} to ωE=ω0−η(gB+βgR)\omega_{\mathrm{E}} = \omega_0 - \eta(g_{\mathcal{B}} + \beta g_{\mathcal{R}}) and strictly lowers loss on past examples beyond a current-batch step.

gER=gB+βgR,PgB=0,d=−PgR,P(ωE−ω0)=ηβd,P(ωC−ω0)=0,g_{\mathrm{ER}} = g_{\mathcal{B}} + \beta g_{\mathcal{R}}, \qquad P g_{\mathcal{B}} = 0, \qquad d = -P g_{\mathcal{R}}, \qquad P(\omega_{\mathrm{E}} - \omega_0) = \eta\beta d, \qquad P(\omega_{\mathrm{C}} - \omega_0) = 0,

LR(ω0−ηgER)−LR(ω0−ηgB)=−ηβ(∥PgR∥22+∥(I−P)gR∥22)+O(η2)<0.L_{\mathcal{R}}\bigl(\omega_0 - \eta g_{\mathrm{ER}}\bigr) - L_{\mathcal{R}}\bigl(\omega_0 - \eta g_{\mathcal{B}}\bigr) = -\eta\beta\Bigl(\lVert P g_{\mathcal{R}}\rVert_2^2 + \lVert (I - P) g_{\mathcal{R}}\rVert_2^2\Bigr) + \mathcal{O}(\eta^2) < 0.

Furthermore, we agree that this point deserves a clearer explanation, so we added empirical trajectory and loss-landscape visualizations to the revised manuscript. Please see the blue-highlighted updates in Section 3.3 and Appendix B (Figure 11) of our revised manuscript. Specifically, we recorded the actual LoRA checkpoint weights and update angles from Qwen3-1.7B training runs on Spider (lr=4e−5\mathrm{lr} = 4\mathrm{e}{-5}, seed 2323) and evaluated their losses on the Spider test set. As shown in Appendix Figure 11, across six epochs all three methods reach comparable total update norms (∥ΔW∥F=15.0\lVert \Delta W \rVert_F = 15.0, 15.315.3, and 15.515.5), ruling out step-size differences. While SFT remains unaligned (JEPA=0.70\text{JEPA} = 0.70, test CE=0.153\text{CE} = 0.153), both LLM-JEPA and ER-JEPA reach near-zero test JEPA loss (0.00020.0002 and 0.00010.0001). However, LLM-JEPA moves 65.1∘65.1^\circ out of the (SFT,ER-JEPA)(\text{SFT}, \text{ER-JEPA}) plane and stops at a test SQL cross-entropy of 0.1420.142, whereas episodic replay rotates ER-JEPA 72.5∘72.5^\circ away into a deeper Spider test cross-entropy valley (0.1300.130 vs. 0.1420.142).

Based on the theoretical and empirical analyses above, introducing episodic replay changes the gradient update direction during training, guiding optimization to a lower-loss region and bridging the gap between representation alignment and prediction accuracy.

AuthorsOpenReview ↗

Response to Reviewer v4HU – W3, W4 and Q2

W3, W4 and Q2: Generalization to other model families and scalability to larger models

Thank you for these comments. We fully understand your concerns regarding generalization across different model families and parameter scales, and we have added additional experiments across diverse model families and larger model scales in the revised manuscript (see the blue-highlighted text in Section 3.2 and Table 1).

ER-JEPA generalizes consistently across newer model families and scales. Importantly, our proposed method is not restricted to any specific model series or parameter size, and instead generalizes effectively across diverse backbones. To verify this beyond Llama-3.2-1B-Instruct, we conducted additional five-seed experiments on Gemma-2-2B-it (2B2\text{B}, released July 31, 2024 [9]), Qwen3-1.7B (1.7B1.7\text{B}, released April 29, 2025 [10]), and Granite-4.2-3B (3B3\text{B}, released August 25, 2026 [11]). Moreover, beyond NL-RX-SYNTH, we also extended our evaluation and analysis to Spider and GSM8K. NL-RX was previously used mainly as an illustrative example, whereas ER-JEPA readily extrapolates across both model families and task domains.

  • (1) Gemma-2-2B-it (2B2\text{B}) on NL-RX-SYNTH (λ=1\lambda = 1, k=1k = 1, lr=1×10−5\text{lr} = 1 \times 10^{-5}). Across five seeds, ER-JEPA (Content Replay) outperforms both SFT (+11.65+11.65 pp) and LLM-JEPA (+4.98+4.98 pp) on Exact Match, and improves Official DFA-Equal by +10.03+10.03 pp and +3.80+3.80 pp, respectively. It also substantially reduces the cross-seed variance of LLM-JEPA on both metrics.

Response Table 1. Five-seed results of Gemma-2-2B-it on NL-RX-SYNTH (λ=1\lambda = 1, k=1k = 1, lr=1×10−5\text{lr} = 1 \times 10^{-5}).

MethodExact Match (%) ↑\uparrowOfficial DFA-Equal (%) ↑\uparrow
SFT68.21±4.7468.21 \pm 4.7474.40±4.1174.40 \pm 4.11
LLM-JEPA74.88±12.5174.88 \pm 12.5180.63±10.6180.63 \pm 10.61
ER-JEPA (Content Replay)79.86±7.62\mathbf{79.86 \pm 7.62}84.43±6.74\mathbf{84.43 \pm 6.74}
  • (2) Qwen3-1.7B (1.7B1.7\text{B}) on Spider (lr=4×10−5\text{lr} = 4 \times 10^{-5}). Across five seeds on text-to-SQL generation, ER-JEPA (Uniform Replay) reaches 65.6765.67\\% accuracy. It outperforms SFT by +5.61+5.61 pp (p=0.0009p = 0.0009) and LLM-JEPA by +4.71+4.71 pp (p=0.0011p = 0.0011), winning in all five seeds.

Response Table 2. Five-seed results of Qwen3-1.7B on Spider (lr=4×10−5\text{lr} = 4 \times 10^{-5}).

MethodAccuracy (%) ↑\uparrow
SFT60.06±1.0660.06 \pm 1.06
LLM-JEPA60.95±0.9560.95 \pm 0.95
ER-JEPA (Uniform Replay)65.67±0.97\mathbf{65.67 \pm 0.97}
  • (3) Granite-4.2-3B (3B3\text{B}) on GSM8K (lr=2×10−5\text{lr} = 2 \times 10^{-5}). Across five seeds on mathematical reasoning, ER-JEPA (Uniform Replay) reaches 69.1069.10\\% accuracy. It outperforms SFT by +3.70+3.70 pp (p=0.00055p = 0.00055, winning in all five seeds) and LLM-JEPA by +1.08+1.08 pp with lower seed variance.

Response Table 3. Five-seed results of Granite-4.2-3B on GSM8K (lr=2×10−5\text{lr} = 2 \times 10^{-5}).

MethodAccuracy (%) ↑\uparrow
SFT65.40±1.1965.40 \pm 1.19
LLM-JEPA68.02±1.4668.02 \pm 1.46
ER-JEPA (Uniform Replay)69.10±1.22\mathbf{69.10 \pm 1.22}

Furthermore, we added fine-grained analyses on Spider and GSM8K in the revised manuscript (see the blue-highlighted text in Appendix D). The results indicate that ER-JEPA consistently improves both structured SQL generation and multi-step mathematical reasoning. On Spider, ER-JEPA leads in all query hardness levels and SQL clause types, with its largest gains on Hard (+7.6+7.6 pp), JOIN (+7.4+7.4 pp), and nested queries (+6.9+6.9 pp). It also lowers schema hallucination from 10.0610.06\\% and 9.889.88\\% to 8.108.10\\%. On GSM8K, ER-JEPA outperforms SFT and LLM-JEPA across all reasoning depths, including 44-step (63.5263.52\\%) and 5+5+-step (50.4050.40\\%) problems.

Together with Llama-3.2-1B-Instruct, these results across four model families (1B1\text{B} to 3B3\text{B}) and three distinct tasks confirm that ER-JEPA generalizes reliably with zero inference overhead. We have included these new experiments and analyses in the revised manuscript.

Thank you again for your constructive comments, which have greatly improved the clarity and completeness of our manuscript.

Reviewer_kYazOpenReview ↗
Rating
2
Confidence
3
Informativeness
2
Presentation
2
Soundness
2
Read review
Paper summary

The paper proposes ER-JEPA, an extension of LLM-JEPA that revisits stored training examples alongside the current batch. Replayed examples contribute both answer-generation and representation-prediction losses, with their representations recomputed using the current model. The paper compares content-based, uniform, and hard replay, and removes the replay machinery after training.

Experiments fine-tune Llama-3.2-1B-Instruct on five datasets covering regular-expression generation, mathematical reasoning, text-to-SQL generation, and question answering. The strongest evidence comes from SYNTH, where the paper reports comparisons at matched training compute, additional controls for token exposure and current-batch reuse, and analyses across five random seeds. The main contribution is an empirical demonstration that replay can improve this training recipe without adding inference overhead.

Critical strengths
  1. The improvement on SYNTH is substantial and survives a more meaningful comparison than equal training epochs. At the final matched-compute budget, content replay reaches 83.65 ± 1.48% accuracy versus 68.75 ± 4.92% for LLM-JEPA. All three replay policies improve over LLM-JEPA at the evaluated budgets. The five-seed results and individual learning curves make this a useful empirical finding, rather than a result resting on one favorable run.

  2. The paper makes a useful effort to investigate what replay contributes. The token-exposure and current-batch controls in Figures 5–6 go beyond a straightforward comparison against the original method. In particular, historical replay improves over current-batch reuse, and uniform replay works well, suggesting that a sophisticated retrieval rule is not essential. Together with the reported gains on the other tasks and the absence of additional inference cost, this gives the method practical value.

Critical weaknesses
  1. The interpretation of the regex results goes beyond what exact matching establishes. Section 3.1 evaluates whether the generated expression exactly matches the reference, but different expressions can recognize the same set of strings. This matters concretely in Table 4: some highlighted extra .* suffixes are redundant because the reference already ends in .*. Removing them improves exact match without changing the accepted strings. Figure 11 also contains examples of genuine functional corrections, so this does not invalidate the improvement. However, the aggregate results do not establish how much of the reported error correction reflects better functionality rather than closer reproduction of the reference spelling. This limits the evidence for improved semantic understanding, particularly because the detailed analyses concentrate on SYNTH.

  2. The experimental controls need more detail to establish the size of the historical-replay contribution. The current-batch control already reaches 81.24%, compared with 68.75% for LLM-JEPA and 83.65% for content replay. Thus, the increment attributable to historical examples in this comparison is appreciably smaller than the headline gain. The paper does not clearly specify the exact current-batch sampling procedure, the numerical replay weight and replay budget used in the main experiments, or the step counts and learning-rate schedules used for the matched-budget controls. The large variation of the token-matched control, 62.25 ± 20.80%, also needs explanation. These details affect both reproducibility and the interpretation of why the method works.

Questions for authors
  1. Can you evaluate the existing SYNTH predictions using functional equivalence as well as exact match? I am particularly interested in whether the improvements in over-generation remain substantial after equivalent expressions are counted as correct.

  2. Could you clarify the construction of the current-batch and token-matched controls, including the replay weight, number of examples reused, training steps, and learning-rate schedules? What explains the large variation across seeds for token-matched LLM-JEPA?

could test more new models by your method? I have not heard about llama 3.2 for many years , it is to old, could your method generalized on other current sota models? I do not think this is a solid work.

7 comments
AuthorsOpenReview ↗

Response to Reviewer kYaz – W1 & W2

Thank you for taking time to review our paper. We address your questions and comments below, and would appreciate your consideration in raising your score in light of these clarifications and new results.

W1. Interpretation of regex exact match, trailing .* suffixes, and detailed analysis beyond SYNTH

Thank you for this comment. For clarity, the reason why we choose exact match is to strictly adhere to the official benchmark protocol. Section 4.1 of the original LLM-JEPA paper (Huang et al. [1]) defines NL-RX accuracy using exact string match. Moreover, Row 3 of Table 1 in Huang et al. [1] displays a reference regular expression that already terminates in .* and explicitly highlights an appended .* as an extra-token error. Retaining the exact task definition and evaluation metric of the baseline therefore ensures a fair comparison.

Additionally, although extra trailing .* suffixes do not change the accepted string set when the reference already ends in .*, we respectfully disagree with manually removing them via post-hoc stripping during evaluation. These redundant suffixes reflect genuine generation errors rather than benign spelling variations. Prior work has shown that repeatedly emitting suffixes such as .* indicates that the autoregressive decoder falls into repetitive degeneration [2] and cannot reliably determine the stopping boundary when a symbolic constraint is complete [3]. Altering raw model predictions to strip extra .* suffixes would mask real stopping-boundary failures. We therefore do not remove these suffixes, ensuring both evaluation accuracy and a fair comparison across methods.

At the same time, we fully agree that evaluating functional equivalence alongside exact match is important to separate semantic gains from surface spelling. We therefore added a functional equivalence analysis in the revised manuscript (see the blue-highlighted text in Section 3.2 and Figure 5) using DFA-Equal (following the original NL-RX protocol [4]) and false acceptance rate (FPR). The results show that ER-JEPA (Uniform Replay) improves semantic equivalence and suppresses false acceptance with less compute. Across all six matched budgets, ER-JEPA (Uniform Replay) outperforms both SFT and LLM-JEPA in five-seed mean DFA-Equal accuracy, matching final SFT and final LLM-JEPA using 75.5%75.5\% and 25.6%25.6\% fewer FLOPs, respectively. At the final 240.3240.3 PFLOPs budget, ER-JEPA (Uniform Replay) outperforms LLM-JEPA in all five seeds, raising mean DFA-Equal accuracy from 88.91%88.91\% to 90.00%90.00\% while reducing false acceptance FPR from 4.15%4.15\% to 2.72%2.72\%.

Furthermore, we added fine-grained analyses on Spider and GSM8K in the revised manuscript (see the blue-highlighted text in Appendix D). The results indicate that ER-JEPA consistently improves both structured SQL generation and multi-step mathematical reasoning. On Spider, ER-JEPA leads in all query hardness levels and SQL clause types, with its largest gains on Hard (+7.6+7.6 pp), JOIN (+7.4+7.4 pp), and nested queries (+6.9+6.9 pp). It also lowers schema hallucination from 10.06%10.06\% and 9.88%9.88\% to 8.10%8.10\%. On GSM8K, ER-JEPA outperforms SFT and LLM-JEPA across all reasoning depths, including 44-step (63.52%63.52\%) and 5+5+-step (50.40%50.40\%) problems.

W2. Establishing the historical-replay contribution and experimental control details

Thank you for your constructive comments! We agree that clarifying these experimental controls is important for both reproducibility and understanding the contribution of historical replay. Please see Q2 below for the detailed experimental configurations, together with our explanation and analysis of the historical-replay contribution and the token-matched seed variance.

AuthorsOpenReview ↗

Response to Reviewer kYaz – Q1 & Q2(part 1)

Thank you for your insightful questions and we will discuss more about them as below.

Q1. Can you evaluate the existing SYNTH predictions using functional equivalence as well as exact match?

Thank you for this insightful question. We added a functional equivalence analysis in the revised manuscript (see the blue-highlighted text in Section 3.2 and Figure 5) using DFA-Equal [4] and false acceptance rate (FPR). Evaluating ER-JEPA (Uniform Replay), our results show that ER-JEPA improves semantic equivalence and suppresses false acceptance with less compute, matching the final DFA-Equal accuracy of SFT and LLM-JEPA using 75.5%75.5\% and 25.6%25.6\% fewer FLOPs, respectively. Overall, for DFA-Equal, ER-JEPA (Uniform Replay) consistently and significantly outperforms both LLM-JEPA (90.00±0.31%90.00 \pm 0.31\% vs. 88.91±0.75%88.91 \pm 0.75\%, p=0.0134p = 0.0134) and SFT (78.78±2.81%78.78 \pm 2.81\%, p=0.0004p = 0.0004) across all five seeds, and for FPR, ER-JEPA (Uniform Replay) also outperforms both LLM-JEPA (2.72±0.14%2.72 \pm 0.14\% vs. 4.15±0.72%4.15 \pm 0.72\%, p=0.0067p = 0.0067) and SFT (2.87±0.24%2.87 \pm 0.24\%). Moreover, all three ER-JEPA policies (Content, Uniform, and Hard Replay) achieve higher mean DFA-Equal accuracy than LLM-JEPA (89.61%89.61\%, 90.00%90.00\%, and 89.51%89.51\% vs. 88.91%88.91\%; see Response Table 2 in Q2 below).

Q2. Could you clarify the construction of the current-batch and token-matched controls, including the replay weight, number of examples reused, training steps, and learning-rate schedules? What explains the large variation across seeds for token-matched LLM-JEPA?

Thank you for this question. We clarify the control configurations, the contribution of historical memory retrieval, and the cross-seed variance below.

1. Construction and training configurations of the two controls

Both controls are trained on NL-RX-SYNTH across the same five paired seeds under the exact non-padding token budget of paired ER-JEPA (≈4.68M\approx 4.68\text{M} tokens) to isolate the source of ER-JEPA's gains. We also added a direct method comparison summarizing their shared modules and configuration differences in the revised manuscript (see the blue-highlighted text in Section 3.3 and Appendix Table 3).

(1) Current-Batch Control. This control tests whether replaying historical examples adds value beyond applying the auxiliary replay loss to the current mini-batch. It disables episodic memory and retrieval, recomputing the auxiliary replay loss directly on the current micro-batch per GPU under separate per-branch token normalization. Its replay weight is β=1.0\beta = 1.0, and each current-batch example is reused once per step (1×1\times current, matching ER-JEPA's 1×1\times historical replay ratio).

(2) Token-Matched LLM-JEPA. This control tests whether ER-JEPA's improvement comes merely from exposing standard LLM-JEPA to more training tokens. It runs single-branch LLM-JEPA without a replay branch until cumulative fresh tokens match ER-JEPA. Its replay weight is β=0\beta = 0, and 00 examples are reused within a step.

For training steps and learning-rate schedules, both controls share the same AdamW optimizer, an initial learning rate of 1×10−51 \times 10^{-5} with zero warmup, and HuggingFace's default linear learning-rate decay over the configured epoch ceiling. Unlike main ER-JEPA—which uses a pre-calibrated fixed compute step budget (416416 steps for 240.31240.31 PFLOPs)—each control seed dynamically tracks non-padding effective input tokens and stops via callback at the step boundary where cumulative tokens reach its paired ER-JEPA seed's 4.68M4.68\text{M} budget; because Current-Batch Control evaluates both fresh and reused branches per step whereas Token-Matched LLM-JEPA (β=0\beta = 0) evaluates only the fresh branch, they stop at 208–209208\text{–}209 and 442–443442\text{–}443 steps, respectively. Response Table 1 summarizes all settings.

Response Table 1. Configurations of the two token-matched controls alongside the paired ER-JEPA (content) reference across five seeds.

VariantReplay Weight β\betaExamples ReusedTraining StepsLR ScheduleInput Tokens (M)Exact Match (%)
Current-Batch Control1.01.01×1\times current208–209208\text{–}209Linear from 10−510^{-5}4.68–4.714.68\text{–}4.7181.24±1.1281.24 \pm 1.12
Token-Matched LLM-JEPA0000442–443442\text{–}443Linear from 10−510^{-5}4.68–4.694.68\text{–}4.6962.25±20.8062.25 \pm 20.80
ER-JEPA (content)1.01.01×1\times historical416416Linear from 10−510^{-5}4.684.6883.65±1.48\mathbf{83.65 \pm 1.48}
AuthorsOpenReview ↗

Response to Reviewer kYaz – Q2(part 2)

2. Contribution of historical memory retrieval beyond the current-batch control

Comparing ER-JEPA against Current-Batch Control (which differs only by disabling historical memory retrieval Mt\mathcal{M}_t, Appendix Table 3) confirms the effectiveness of historical replay from two perspectives:

  1. Statistically significant gain on Exact Match. Relative to Current-Batch Control (81.24±1.12%81.24 \pm 1.12\%), ER-JEPA (Content) raises Exact Match to 83.65±1.48%83.65 \pm 1.48\% (+2.41+2.41 pp, paired one-tailed p=0.0495<0.05p = 0.0495 < 0.05), and ER-JEPA (Uniform) reaches 85.04±0.54%85.04 \pm 0.54\% (+3.80+3.80 pp, outperforming Current-Batch Control in all five seeds with p=0.00041p = 0.00041). Thus, the Exact Match gain brought by historical replay is statistically significant.
  2. Functional equivalence requires historical memory retrieval. As shown in Response Table 2, on Official DFA-Equal, Current-Batch Control (87.47±0.51%87.47 \pm 0.51\%) actually underperforms standard LLM-JEPA (88.91±0.75%88.91 \pm 0.75\%) across all five seeds (−1.44-1.44 pp). By contrast, all three ER-JEPA variants achieve higher mean DFA-Equal than LLM-JEPA and outperform Current-Batch Control in all five seeds (p=0.00041p = 0.00041, 0.0000560.000056, and 0.0000390.000039), with ER-JEPA (Uniform) also outperforming LLM-JEPA in all five seeds, further demonstrating the effectiveness of historical memory retrieval.

Response Table 2. Five-seed functional equivalence (Official DFA-Equal, %) on NL-RX-SYNTH and paired one-tailed tt-test pp-values relative to Current-Batch Control.

MethodSeed 4Seed 23Seed 37Seed 82Seed 84Mean ±\pm SD (%)pp-value (vs. Current-Batch)
LLM-JEPA87.6087.6089.1089.1089.5089.5089.2089.2089.1589.1588.91±0.7588.91 \pm 0.75—
Current-Batch Control86.9586.9587.7587.7586.9586.9588.1088.1087.6087.6087.47±0.5187.47 \pm 0.51—
ER-JEPA (Content)90.00\mathbf{90.00}89.55\mathbf{89.55}88.85\mathbf{88.85}89.90\mathbf{89.90}89.75\mathbf{89.75}89.61±0.46\mathbf{89.61 \pm 0.46}0.000410.00041
ER-JEPA (Uniform)89.75\mathbf{89.75}89.95\mathbf{89.95}89.70\mathbf{89.70}90.15\mathbf{90.15}90.45\mathbf{90.45}90.00±0.31\mathbf{90.00 \pm 0.31}0.0000560.000056
ER-JEPA (Hard)88.80\mathbf{88.80}89.95\mathbf{89.95}89.20\mathbf{89.20}90.35\mathbf{90.35}89.25\mathbf{89.25}89.51±0.63\mathbf{89.51 \pm 0.63}0.0000390.000039

3. Why Token-Matched LLM-JEPA varies across seeds

Evaluating the same Token-Matched LLM-JEPA predictions with Official DFA-Equal shows that its large Exact Match variance (62.25±20.80%62.25 \pm 20.80\%) comes from redundant trailing suffixes on certain seeds rather than semantic collapse. Across the five seeds, Token-Matched LLM-JEPA scores 38.05%38.05\%, 71.90%71.90\%, 81.00%81.00\%, 41.55%41.55\%, and 78.75%78.75\% on Exact Match (62.25±20.80%62.25 \pm 20.80\%), while reaching 84.30%84.30\%, 89.80%89.80\%, 90.25%90.25\%, 87.15%87.15\%, and 89.80%89.80\% on Official DFA-Equal (88.26±2.53%88.26 \pm 2.53\%). Inspecting the raw predictions explains the low Exact Match on Seeds 44 and 8282. On Seeds 3737 and 8484, only 5.25%5.25\% and 6.30%6.30\% of predictions append redundant .* or () suffixes after the exact ground truth, whereas this rate jumps to 42.00%42.00\% on Seed 44 and 39.90%39.90\% on Seed 8282.

Once alignment converges early in training, continuing single-branch updates without historical replay leaves stopping boundaries sensitive to mini-batch fluctuations across seeds. As discussed in W1, these redundant suffixes reflect genuine generation errors rather than benign spelling variations, so we do not remove them via post-hoc stripping during evaluation. Instead, ER-JEPA resolves this issue effectively during training. By replaying historical examples alongside the current batch, ER-JEPA stabilizes stopping boundaries across all five seeds. It raises Exact Match to 83.65±1.48%83.65 \pm 1.48\% for Content Replay and 85.04±0.54%85.04 \pm 0.54\% for Uniform Replay, while also achieving higher Official DFA-Equal accuracy (89.61±0.46%89.61 \pm 0.46\% and 90.00±0.31%90.00 \pm 0.31\%).

Comments on the paper

2 comments

Dear Reviewers,

We hope this message finds you well.

As the discussion period is nearing its end, we wanted to ensure we have addressed all your concerns satisfactorily. If there are any additional points or feedback you'd like us to consider, please let us know. Your insights are invaluable to us, and we're glad to address any remaining issues to improve our work.

Thank you for your time and effort in reviewing our paper.

Dear Area Chair and Reviewers,

As the discussion period comes to a close, we sincerely thank the Area Chair for overseeing the review process and all reviewers for their constructive feedback.

In this work, we show that LLM-JEPA still makes generation errors and suffers accuracy drops during training even after its alignment loss converges near zero. We propose ER-JEPA, which replays stored training pairs alongside each fresh batch during fine-tuning and removes the replay pathway afterward for zero inference cost. At matched training compute, ER-JEPA raises NL-RX-SYNTH accuracy from 68.75% to 85.04% and matches baseline semantic equivalence using 25.6% to 75.5% fewer FLOPs. Notably, simple uniform sampling without complex memory addressing suffices to match or exceed elaborate retrieval rules, delivering consistent gains across five benchmarks and four model families (1B to 3B). During this period, we have carefully addressed all reviewers' questions and concerns, and we summarize our main updates below.

  • Reviewer kYaz pointed out that our improvement is "substantial", our evaluation across seeds provides "a useful empirical finding", and the "absence of additional inference cost" gives our method clear "practical value". Their main concern was whether our method generalizes to newer model families. To address this, we added experiments on three new models (Gemma-2-2B-it, Qwen3-1.7B, and Granite-4.2-3B) over five seeds in Table 1 and Appendix D, along with formal functional equivalence tests in Fig. 5. These results show consistent gains across all backbones and tasks, demonstrating the strong generalizability of our method.

  • Reviewer v4HU remarked that our proposed method achieves "strong performance improvements" over the LLM-JEPA baseline. They asked why replaying past examples improves prediction accuracy after alignment converges, and whether this benefit applies beyond JEPA. We addressed their questions with parameter trajectory and loss landscape analyses in Appendix B (Fig. 11) and SFT replay ablations in Sec. 3.3 (Table 11), while also refining our presentation in Sec. 2. The results show that episodic replay guides optimization toward regions with lower test loss while preserving alignment and improves both SFT and JEPA, verifying the effectiveness of our mechanism.

  • Reviewer Xv4h said that our "research question is valuable", our "methodology is simple", and our "experimental design is relatively solid". They also noted that our matched controls show how historical replay "provides additional supervision" beyond extra token exposure. They asked us to clarify the replay budget, its ablation results, and the memory update rule. We answered their questions by clarifying the memory eviction rule in Appendix E.2 and adding ablation studies on the replay budget and training cost in Appendix F.2 (Fig. 22 and Table 10). The results showcase steady accuracy gains and faster convergence to baseline accuracy, further confirming the effectiveness and efficiency of our method.

Overall, these new experiments, ablations, and optimization analyses establish ER-JEPA as a simple, general, and efficient fine-tuning method with zero inference overhead. We have included all clarifications and additional results in our revised manuscript. Once again, we sincerely thank the Area Chair and all reviewers for their time and feedback, which have improved our paper.

Best regards,

The Authors

From OpenReview, CC BY 4.0. Authors as signed there; shown unedited.

Related papers

Loading the map…

Discussion (0)

Sign in to comment.