acceptodds
Under review as a conference paper at ICLR 2027

Can Test-Time Memory Evolution Outperform Test-Time Training for Scientific Discovery?

14 views

Abstract

Recent test-time training (TTT) frameworks have enabled small open-weight Large Language Models (LLMs) to outperform inference-only systems powered by frontier models on scientific discovery tasks. However, TTT introduces additional computational cost for online weight updates during the discovery process, making computational efficiency an important consideration when scaling to larger models. This motivates us to explore an efficient test-time adaptation method for scientific discovery. Under the same discovery framework, we introduce Test-Time Memory Evolution (TTME), a plug-and-play test-time adaptation method, as an alternative to TTT. TTME continually updates the external memory by distilling the search experience to guide subsequent discovery without updating model weights. By controlling the discovery framework, we have a clean comparison between TTME and TTT on both performance and cost. Our evaluation shows that TTME outperforms TTT in all five mathematical tasks, while reducing computational cost by in FLOPs and in runtime. These results establish TTME as an effective and computationally efficient alternative to TTT in the evaluated discovery settings.

Primary area:
generative models
Keywords:
Test-time training, Self-Evolve Memory, Recursive Self Improvement, AI for Scientific Discovery

Reviews

Among:
Shape:
1234this paper, Bottom 27%
Figure 1. Mean review rating of the reviewed papers at ICLR 2027. This paper averages 2.00 over 3 reviews, above 27% of the others.
Reviewer_gaXAOpenReview ↗
Rating
2
Confidence
4
Informativeness
2
Presentation
3
Soundness
2
Read review
Paper summary

The paper studies whether test-time adaptation through external memory can serve as a more efficient alternative to test-time weight updates for scientific discovery. The authors introduce Test-Time Memory Evolution (TTME), which keeps the underlying LLM weights fixed and continually distills evaluated search experience into two memory components: Guideline Memory, which stores operational lessons from execution and validity feedback, and Strategy Memory, which extracts and consolidates reusable algorithmic ideas from successful parent-child modifications. The stored strategies are subsequently selected and adapted to guide candidate generation.

TTME is evaluated within two existing test-time-training-based discovery frameworks, TTT-Discover and ThetaEvolve, on five mathematical construction tasks and three model configurations. The reported results show that TTME achieves better best-found scores than the reported TTT baselines in all evaluated settings. On TTT-Discover with Qwen3-8B, the authors additionally report substantial computational savings, with 82.8–84.6% fewer FLOPs and 47.7–58.4% lower runtime excluding evaluation. Ablations suggest that both Guideline Memory and Strategy Memory contribute to performance.

Critical strengths
  1. The paper asks an important and timely question about the necessity of test-time weight updates in scientific discovery. Rather than assuming that online RL is necessary to accumulate search experience, the paper directly compares parameter adaptation with structured external-memory adaptation. This is a useful question both scientifically and practically, given the considerable computational cost of test-time training.

  2. TTME appears substantially more computationally efficient while remaining competitive or better in the reported experiments. The reported 82.8–84.6% reduction in FLOPs and 47.7–58.4% reduction in runtime on TTT-Discover/Qwen3-8B are large enough to be practically meaningful. The decomposition into Guideline Memory and Strategy Memory is also sensible, and the component ablations indicate that both contribute to the final performance.

Critical weaknesses
  1. The main claim that TTME “outperforms TTT” is not sufficiently supported by repeated-run statistics. The evaluated search procedures are stochastic and highly path-dependent, yet the main tables report individual best-found scores without means, standard deviations, confidence intervals, or paired-seed comparisons. This is particularly important because several improvements over TTT are numerically very small (e.g., Erdős under Qwen3-8B and gpt-oss-120b). In the ThetaEvolve experiments, Appendix B explicitly indicates that the authors' w/o-TTT and TTME conditions use only seed 3407, whereas the original inference-only configuration contains multiple seeds. Without repeated runs, it is difficult to determine whether the reported differences reflect the adaptation method or ordinary search variance. This directly affects the paper's central conclusion.

  2. The comparison with TTT is not fully controlled end-to-end because the TTT numbers are primarily taken from the original papers rather than rerun alongside TTME under an identical implementation. Although the authors carefully preserve many framework-level settings, there remain implementation differences; for example, the gpt-oss-120b comparison uses the originally reported TTT result served through Tinker, whereas the authors' fixed-weight runs use vLLM. For a stochastic generative search, especially when some final score differences are very small, differences in serving/sampling implementations, seeds, or other implementation details may matter. Therefore, I find the paper's characterization of this as a “clean comparison” somewhat stronger than the experimental setup supports. A direct rerun of TTT and TTME using the same codebase, serving stack, initialization, and multiple paired seeds would substantially strengthen the paper.

Questions for authors

1.Can the authors report repeated runs for the main TTME-vs-TTT comparisons, ideally using at least 3–5 paired random seeds? In particular, what is the run-to-run variance of the best-found scores on Erdős, AC1, AC2, AC3, and Hadamard? Several of the reported improvements over TTT are small enough that this could materially change my assessment.

2.Can the authors rerun at least the Qwen3-8B TTT baseline in the same implementation used for TTME, with identical seeds, model checkpoint, inference backend, initial search state/archive, and candidate budget? If this has already been done, please clarify which results in Tables 1–2 are newly reproduced versus copied from the original papers and provide the corresponding paired comparison.

3.How sensitive is TTME to its memory design and update hyperparameters? In particular, were K=2K=2, the 24-guideline capacity, the 50/50 direct-vs-strategy-guided allocation, and the memory prompt templates selected using any of the five evaluation tasks? If so, how was hyperparameter selection separated from final evaluation? A clarification would help determine the extent to which the comparison with TTT reflects adaptation mechanism versus method-specific tuning.

4.For the FLOP comparison, could the authors provide the exact accounting procedure? It would be useful to know how prefill/decode FLOPs for the additional memory-maintenance, strategy-selection, and strategy-adaptation calls are calculated, and how the forward/backward FLOPs of LoRA-based TTT are estimated. The reported efficiency improvement is an important strength of the paper and deserves a more reproducible accounting.

6 comments

Thank you for your constructive review. We sincerely appreciate your recognition that the paper asks an important and timely question, and that TTME is substantially more computationally efficient while remaining competitive or better. We highly value your review and have carefully addressed each of your comments below. We hope our clarifications and additional experiments help resolve your concerns.

W1&Q1. Support the main TTME-vs-TTT comparison with repeated-run statistics.

Thank you for raising the question of run-to-run variation. We agree that it matters when the differences are small, and we now report repeated runs for all three model–framework settings. Each run contributes the best score found within that run, and we report the sample mean and SD across runs.

  1. ThetaEvolve, Distill-Qwen3-8B (three seeds for both methods). The RL rows use the per-seed scores reported by ThetaEvolve. The TTME rows use our runs with the same seeds.
TaskMethodSeed 42Seed 1234Seed 3407MeanSD
AC3 ↓\downarrowWith RL1.55511.55511.49301.49301.51501.51501.52101.52100.03150.0315
AC3 ↓\downarrowTTME1.45721.45721.45751.45751.46421.46421.4596\mathbf{1.4596}0.00400.0040
Hadamard ↑\uparrowWith RL0.57330.57330.57640.57640.55910.55910.56960.56960.00920.0092
Hadamard ↑\uparrowTTME0.57330.57330.57680.57680.57990.57990.5767\mathbf{0.5767}0.00330.0033
AC2 ↑\uparrowWith RL0.93990.93990.94690.94690.94650.94650.94440.94440.00390.0039
AC2 ↑\uparrowTTME0.94590.94590.94420.94420.94810.94810.9461\mathbf{0.9461}0.00200.0020

TTME has a better mean on all three tasks and a smaller SD. On AC3, every TTME seed scored better than every RL seed.

  1. TTT-Discover, Qwen3-8B (three TTT reruns). We reran TTT three times in the same codebase and serving stack as TTME.
TaskTTT (paper)TTT run 1TTT run 2TTT run 3TTT meanTTME
AC1 ↓\downarrow1.505251.505251.50579701.50579701.50457321.50457321.50659321.50659321.50565451.50565451.5043288\mathbf{1.5043288}
AC2 ↑\uparrow0.94720.94720.94483980.94483980.94640020.94640020.94665370.94665370.94596460.94596460.9492480\mathbf{0.9492480}
Erdős ↓\downarrow0.3809320.3809320.38098620.38098620.38092330.38092330.38094420.38094420.38095120.38095120.3809271\mathbf{0.3809271}

The TTME run scored better than all three TTT reruns on AC1 and AC2, and better than two of the three on Erdős.

  1. TTT-Discover, gpt-oss-120b (three TTME runs).
TaskTTT (paper)Run 1Run 2Run 3MeanSD
AC1 ↓\downarrow1.50287001.50287001.50274511.50274511.50326641.50326641.50359431.50359431.50320191.50320190.00042830.0004283
AC2 ↑\uparrow0.95910000.95910000.96078400.96078400.95857170.95857170.95808850.95808850.95914810.95914810.00143720.0014372
Erdős ↓\downarrow0.38087600.38087600.38086150.38086150.38086670.38086670.38086540.38086540.38086450.38086450.00000270.0000027

On Erdős, all three TTME runs improved on the TTT score. On AC2, the TTME mean matches the TTT score. On AC1, the TTME mean is slightly worse, within one SD.


W2 & Q2. Rerun TTT and TTME under the same implementation, and clarify which results in Tables 1–2 are reproduced.

Thank you for this suggestion.

Imported vs. reproduced. The TTT scores in Tables 1 and 2 are taken from the TTT-Discover and ThetaEvolve papers. All other TTT results come from our own runs: the three Qwen3-8B reruns in W1, the runtime and FLOPs in Table 3, and the analyses in Figures 4–5.

Qwen3-8B rerun under the same implementation. We reran TTT three times with the same setting as TTME. The TTME run scored better than all three TTT reruns on AC1 and AC2, and better than two of the three on Erdős (see table in W1). Under ThetaEvolve, we compare both methods at the same three seeds (see table in W1).

Why Tables 1–2 keep the published scores. At Qwen3-8B, the published TTT scores are better than the mean of our reruns on all three tasks, so comparing TTME with them is conservative. We will state these sources explicitly in the revision.

Q3. How sensitive is TTME to its memory design and update hyperparameters, and how were they selected?

Thank you for this question. We did not tune these hyperparameters by comparing scores on the evaluation tasks. They were set from simple design considerations, and the same values and prompt templates are used for all tasks and all models.

  • Update interval (K=2K=2). One step often yields too few improving children to distill a strategy, so we pool two steps per update.
  • Guideline capacity (24). An initial capacity of 12 evicted most new lessons in some early experiments, so we raised it to 24. This change was based on eviction, not on scores.
  • 50/50 allocation. An equal split is the simplest way to use both direct and strategy-guided generation for each parent.

Our results reflect one shared configuration, and we have not yet measured sensitivity to these choices. Our claim does not rely on them being optimal. We will add a sensitivity analysis in the revision.


Q4. Provide the exact FLOP accounting procedure.

Thank you for your good suggestion. The details are listed in follows.

  1. What is counted: We count candidate generation, memory-related LLM calls, and policy training. We exclude CPU evaluation of candidate programs, GPU idle time during evaluation, model loading, and queueing.

  2. FLOPs: Let NN be the number of weights in the attention projections, MLPs, and output layer. For PcP_c computed prompt tokens and OO output tokens, both candidate generation and memory calls cost Finference≈2N(Pc+O)+Fattention,F_{\mathrm{inference}} \approx 2N(P_c+O)+F_{\mathrm{attention}}, where the attention cost at token position tt is approximated by 4Ldt4Ldt (LL layers, attention width dd), using mean prompt and output lengths.

  3. Runtime: We measure the GPU time of each component directly: the duration of each training step, generation window, and memory call, times the number of GPUs used.

AuthorsOpenReview ↗

Follow-up discussion

Dear Reviewer gaXA,

We greatly appreciate your thorough and insightful review, which is invaluable in improving our work. We have tried our best to carefully address your concerns: we added repeated-run statistics, reran TTT under the same implementation as TTME, and clarified how the hyperparameters were selected and how FLOPs were counted.

We hope that these additional results and explanations have adequately addressed your concerns. Nevertheless, we remain fully prepared to provide further clarification or discussion on any outstanding points. If you find that our responses have resolved your concerns, we would be deeply grateful if you would consider revising your score in support of our submission. Thank you again for your thoughtful and valuable review!

Best Regards,

Paper5675 Authors

Reviewer_1ZtsOpenReview ↗
Rating
2
Confidence
3
Informativeness
2
Presentation
3
Soundness
2
Read review
Paper summary

The paper proposes Test-Time Memory Evolution (TTME), a fixed-weight alternative to online reinforcement learning in program-based scientific discovery. TTME maintains 2 external memories built from evaluated search attempts: Guideline Memory and Strategy Memory. It selects strategies from Strategy Memory and adapts them to individual parent programs, the resulting instructions guide half of the candidate generation. Evaluated within TTT-Discover and ThetaEvolve, TTME improves on the reported TTT results across five mathematical construction tasks and three models. The authors also report substantial lower runtime and FLOPs.

Critical strengths
  • Empirical gains: TTME improves upon both the reported TTT baselines and the authors’ fixed-weight search baselines across all nine model–task configurations.
  • Retain useful knowledge beyond the programs kept in the archive. Consolidate related notes into strategy families, and re-adapt them to a different parent is a sensible design.
  • Clear description and reproducibility.
Critical weaknesses
  • Contribution over existing reflective search is not isolated: ReEvo and related methods already compare programs, extract improvement suggestions, and accumulate reflections to guide later generations. TTMe’s distinguishing design maybe its strategy-family organization and parent-specific selection and adaptation. And the experiments do not compare against a reflection-based baseline in the same framework.
  • Small margins, with no variance information. The main TTT numbers are taken from the original papers, and others appear to come from single runs.
  • The TTT+TTME result is unexplained and weakens the ‘alternative’ framing. Table 4 shows TTT+TTME (0.944141) is worse than neither (0.944771): RL-updated model also performs the memory operations, the result may reflect degraded memory-writing quality rather than a real incompatibility between the 2 forms of adaptation.
Questions for authors
  • Under the same framework and base model, how does TTME compare with a simple accumulated-reflection baseline such as ReEvo-style?
  • Where do the TTT runtime/FLOP measurements in Table 3 and the validity and idea-combination statistics in Figures 4 and 5 come from: your own runs, released logs, or estimates? How do these relate to the published TTT scores in Tables 1 and 2? If new TTT runs were conducted, could their final scores be reported alongside TTME under matched generation and evaluation conditions, with any unavoidable infrastructure differences documented?
  • Can you clarify which TTT results or timings are reproduced and which were imported?
  • Can you provide the variance information or confidence intervals?
  • Table 7 lists one seed for the authors’ ThetaEvolve experiments. Could you clarify the number of independent runs and the aggregation procedure for all results in Tables 1, 2, 4, and 5? Please distinguish the best candidate within a run from the best result across runs. Could you report individual results and mean with standard deviation, or median and range, over at least three seeds for TTME and the fixed-weight baseline, and a representative TTT comparison where feasible?
  • How are the runtime/flops figures obtained? E.g. FLOP accounting procedure.
  • How is a “combination of ideas” identified in Figure 5? Were the labels produced by human inspection, an LLM, or a deterministic procedure, and was the same procedure applied to TTT and TTME? Concrete examples and a check of labeling reliability would help assess the claim that TTME improves the reuse of ideas across search branches.
  • What hardware, parallelism, and measurement or estimation procedure were used for Table 3? How much computation and time are spent on candidate generation, memory maintenance, strategy selection/adaptation, and RL updates? Could you include candidate evaluation time and show the cost required to reach a common score threshold?
5 comments

Thank you for your constructive review. We sincerely appreciate your recognition of the empirical gains of TTME, of its design that consolidates notes into strategy families and re-adapts them to new parents, and of the clear description and reproducibility of the paper. We highly value your review and have carefully addressed each of your comments below. We hope our clarifications and additional experiments help resolve your concerns.

W1&Q1. Isolate the contribution over existing reflective search by comparing with a reflection-based baseline in the same framework.

Thank you for your good suggestion. As requested, we added a ReEvo baseline. With gpt-oss-120b, TTME found better programs than this baseline on all three tasks.

TaskTTT-DiscoverReEvo-styleTTME
AC1 ↓\downarrow1.502871.502871.50561841.50561841.502745\mathbf{1.502745}
AC2 ↑\uparrow0.95910.95910.95225840.95225840.960784\mathbf{0.960784}
Erdős ↓\downarrow0.3808760.3808760.38093250.38093250.380862\mathbf{0.380862}

We further investigate why ReEvo falls short on these tasks, and how TTME differs. Two main differences follow.

  1. Verified vs. speculative: ReEvo asks the model to compare two programs and guess why the better one is better. TTME learns from parent-child changes that improved the score with verification.

  2. Single summary vs. separate entries: ReEvo compresses its long-term memory into a short summary. TTME keeps each strategy as a separate entry and updates it dynamically as new evidence arrives.


W2&Q4&Q5. Report variance information for the small margins, since most results come from single runs.

Thank you for raising this point. We now report repeated runs for all three model–framework settings. Each run contributes the best score found within that run, and we report the sample mean and SD across runs.

  1. ThetaEvolve, Distill-Qwen3-8B (three seeds for both methods). The RL rows use the per-seed scores reported by ThetaEvolve. The TTME rows use our runs with the same seeds.
TaskMethodSeed 42Seed 1234Seed 3407MeanSD
AC3 ↓\downarrowWith RL1.55511.55511.49301.49301.51501.51501.52101.52100.03150.0315
AC3 ↓\downarrowTTME1.45721.45721.45751.45751.46421.46421.4596\mathbf{1.4596}0.00400.0040
Hadamard ↑\uparrowWith RL0.57330.57330.57640.57640.55910.55910.56960.56960.00920.0092
Hadamard ↑\uparrowTTME0.57330.57330.57680.57680.57990.57990.5767\mathbf{0.5767}0.00330.0033
AC2 ↑\uparrowWith RL0.93990.93990.94690.94690.94650.94650.94440.94440.00390.0039
AC2 ↑\uparrowTTME0.94590.94590.94420.94420.94810.94810.9461\mathbf{0.9461}0.00200.0020

TTME has a better mean on all three tasks and a smaller SD. On AC3, every TTME seed scored better than every RL seed.

  1. TTT-Discover, Qwen3-8B (three TTT reruns). We reran TTT three times in the same codebase and serving stack as TTME.
TaskTTT (paper)TTT run 1TTT run 2TTT run 3TTT meanTTME
AC1 ↓\downarrow1.505251.505251.50579701.50579701.50457321.50457321.50659321.50659321.50565451.50565451.5043288\mathbf{1.5043288}
AC2 ↑\uparrow0.94720.94720.94483980.94483980.94640020.94640020.94665370.94665370.94596460.94596460.9492480\mathbf{0.9492480}
Erdős ↓\downarrow0.3809320.3809320.38098620.38098620.38092330.38092330.38094420.38094420.38095120.38095120.3809271\mathbf{0.3809271}

The TTME run scored better than all three TTT reruns on AC1 and AC2, and better than two of the three on Erdős.

  1. TTT-Discover, gpt-oss-120b (three TTME runs).
TaskTTT (paper)Run 1Run 2Run 3MeanSD
AC1 ↓\downarrow1.50287001.50287001.50274511.50274511.50326641.50326641.50359431.50359431.50320191.50320190.00042830.0004283
AC2 ↑\uparrow0.95910000.95910000.96078400.96078400.95857170.95857170.95808850.95808850.95914810.95914810.00143720.0014372
Erdős ↓\downarrow0.38087600.38087600.38086150.38086150.38086670.38086670.38086540.38086540.38086450.38086450.00000270.0000027

On Erdős, all three TTME runs improved on the TTT score. On AC2, the TTME mean matches the TTT score. On AC1, the TTME mean is slightly worse, within one SD.

W3. Explain the TTT+TTME result.

Thank you for pointing this out. We conducted a detailed analysis on the AC2 task. In TTT+TTME, the RL-updated model writes memory and generates candidate programs. We compared one run per method, each with Qwen3-8B, 50 search steps, and 25,600 candidate programs. Here a program counts as passed only if it finished within the time limit and passed verification.

AC2 outcomeTTME onlyTTT+TTME
Final best score (higher is better)0.949248\mathbf{0.949248}0.9441410.944141
Passed within the time limit75.6%\mathbf{75.6\%}48.9%48.9\%
Timed out19.8%\mathbf{19.8\%}38.7%38.7\%
Programs that improved on their parent1,915\mathbf{1{,}915}716716
Most-used strategyCubic interpolationSwitch to float64
Passed ↑\uparrow / timed out ↓\downarrow among candidates given that strategy90%/6%\mathbf{90\% / 6\%}12%/83%12\% / 83\%

In TTT+TTME, the RL-updated model performs both candidate generation and all memory operations. This creates two problems. First, RL is rewarded only for the programs it generates, so memory extraction, strategy selection, and adaptation receive no training signal and may degrade as the weights change. Second, the two mechanisms change each other’s inputs: memory changes the prompts that RL trains on, and RL changes the model that writes and reads memory. Combining the two forms of adaptation therefore may require a more careful design that coordinates them, which we leave to future work. Our claim in this work is only that TTME can serve as an alternative to TTT.


Q2&Q3. Clarify the source of the TTT measurements in Table 3 and Figures 4 and 5, and report the final scores of any new TTT runs.

Thank you for your careful reading. The TTT scores in Table 1 are taken from the TTT-Discover paper, and those in Table 2 from the ThetaEvolve paper. All other TTT measurements come from our own Qwen3-8B reproduction: the runtime and FLOPs in Table 3, the validity rates in Figure 4, and the idea-combination rates in Figure 5.

This reproduction is TTT run 1 in the Qwen3-8B table in W2. It uses the same codebase and evaluation settings as TTME. Its scores are below the published TTT-Discover scores. Figure 4 currently pairs validity rates from this run with the published best-found scores. In the revision, we will clarify them.


Q6. Explain how the runtime and FLOP figures are obtained.

Thank you for your good suggestion. The details are listed in follows.

  1. What is counted: We count candidate generation, memory-related LLM calls, and policy training. We exclude CPU evaluation of candidate programs, GPU idle time during evaluation, model loading, and queueing.

  2. FLOPs: Let NN be the number of weights in the attention projections, MLPs, and output layer. For PcP_c computed prompt tokens and OO output tokens, both candidate generation and memory calls cost Finference≈2N(Pc+O)+Fattention,F_{\mathrm{inference}} \approx 2N(P_c+O)+F_{\mathrm{attention}}, where the attention cost at token position tt is approximated by 4Ldt4Ldt (LL layers, attention width dd), using mean prompt and output lengths.

  3. Runtime: We measure the GPU time of each component directly: the duration of each training step, generation window, and memory call, times the number of GPUs used.

Q7. Explain how a combination of ideas is identified in Figure 5, and provide concrete examples.

Thank you for this good question. To identify combinations in Figure 5, an LLM lists the ideas in each task’s programs, and a deterministic script then labels each program. The same procedure is applied to TTT and TTME.

  • What counts as a combination:

    • The child keeps at least one idea from its direct parent.
    • The child has at least one idea the parent does not have.
    • That added idea already appeared at an earlier step in a program on another branch, that is, a program that is not one of the child's ancestors.
  • Identification steps:

    • Step 1: For each task, an LLM reads the programs generated by both TTT and TTME and lists the ideas they use.
    • Step 2: A fixed script checks each program for every idea in the list and counts combinations. The same script and counting rule are used for TTT and TTME.
  • Concrete examples of combinations. Each of the three TTME children below kept at least one idea from its parent and added an idea absent from that parent but seen earlier on another branch.

ExampleTaskParent →\to childIdeas retainedAdded idea (earlier occurrence)Score
1AC2 ↑\uparrows2r98 →\to s3r59Three-level upsampling, warm fillCubic interpolation (s0r113, step 0)0.92754→0.928370.92754 \to 0.92837
2aAC1 ↓\downarrows26r22 →\to s27r122Ternary search, warm startFFT convolution (s26r95, branch diverged at step 24)1.5046308→1.50453091.5046308 \to 1.5045309
2bAC1 ↓\downarrows27r122 →\to s28r152FFT convolutionSparse LP constraint matrix (s6r327, s27r23)1.5045309→1.50442711.5045309 \to 1.5044271

Q8. Report the cost breakdown behind Table 3, and the cost to reach a common score threshold including candidate evaluation time.

Thank you for this question. We report the AC1 breakdown below (Qwen3-8B, excluding candidate evaluation).

ComponentTTT hoursTTME hoursTTT EFLOPsTTME EFLOPs
Candidate generation5.92315.92315.28415.28414.93244.93244.39294.3929
Calls attributed to memory windows0.00310.00310.07250.07250.00220.00220.13320.1332
Policy training5.83595.83590024.531224.531200
Total11.7621\mathbf{11.7621}5.3565\mathbf{5.3565}29.4659\mathbf{29.4659}4.5261\mathbf{4.5261}

For TTME, calls attributed memory windows are operations: strategy selection, adaptation, extraction, and guideline distillation. Policy training accounts for most of TTT's cost.

Time to reach a common score. TTT and TTME evaluate the same number of candidates per step under the same per-program time limit (1,000 s for all three tasks). We take the worse of the two final scores as the common score and report the wall-clock time, including candidate evaluation, at which each method first reaches it.

TaskCommon scoreTTTTTME
AC11.505801.5058037.337.3 h14.914.9 h
AC20.944840.9448445.245.2 h7.77.7 h
Erdős0.3809860.38098640.740.7 h1.91.9 h

These times are longer than those in Table 3 because they are wall-clock times and include candidate evaluation.

Reviewer_uiExOpenReview ↗
Rating
2
Confidence
3
Informativeness
1
Presentation
3
Soundness
1
Read review
Paper summary

This paper studies whether external memory can replace test-time training (TTT) in LLM-based scientific discovery. It proposes Test-Time Memory Evolution (TTME) that keeps the model weights fixed and updates two memory modules during search. Guideline Memory distills execution and evaluation feedback into operational lessons. Strategy Memory extracts algorithmic ideas from modifications that improve on the parent program. The paper replaces the RL part of two TTT frameworks, TTT-Discover and ThetaEvolve, with TTME. It evaluates five mathematical construction tasks and show reduceed FLOPs and runtime.

Critical strengths
  1. The research question is clear and practical: can external memory replace test-time weight updates in scientific discovery? The paper tries to compare the two adaptation methods inside the same search framework, with the same verifier and candidate budget.

  2. TTME is plug-and-play and much cheaper.

Critical weaknesses

Overall, the paper does not convince me in the following aspects:

  1. Idea is not new. Before TTT, the main approach kept the model weights fixed and updated the context. For example, FunSearch and AlphaEvolve put good earlier programs into the prompt. These are already forms of memory. TTT-Discover and ThetaEvolve then proposed a different approach: they update the model weights. TTT-Discover describes methods such as AlphaEvolve as search with a frozen LLM, and it uses reinforcement learning at test time instead. However, this paper goes back to fixed weights plus memory.

Its main ideas also have much prior work. For Guideline Memory, Reflexion[1] store reflections and experience, ReEvo[2] adds reflections to evolutionary search, and ACE[3] keeps an "experience playbook" that it updates continuously. Strategy Memory extracts algorithmic ideas from modifications where the child program scores higher than its parent, which DeltaEvolve[4] uses similar idea.

  1. Experiments lack baseline. Reflexion, ACE, and ReEvo are all in the related work, but the experiments only ablate the two memory modules of TTME. Thus, the paper does not show that the TTME design is better than a standard memory mechanism.

  2. Scope is narrower. The original TTT-Discover paper covers four domains: mathematics, GPU kernels, algorithm competitions, and biology. This paper uses only mathematical construction tasks, but the title says "scientific discovery".

[1] Shinn et al., Reflexion: Language Agents with Verbal Reinforcement Learning, NeurIPS 2023.

[2] Ye et al., ReEvo: Large Language Models as Hyper-Heuristics with Reflective Evolution, NeurIPS 2024.

[3] Zhang et al., Agentic Context Engineering: Evolving Contexts for Self-Improving Language Models, ICLR 2026.

[4] Jiang et al., DeltaEvolve: Accelerating Scientific Discovery through Momentum-Driven Evolution, ICML 2026.

Questions for authors

Q1. Can you explain Table 4? The combination result is not as expected: TTT + TTME is worse than both TTT alone and TTME alone. For example, on AC2 the combination gets 0.9441. This is lower than TTT (0.9472) and TTME (0.949248), and close to the result without either method (0.944771). Similarly, In Table 5 on AC1, each memory alone gives almost no gain, but both memories together give 1.502745. Why does the gain appear only when you use both? I think this more likely shows that the results are very noisy. Is this difference larger than the seed variance?

Q2. TTME does not need access to model weights. Can you show results with a closed API model, where TTT is not possible?

6 comments

Thank you for your constructive comments. We sincerely appreciate your recognition that our research question is clear and practical, and that TTME is plug-and-play and much cheaper than test-time training. We highly value your review and have carefully addressed each of your comments below. We hope our clarifications and additional experiments help resolve your concerns.

W1. Clarify the contribution and novelty relative to prior fixed-weight, memory-based methods.

Thank you for the careful reading and for pointing out the connections to prior work. We would like to highlight our core contribution and comparison with prior works as follows.

  • Our core contribution: We would like to clarify that our core contribution is a controlled study of TTT and TTME for scientific discovery. Existing TTT methods (e.g., TTT-discover and ThetaEvolve) show promising performance compared with the fixed-weight search (e.g., AlphaEvolve). However, they changed both the adaptation method and search algorithm. Within their own frameworks, both papers show that test-time RL improves over fixed-weight search. These fixed-weight baselines, however, reuse only archived states and distill no lessons across attempts. We therefore ask whether a comparable improvement can be obtained without updating the weights: under the same discovery framework, can a fixed-weight model with evolving memory match or exceed TTT, and at what cost? We study it by keeping TTT framework fixed and replacing only the weight update with a memory update, so that the comparison isolates the adaptation mechanism. To the best of our knowledge, this is the first study to compare memory-based adaptation with test-time RL under the same discovery framework. We believe the question matters for recursive self-improvement, where a system improves from its own experience: whether that improvement must be written into the weights or can be kept in context determines its cost and whether it applies to models whose weights are not accessible.

  • Relation to prior memory methods: We agree that there are some memory-based works for evolutionary search. We would like to highlight the specialty of our method. Our design principle is to have memory serve the function that TTT serves in these frameworks. Every entry in TTME's strategy memory comes from a parent-child change that verifiably improved the score. This design serves the same purpose as the weight update in TTT, carrying what earned higher scores over to new parents, but through memory rather than weights. This is different from previous works. For example, ReEvo reflects on why one of two randomly paired programs is better, and DeltaEvolve records changes at generation time, before they are evaluated.

We will add this discussion to the revised manuscript to clarify our contribution and its relationship to the prior works you mentioned, including Reflexion, ReEvo, ACE, and DeltaEvolve.


W2. Compare the performance of TTME with a standard memory baseline.

Thank you for your good suggestion. As requested, we added a ReEvo baseline. With gpt-oss-120b, TTME found better programs than this baseline on all three tasks.

TaskTTT-DiscoverReEvoTTME
AC1 ↓\downarrow1.502871.502871.50561841.50561841.502745\mathbf{1.502745}
AC2 ↑\uparrow0.95910.95910.95225840.95225840.960784\mathbf{0.960784}
Erdős ↓\downarrow0.3808760.3808760.38093250.38093250.380862\mathbf{0.380862}

We further investigate why ReEvo falls short on these tasks, and how TTME differs. Two main differences follow.

  1. Verified vs. speculative: ReEvo asks the model to compare two programs and guess why the better one is better. TTME learns from parent-child changes verified to improve the score.

  2. Single summary vs. separate entries: ReEvo compresses its long-term memory into a short summary. TTME keeps each strategy as a separate entry and updates it dynamically as new evidence arrives.


W3. Extend the evaluation beyond mathematical construction tasks.

Thanks for your good suggestion. To extend the evaluation beyond mathematics, we ran TTME on the single-cell RNA denoising task from TTT-Discover.

Specifically, we use TTME with gpt-oss-120b to search for a denoising program on pancreas data and evaluate it on two held-out datasets, PBMC and Tabula. The score is the mean of normalized MSE and normalized Poisson scores. Higher is better. The results are presented in the following table. The first three rows report published scores from the TTT-Discover paper. As the table shows, TTME achieves results comparable to TTT-Discover.

MethodPBMC ↑\uparrowTabula ↑\uparrow
MAGIC (best human)0.640.640.640.64
OpenEvolve0.700.700.710.71
TTT-Discover0.710.710.73\mathbf{0.73}
TTME0.72\mathbf{0.72}0.710.71

Q1. Explain the results in Table 4 and Table 5.

1. Explain why TTT + TTME is worse than both TTT alone and TTME alone.

We conducted a detailed analysis on the AC2 task. In TTT+TTME, the RL-updated model writes memory and generates candidate programs. We compared one run per method, each with Qwen3-8B, 50 search steps, and 25,600 candidate programs. Here a program counts as passed only if it finished within the time limit and passed verification.

AC2 outcomeTTME onlyTTT+TTME
Final best score (higher is better)0.949248\mathbf{0.949248}0.9441410.944141
Passed within the time limit75.6%\mathbf{75.6\%}48.9%48.9\%
Timed out19.8%\mathbf{19.8\%}38.7%38.7\%
Programs that improved on their parent1,915\mathbf{1{,}915}716716
Most-used strategyCubic interpolationSwitch to float64
Passed ↑\uparrow / timed out ↓\downarrow among candidates given that strategy90%/6%\mathbf{90\% / 6\%}12%/83%12\% / 83\%

In TTT+TTME, the RL-updated model performs both candidate generation and all memory operations. This creates two problems. First, RL is rewarded only for the programs it generates, so memory extraction, strategy selection, and adaptation receive no training signal and may degrade as the weights change. Second, the two mechanisms change each other’s inputs: memory changes the prompts that RL trains on, and RL changes the model that writes and reads memory. Combining the two forms of adaptation therefore may require a more careful design that coordinates them, which we leave to future work. Our claim in this work is only that TTME can serve as an alternative to TTT.

2. Explain why the AC1 gain in Table 5 appears only with both memories, and whether it exceeds the seed variance.

In the following table, We repeated each configuration three times with gpt-oss-120b. It reports the best AC1 score of each run (lower is better).

ConfigurationRun 1Run 2Run 3MeanSD
w/o TTT1.505971.505971.504521.504521.505461.505461.505321.505320.000730.00073
Guideline only1.505981.505981.505881.505881.506371.506371.506081.506080.000260.00026
Strategy only1.505641.505641.503761.503761.503861.503861.504421.504420.001060.00106
Full TTME1.502751.502751.503271.503271.503591.503591.503201.503200.000430.00043

The gain of full TTME is not due to run-to-run noise. The mean gap between Full TTME and w/o TTT (0.0021) is about three times the larger SD (0.00073), and every Full TTME run scored better than every w/o TTT run.

Strategy Memory alone does help; Guideline Memory helps only in combination. With three runs, Strategy-only improves on w/o TTT on average (1.50442 vs. 1.50532). Guideline-only gives no gain on its own (1.50608), but adding it to Strategy Memory further improves the mean from 1.50442 to 1.50320. Strategy Memory supplies the ideas that improve the score, and Guideline Memory helps implement them correctly.


Q2. Show results with a closed API model, where TTT is not possible.

We ran TTME on AC2 and Erdős using GPT-6 Luna. The model generated candidate programs and updated only the external memory through API calls. The results are presented in the following table. TTME achieves comparable or better performance compared with TTT-Discover. This shows a practical advantage of TTME over TTT: because it does not require access to model weights, it can be applied to closed models served only through an API, where TTT is not possible.

MethodModelAC2 ↑\uparrowErdős ↓\downarrow
TTT-Discovergpt-oss-120b0.95910.95910.3808760.380876
TTMEGPT-6 Luna0.9592\mathbf{0.9592}0.380863\mathbf{0.380863}
AuthorsOpenReview ↗

Follow-up discussion

Dear Reviewer uiEx,

We greatly appreciate your constructive feedback, which has been invaluable in improving our work. We have tried our best to carefully address your concerns regarding the core contribution, discussion of previous works, and extra experiments and analysis.

We hope that these additional results and explanations have adequately addressed your concerns. Nevertheless, we remain fully prepared to provide further clarification or discussion on any outstanding points. If you find that our responses have resolved your concerns, we would be deeply grateful if you would consider revising your score in support of our submission. Thank you again for your thoughtful and valuable review!

Best Regards,

Paper5675 Authors

From OpenReview, CC BY 4.0. Authors as signed there; shown unedited.

Related papers

Loading the map…

Discussion (0)

Sign in to comment.