- Rating
- 1
- Confidence
- 2
- Informativeness
- 2
- Presentation
- 2
- Soundness
- 2
Read review
Paper summary
The paper introduces FEED, a test-time training method for scientific discovery. Built upon TTT-Discover's PUCT search, FEED replaces its entropic objective with quality-weighted self-training on valid programs plus a DPO-style loss that prefers valid over invalid programs from the same prompt. Using Qwen3-8B and a one-hour budget on six tasks, FEED reaches target scores faster than TTT-Discover and OpenEvolve. Ablations show that the joint objective outperforms its single-objective variants mainly on CVRP-ACO and AC2, and that parameter updates speed up discovery over a fixed-model PUCT baseline on most but not all tasks.
Critical strengths
- The evaluation uses a wall-clock budget that includes generation, evaluation, and training time, with a detailed time breakdown, which makes the comparison between learning-based and search-only methods fair.
Critical weaknesses
- FEED keeps TTT-Discover's PUCT search unchanged and only replaces its learning objective. Each component of the new objective has been explored in prior work, including reward-weighted self-training (e.g., AWR [1], ReST [2]), same-prompt valid-invalid preference learning (e.g., EvoTune [3]), and joint SFT + DPO learning (e.g., Iterative RPO [4]). The paper itself notes that EvoTune suggested combining self-training and preference learning as future work. The contribution is therefore mainly a specific combination of existing objectives.
- FEED shows a clear advantage on only two of six tasks (CVRP-ACO and AC2) compared against TTT-Discover, and the differences on the other tasks are small. On the remaining tasks, the evidence is much weaker. On AC1, the two methods are practically identical in both hits and RMST. On 32-circle, FEED reaches the threshold in one more run, but the paper notes that the mean trajectories end at similar scores. On MaxDet, the difference comes down to a single run of FEED reaching the threshold. On OBP, FEED's final mean score is lower, yet both methods reach the threshold in only one of five runs, and Table 11 and Figure 6 suggest that the gap in the mean is driven largely by one seed. Moreover, the hit and RMST results in Table 1 depend on a single threshold per task, and the paper does not explain how these thresholds were chosen.
- TTT-Discover is not evaluated under its original setting (512 rollouts per step for 50 steps). Instead, it is run with 32 rollouts per step under a one-hour budget, while keeping hyperparameters such as the learning rate unchanged. As a result, FEED's advantage is shown only in this low-budget regime, and even within this regime, TTT-Discover may underperform because its configuration was not adapted.
[1] Peng et al., Advantage-Weighted Regression: Simple and Scalable Off-Policy Reinforcement Learning, arXiv 2019
[2] Gulcehre et al., Reinforced Self-Training (ReST) for Language Modeling, arXiv 2023
[3] Surina et al., Algorithm Discovery With LLMs: Evolutionary Search Meets Reinforcement Learning, CoLM 2025
[4] Pang et al., Iterative Reasoning Preference Optimization, NeurIPS 2024
[4]
Questions for authors
- All methods here run generation and evaluation in sequence, but search-only methods like OpenEvolve can run them in parallel, since generation uses the GPU and evaluation uses the CPU. On 32-circle, for example, PUCT spends about 39 minutes generating and 21 minutes evaluating (Table 10), so running them in parallel could let it evaluate many more candidates in an hour. Did you try this? I wonder whether FEED would still be faster in that case.
- The preference loss only uses failed programs as negatives, and low-scoring valid programs are not treated as failures. I understand that quality differences among valid programs are partly handled by the SFT weights, but those weights are all positive, so weaker valid programs are never pushed down. Did you try using low-scoring valid programs as negatives?
- FEED changes several things relative to TTT-Discover, not only the loss. It also reuses past programs for training and filters updates with a KL check. The ablations only test the loss terms, so I could not tell which change actually leads to the gains. Have you tried adding the historical pool or the KL-based update control to TTT-Discover?
5 comments
Q1. Practical contribution and evaluation budgets
Thank you for emphasizing the importance of practical contribution. We chose a common one-hour horizon to assess discovery efficiency under limited resources and make repeated comparisons across six tasks feasible within our available compute.
Our evaluation also includes 180-minute experiments on 32-circle across multiple model sizes and families (Section 3.3.2). Table 4 reports RMST reductions relative to TTT-Discover of 34.7%, 9.8%, and 17.6% with Qwen3-8B, Qwen3-14B, and Qwen3-32B, respectively. Figure 5 additionally shows higher mean best-so-far scores than TTT-Discover at the three-hour endpoint with both Ministral and OLMo. These results provide evidence of discovery-efficiency and solution-quality benefits beyond the one-hour main setting.
FEED’s methodological contribution changes how evaluated programs enter the learning objective. TTT-Discover derives updates from entropic reward advantages; even a valid program can receive a negative reward-derived advantage when its reward is low relative to other candidates. FEED gives every selected valid program a positive self-training term and adds explicit preference comparisons against failures from the same prompt. Shared quality weights control both terms, while historical and unpaired valid programs remain reusable positive examples. The component ablations and augmented TTT-Discover comparisons in Table 12 of our previous response provide empirical support for this learning design.These experiments support FEED’s methodological contribution and practical value across the evaluated settings.
Q2. Statistical evidence and learning-rate coverage
Thank you for following up. The original CVRP-ACO comparison provides statistical support for FEED’s improvement: the mean one-hour cost difference is −0.0881 (FEED minus TTT-Discover), with a 95% pointwise Welch confidence interval of [−0.1464, −0.0298]. FEED also achieves 5/5 hits versus 0/5 for TTT-Discover. Broader practical benefits appear on 32-circle and AC1, where FEED achieves 5/5 hits versus 0/5 for OpenEvolve. In Table 11, FEED achieves the best observed mean on all three tasks when each method uses its best observed learning rate within the tested grid. We hope this clarifies the statistical evidence and the scope of the additional comparison.
Thank you for the detailed explanation and additional experiments. I still have some remaining concerns as follows: