- Rating
- 3(+2)
- Confidence
- 3
- Informativeness
- 1
- Presentation
- 2
- Soundness
- 2
Read review
Paper summary
This paper proposes PAST, which extends TTT-Discover with an archive of historical training states, allowing optimization to resume from earlier checkpoints and create new branches. Its ProgressEMA selector combines historical best generation performance with observed parent-to-child reward improvements to choose the starting point for the next update. The method is evaluated on three automated discovery tasks and shows improvements over continuing exclusively from the latest checkpoint.
Critical strengths
The paper studies test-time training for automated discovery, which is an important research direction, particularly regarding how to allocate adaptation budgets effectively during search.
Critical weaknesses
W1. Limited conceptual novelty beyond checkpoint selection and branching. PAST retains TTT-Discover’s solution-side search and training objective, while introducing an archive of historical checkpoints and a rule for selecting which checkpoint to resume. This is a potentially useful extension, but the methodological advance appears incremental. The paper does not clearly establish what new algorithmic principle is introduced beyond checkpoint reuse, branching, and heuristic allocation.
W2. The presentation obscures a relatively simple underlying procedure. The method introduces numerous overlapping constructs, i.e., continuation states, two archives, pending and active nodes, direct and historical-fresh batches, and several within-round archive versions before providing a compact operational explanation. Although these details specify the implementation, their presentation makes the central contribution unnecessarily difficult to identify. A minimal algorithm first, followed by bookkeeping details, would improve clarity.
W3. The scope of the contribution is insufficiently delineated. The narrative alternates between presenting PAST as a general framework, advocating complete adapter–optimizer state reuse, and proposing ProgressEMA as an allocation strategy. These are distinct claims with different evidential requirements. The paper does not consistently distinguish which component constitutes the main methodological contribution, which components are inherited, and which are implementation choices. This weakens both the positioning and the interpretation of the results.
W4. Resource efficiency and long-horizon scalability remain insufficiently evaluated. The actual-rollout-matched comparisons are useful, but rollout counts do not capture generated-token volume, verifier execution cost, training cost, or end-to-end runtime. Moreover, retaining 50 trainable states requires approximately 127 GB in the reported configuration. Without fixed-storage-budget experiments or archive retention and pruning strategies, the practicality of the approach for longer discovery runs remains unclear.
Questions for authors
Please refer to weakness.
9 comments
Response to Reviewer eEUn(2/2)
2. Presentation Clarity and Method Organization (W2 & W3)
We appreciate the reviewer's suggestion to improve presentation clarity. We agree that introducing detailed execution chronology and state bookkeeping too early can obscure the central algorithmic idea.
In the revised manuscript, we will restructure Section 3 to begin with a compact, four-step operational overview of PAST:
- Evaluate: Generate and verify a batch from the current policy and update the shared Solution Archive. This policy is the root initially and the child produced by the preceding round thereafter.
- Select: Choose the starting state for the next update from the Policy Archive.
- Recontextualize: Reuse the current batch if the current policy is selected; otherwise restore the selected historical state and collect fresh on-policy candidates under the updated Solution Archive.
- Continue & Branch: Update the selected policy using its on-policy data and save the resulting child in the Policy Archive as a new continuation option.
Following this high-level workflow, we will present the bookkeeping details—including pending/active nodes, direct/historical-fresh batches, and within-round archive updates—in a dedicated subsection. This separates the core algorithmic concept from implementation mechanics.
3. Computational Costs and Archive Storage (W4)
The resource measurements for nine rollout-matched trajectories across three methods and three tasks, covering 47,616 verifier-evaluated candidates, including prompt and completion token counts, cumulative verifier execution time, and NPU-hours for training and state restoration, are shown in Table 2.
At matched actual-rollout budgets, the results support the effectiveness of PAST's allocation of candidate-generation and policy-update budgets:
(1) PAST performs 50 optimizer updates on each task, compared with 92, 85, and 71 updates for the extended Current-only baselines on CP26, AC1, and Erdős, respectively.
(2) PAST reduces measured training/restoration NPU-hours by 50.2%, 35.1%, and 20.7%, and total prompt-plus-completion tokens by 7.3%, 6.4%, and 3.2%, respectively.
Table 2. Computational Resource Accounting .
| Task (Rollouts) | Method | Training rounds | Tokens (M) Prompt / Completion | Train + Restore (NPU·h) |
|---|---|---|---|---|
| CP26 (5,888) | Current-only | 92 | 11.50 / 61.68 | 33.19 |
| UG-TTT | 92 | 12.54 / 47.70 | 50.05 | |
| PAST | 50 | 11.82 / 56.01 | 16.54 | |
| AC1 (5,440) | Current-only | 85 | 22.88 / 46.44 | 26.19 |
| UG-TTT | 85 | 20.15 / 35.79 | 45.84 | |
| PAST | 50 | 19.74 / 45.17 | 17.01 | |
| Erdős (4,544) | Current-only | 71 | 7.28 / 42.15 | 19.02 |
| UG-TTT | 71 | 7.56 / 39.46 | 38.81 | |
| PAST | 50 | 7.81 / 40.04 | 15.09 |
Regarding storage, Appendix F reports about 2.537 GB per recoverable state (~126.84 GB for 50 states). We retain complete archives primarily to support detailed retrospective analyses of historical continuation and Policy–Solution interactions. Crucially, a retrospective analysis of historical-state accesses reveals that only 8, 6, and 8 distinct historical states are needed to cover all recorded historical continuation selections on CP26, AC1, and Erdős, respectively. Retaining at most eight historical states plus the current state corresponds to an estimated footprint of approximately 22.83 GB.They indicate substantial potential for selective state retention.
Conclusion
In summary, PAST’s methodological contribution should be assessed as a historical continuation-search protocol coupled to an evolving Solution Archive. The comparisons above separately evaluate the historical-continuation decision, the allocation strategy within that framework, and the optimizer-state restoration choice, providing concrete experimental evidence for these distinct claims.
We respectfully ask the reviewer to reconsider the assessment of PAST's novelty and technical contribution in light of these clarifications and additional evidence.
We sincerely appreciate your valuable comments and constructive suggestions,which have helped us improve our work. We have made every effort to address your concerns in ourresponse. As the author-reviewer discussion period is coming to an end, we would be grateful to hear your thoughts on whether our revisions have sufficiently addressed the points you raised. We look forward to continuing the discussion and welcome any further feedback.
Response to Reviewer eEUn(1/2)
The review underestimates PAST's algorithmic novelty by narrowly characterizing it as a combination of checkpoint selection, branching, and heuristic allocation, and overlooking the defining feature of PAST—dynamic Solution–Policy coupling—which addresses a fundamentally different search paradigm from the solution-side search employed by TTT-Discover.
The core contribution of PAST lies in the dynamic coupling between the Solution Archive and the Policy Archive, a mechanism that is particularly important for automated scientific discovery. Solutions serve both as discovery outcomes and as contexts for subsequent proposal generation. As new discoveries accumulate, historical policies encounter evolving contexts that may change their proposal behavior and potential for further learning. When reactivating a historical policy, PAST preserves the current Solution Archive, allowing the policy to generate fresh candidates and continue learning under the latest discovery context. In turn, historical policy continuation can produce new discoveries that further advance the Solution Archive. This bidirectional interaction constitutes PAST's central algorithmic innovation: the joint exploration of solution space and historical policy space, rather than merely storing checkpoints or applying a heuristic for checkpoint selection.
To make the contribution boundaries explicit, we distinguish the core method, allocation strategy, state-restoration mechanism, and inherited components:
Furthermore, to verify the effectiveness of PAST's individual components, we compare PAST against Frozen-history and Latest-only-extra to isolate the contribution of historical policy continuation, and also compare ProgressEMA against Capability-only, Progress-only, Uniform, and ScorePUCT to assess the effectiveness of its continuation-allocation strategy. PAST with ProgressEMA achieves the best observed upper bound among all compared methods. The comparisons with Frozen-history and Latest-only-extra demonstrate the additional value of allowing historical policies to resume training, while those with alternative selectors establish the practical benefit of combining capability and delayed progress signals to guide continuation allocation.
Table 1. Historical-continuation controls and selector ablations on AC1 at 3,200 actual rollouts.
In the experiments above, Frozen-history allows historical policies to generate candidates under the current Solution Archive and update the archive, while training only the latest policy; Latest-only-extra allocates additional sampling to the latest policy. PAST, Frozen-history, and Latest-only-extra follow identical per-round batch-collection and policy-update schedules, with the same number of policy updates and actual rollouts. Frozen-history updates the latest policy using on-policy data generated by that policy. Capability-only (ρ = 1) always resumes the eligible checkpoint with the highest observed on-policy batch-best reward, implementing best-observed-checkpoint continuation within our protocol.