- Rating
- 3(+1)
- Confidence
- 2
- Informativeness
- 3
- Presentation
- 3
- Soundness
- 3
Read review
Paper summary
This paper introduces HarnessTTT, a test-time RL framework that trains a harness proposer while keeping the executor frozen. The proposer generates executable harnesses and learns from discovery rollouts. Experiments on 13 scientific discovery tasks show strong performance and promising cross-executor transferability.
Critical strengths
-
Compelling motivation. Learning to improve the search procedure rather than the executor itself is an interesting direction, especially given the potential for reusable and transferable harnesses.
-
Strong empirical results. The method achieves competitive results across diverse scientific discovery tasks, with promising improvements when transferring learned harnesses to stronger executors.
Critical weaknesses
-
Incomplete compute comparison. HarnessTTT incurs additional costs from proposer inference, RL training, harness validation, and tool execution. Comparisons under matched GPU-hours, token budgets, and total evaluation costs would make the results more convincing.
-
Insufficient RL ablations. The paper lacks component-wise ablations of its RL design, such as near-tie filtering and max-sharpening, making it difficult to identify which components contribute most to the improvements.
-
Limited evidence of generalization. While cross-executor replay demonstrates harness transferability, it remains unclear whether the trained proposer can generalize its harness optimization ability to unseen tasks.
Questions for authors
see Weaknesses
3 comments
2. Component-wise RL Ablations (W2)
W2: Insufficient RL ablations. The paper lacks component-wise ablations of its RL design, such as near-tie filtering and max-sharpening, making it difficult to identify which components contribute most to the improvements.
We ablated the main components of the objective, including standard RLOO. We used three seeds on four tasks, with the frozen Qwen3.5-9B executor, K = 32 proposals per round, B = 20, and 15 rounds instead of the 20 rounds of Table 1. The last row is a frozen proposer that receives no weight or context update (the control of Figure 3c).
| Objective | AC1 ↓ | AC2 ↑ | CP26 ↑ | Hadamard ↑ |
|---|---|---|---|---|
| Full method | 1.5049 ± 0.0022 | 0.9268 ± 0.0141 | 2.4043 ± 0.1238 | 0.7656 ± 0.1474 |
| RLOO only (α = 0) | 1.5137 ± 0.0024 | 0.9002 ± 0.0045 | 2.1851 ± 0.3334 | 0.4956 ± 0.0266 |
| RLOO + std normalization | 1.5160 ± 0.0004 | 0.8961 ± 0.0110 | 2.0047 ± 0.0113 | 0.5428 ± 0.0175 |
| Without near-tie filtering (δ = 0) | 1.5141 ± 0.0019 | 0.9086 ± 0.0036 | 1.9804 ± 0.0520 | 0.5292 ± 0.0337 |
| Without uτ (normalize by |bτ|) | 1.5052 ± 0.0012 | 0.9362 ± 0.0060 | 2.4142 ± 0.0735 | 0.5271 ± 0.0159 |
| Frozen proposer (no update) | 1.5131 ± 0.0007 | 0.9066 ± 0.0091 | 1.9988 ± 0.1558 | 0.5406 ± 0.0201 |
Max-sharpening and near-tie filtering are each necessary. Removing either one worsens performance on all four tasks, in most cases to roughly the level of the frozen proposer. Removing the reference uτ leaves performance mostyly unchanged within the seed spread on three tasks, but lowers it on Hadamard. The gains therefore come from the combination of components designed for best-of-search with indirect, noisy rewards (§3.1), rather than from RL training alone.
3. Generalization to Unseen Tasks (W3)
W3: Limited evidence of generalization. While cross-executor replay demonstrates harness transferability, it remains unclear whether the trained proposer can generalize its harness optimization ability to unseen tasks.
HarnessTTT is instance-wise test-time learning for scientific discovery. Each campaign trains the proposer on a single open problem, and the goal is the best solution to that problem, as in TTT-Discover, which likewise trains per problem. Specializing the proposer to the instance at hand is therefore the intended behavior rather than a limitation. We do not claim that the trained proposer transfers to unseen tasks; the phrase "across tasks and rounds" in §2 is inaccurate, since each campaign trains its own proposer, and we will correct it. What we do claim transfers is the harness. Because the harness is explicit and separate from the executor's weights, a harness learned with the 9B executor improves the mean score of all five unseen executors in our replay study (Figure 4). Transferring learned harnesses to GPT-5.5 also matches or exceeds the strongest reported result on six of the eleven non-quarantined tasks (Table 3). Training a proposer that generalizes across problems is a natural extension, which we leave to future work.
Dear Reviewer,
Thank you again for your time and thoughtful comments on our paper. We understand this is a busy period, and we appreciate your time and effort.
As the discussion period closes in less than two days, we wanted to check whether our response has addressed your concerns. If there are any remaining questions or points that would benefit from further clarification, we would be very happy to discuss them before the deadline.
Thank you very much for your consideration.
Best regards, The Authors
We thank the reviewer for the constructive feedback. In response, we report a full GPU-hour accounting with same-executor baselines at comparable compute, add a three-seed component-wise ablation of the RL objective, and clarify the intended scope of generalization. All new results will be included in the revised version.
1. Compute Comparison (W1)
All model costs are counted in GPU-hours. We report H100 GPU-hours from our job logs. They cover proposer inference (including harness validation and tool repair), executor rollouts (including generated-tool execution and paired-control rollouts), and LoRA training. All models in these campaigns are served on our own GPUs, so token usage is reflected in GPU time. We did not aggregate campaign-level token totals and therefore use GPU-hours as the common budget across methods. Evaluation is bounded per rollout by B = 20 full evaluator calls for the 9B campaign and B = 5 for the 27B campaign; subsampled probe() calls are capped separately (Appendix D). Ranges below are across tasks, so components need not sum exactly to the totals. Appendix E describes the paired-control rollouts incorrectly; we will correct it in the revised version.
We ran ShinkaEvolve and GEPA with the same frozen Qwen3.5-9B executor and evaluator at about 400 GPU-hours each. Both evolve programs rather than harnesses, so we match total GPU-hours rather than evaluator calls per trajectory. We list OpenEvolve from Table 1 with its measured cost.
At comparable compute, HarnessTTT finds the best solution on all three tasks, and on CP26 and Hadamard it uses less compute than ShinkaEvolve and GEPA. GEPA did not improve on the initial program on AC2. Across Table 1, the OpenEvolve runs used about 360 GPU-hours per task, somewhat below HarnessTTT-9B's 370–510, and HarnessTTT improves on them on all eleven tasks (nine of nine excluding the quarantined LLM-SQL and Txn). Our new 15-round campaigns (Section 2) use about three quarters of a full campaign (≈ 280–380 GPU-hours), similar to or below OpenEvolve's ≈ 360, and the full method's three-seed means still exceed OpenEvolve's Table 1 results on all four tasks tested there.