- Rating
- 2(+1)
- Confidence
- 3
- Informativeness
- 2
- Presentation
- 1
- Soundness
- 1
Read review
Paper summary
This paper introduces ScienceBuddy, an interactive scientific workspace with recursive-in-recursive self-improvement (R²SI). The inner loop fixes the task model and uses a separate auxiliary model to revise the execution harness from trajectories and feedback; the outer loop fixes the selected harness and updates the task model through GRPO using fresh, independently verified rollouts. Evaluated on 548 training tasks and 70 test tasks across seven scientific categories, the two-cycle system improves accuracy from 43.3% to 71.0%. Crossed training/evaluation harnesses and update-order controls investigate whether the gains arise from execution changes, model learning, or their scheduling.
Critical strengths
+) The crossed training/evaluation design is useful. Comparing learned models under a common evaluation harness helps distinguish improvements in model learning from gains due to test-time program replacement.
+) The paper integrates scientific tools, executable analysis, and inspectable trajectories into a practical workspace. The substantial reported improvement provides encouraging evidence for combining harness adaptation with model training.
Critical weaknesses
-) The methodological novelty over existing harness–model co-optimisation approaches is insufficiently established, and some claimed contributions extend beyond the evaluated pipeline.
-) The main findings rely on single search/training trajectories, while several controls introduce sampling or compute differences. Consequently, the robustness and attribution of the reported gains remain uncertain.
Questions for authors
-
The central methodological idea is to alternate between harness adaptation and model updates. However, closely related approaches, including Co-Harness, SIA, and MetaClaw, already combine harness or skill adaptation with weight updates. Table 14 distinguishes this work through full-program revisions, fresh-rollout GRPO, and execution/training/scheduling analyses. These are potentially useful differences, but the experiments primarily compare against component-only baselines and internal variants. Such comparisons establish the benefit of combining components within ScienceBuddy, but do not establish the added value over existing joint optimisation methods. What is the essential methodological distinction, and what evidence isolates its benefit?
-
The strongest empirical claims concern improved learning under an adapted harness and the advantage of alternating updates. However, Sections 3.4–3.6 and Appendix D clarify that these comparisons generally involve one search/training path per condition, followed by repeated evaluations of fixed checkpoints. Evaluation repeats cannot quantify variability in program search or RL training. Moreover, matching proposal and optimiser-update counts does not match sampled trajectories or tokens; the complete trace-only control changes the refill cap; and Table 13 reports substantially different auxiliary-model usage for full-program and prompt/skill editing. The authors appropriately acknowledge these limitations, but they leave uncertainty about which gains are reproducible and attributable to the intended intervention. Are independent learning runs available, and what are the actual sampling and auxiliary-compute costs for the central comparisons?
-
The contribution statement includes deriving executable tasks and evaluation rubrics from scientific collaboration. Yet Sections 2.2–2.4 and Appendix C specify that the experiments use supplied tasks and fixed verifiers, without environment augmentation, while Appendix H explicitly separates real researcher interactions from the evaluated learning process. The quantitative experiments instead use simulated user replies. This supports a narrower claim about alternating optimisation on predefined scientific tasks, but does not validate the full interaction-to-learning pipeline. Which parts of this claimed contribution have been implemented and evaluated, and should the contribution statement be narrowed accordingly?
5 comments
Response to Reviewer 52kQ (Part 2/2)
Q2: The main findings rely on one search/training path per condition, and some controls differ in sampling or auxiliary compute. Are independent learning runs available, and what are the actual sampling and auxiliary-compute costs for the central comparisons?
A2: The paper reports one learning run per condition, which is common for full self-improvement runs (e.g., the main SWE-bench result of the Darwin Gödel Machine comes from a single run), and it contains two complete, independent alternating runs whose results agree. The central comparisons are matched in the number of attempts used for training, and the remaining differences in sampling and auxiliary compute either produce no gradient or favor the condition that R²SI is compared against.
1. Independent learning runs.
- The paper contains two complete alternating runs. Besides the main run, the trace-only run (Appendix D.2) performs its own harness search and RL starting from M0. The two end at 71.0 and 62.86, both far above the single-component baselines (47.62) and the one-shot schedule (54.29).
- Both runs show the same pattern. Every RL phase improves accuracy (main: 38.57→60.95 and 64.29→71.0; trace-only: 49.52→51.43 and 61.43→62.86), and so does returning to harness adaptation after the first RL phase (60.95→64.29; 51.43→61.43).
- The effect of the training harness holds under both evaluation harnesses (+6.19 and +9.52; Table 11).
2. Sampling cost. Each harness proposal is evaluated on 16 Train interactions, and each RL update trains on 8 task groups × 8 attempts (Table 3).
| Condition | Harness proposals | Train interactions for harness search | RL updates | RL attempts used in updates |
|---|---|---|---|---|
| Harness-only | 30 | 480 | 0 | 0 |
| RL-only | 0 | 0 | 40 | 2,560 |
| R²SI (alternating) | 14 | 224 | 30 | 1,920 |
| One-shot (all proposals, then all RL updates) | 14 | 224 | 30 | 1,920 |
- Alternating and one-shot have the same training signal. Both use the same proposals, interactions, updates, and attempts used in updates. The number of sampled trajectories can differ, because each update keeps sampling batches until it has 8 groups with mixed rewards. The extra groups have identical rewards within the group, so their GRPO advantage is zero (Eq. 4) and they contribute no gradient.
- Each single-component baseline receives more budget for its own component than R²SI.
- The trace-only comparison favors the trace-only run. Its refill cap was removed, so it always collects the full 8 mixed-reward groups per update, whereas the main run uses fewer groups when 4 batches do not suffice. Normal feedback still reaches higher accuracy (71.0 vs. 62.86).
3. Auxiliary compute. The alternating and one-shot schedules use the same auxiliary model and the same 14 proposals. Table 13 compares two harness-adaptation conditions with the model fixed. They have identical budgets (7 proposals, 112 Train interactions, and the same per-candidate limits of 3 evidence reads, 2 repair attempts, and 6 auxiliary calls; Appendix B.1) and similar numbers of auxiliary requests (201 vs. 182). This comparison does not involve the joint optimization.
We have added the budget table and these clarifications to Appendix B.
Q3: The contribution statement includes deriving tasks and rubrics from collaboration, but the experiments use supplied tasks, fixed verifiers, and simulated replies. Which parts are implemented and evaluated, and should the statement be narrowed?
A3: Yes, we have narrowed the contribution statement to what is implemented and evaluated: the ScienceBuddy workspace, R²SI on given scientific tasks with fixed verifiers and simulated user replies, and the controlled comparisons. Constructing tasks and rubrics from researcher collaboration and augmenting the environment belong to the general framework, which we now describe separately in Appendix I. The revised contributions are:
- ScienceBuddy, a scientific research workspace with a pluggable harness interface, on which all experiments run.
- R²SI, which alternates whole-program harness revision guided by interaction feedback with GRPO training on verifier rewards.
- Controlled comparisons that separate three effects: the harness at test time, the training harness on the model, and the order of the updates.
We now present the researcher cases in Appendix H as illustrations of the workspace, and we have removed "returning the updated system to researchers" from the claims. We have also restructured Sections 3-4 so that the main text describes exactly the configuration that was run, which removes most of the qualifiers that made the paper hard to follow.
We thank the reviewer again for the detailed feedback. We hope these clarifications and revisions address your concerns, and we would kindly ask you to reconsider your score. Please feel free to let us know if you have any further questions.
Gentle Reminder
Dear Reviewer 52kQ,
We sincerely appreciate the time and effort you dedicated to reviewing our paper. Your thoughtful questions and insightful feedback have been extremely beneficial. In response to your queries and concerns, we have prepared detailed answers .
Should there be any further points that require clarification or improvement, please know that we are fully committed to addressing them promptly.
Thank you once again for your invaluable contribution to our research.
Warm Regards, The Authors
Response to Reviewer 52kQ (Part 1/2)
We sincerely thank the reviewer for the careful reading and for recognizing the value of our crossed training/evaluation design. We address the three questions point by point below; all changes in the revised manuscript are highlighted in blue. Below, H0/M0 are the initial harness/model, Hk/Mk those after cycle k, and all numbers are accuracy (%) on the 70 held-out Test tasks.
A1: The key methodological difference is that R²SI updates the harness and the model with different learning signals: the harness is revised only from interaction feedback (execution traces and user replies), without access to verifier outcomes or reference answers, while the model is trained with RL only on verifier rewards. In addition, R²SI measures how the harness used during training changes what the model learns, separately from how the harness helps at test time. Co-Harness, SIA, and MetaClaw show that combining harness adaptation with weight updates is effective, and R²SI builds on this line of work.
1. How R²SI differs from these methods.
2. Evidence for each difference.
3. Why we compare design choices within one setting. Tasks, backbones, and budgets differ across these papers, so their numbers are not directly comparable (Appendix G). We therefore compare each design choice with its alternative in one setting, as these works do for their own designs: Co-Harness compares its joint rounds with a hand-designed harness and harness-only evolution, SIA compares harness-only with harness-plus-weight updates, and MetaClaw compares skill-only adaptation with its full system. We follow the same protocol and add an RL-only baseline, the crossed comparison, and the schedule comparison.
Following your suggestion, we have revised Table 14 and Appendix A to state these differences and their evidence.