- Rating
- 3
- Confidence
- 3
- Informativeness
- 2
- Presentation
- 3
- Soundness
- 2
Read review
Paper summary
AutoScaffold uses a frozen teacher LLM to adapt scaffolding during student reinforcement learning. Every K student updates, the teacher inspects reward statistics and failed trajectories, proposes class-level strategies and instance-level hints, adjusts injection probabilities, and assesses class-level edits on held-out tasks. The additional guidance is supplied during training only; evaluations remove the scaffolds. The framework is evaluated on ALFWorld, search-based question answering, mathematical reasoning, and GPU kernel generation with several student models and RL algorithms. The paper reports improvements over corresponding published baselines, including 93.9% ALFWorld success and 45.9% average QA exact match. An ALFWorld ablation examines interactive evidence access, held-out validation, adaptation frequency, static guidance, and injection strength.
Critical strengths
S1. A coherent and useful automated RL-training intervention. The teacher is allowed to diagnose failures using queried experience, update class-level and instance-level assistance, and empirically test class-level revisions instead of using a single hand-designed scaffold. This is a meaningful systems contribution addressing how assistance should evolve with student capabilities.
S2. Breadth and a convincing within-domain analysis. The evaluation spans embodied task execution, tool-assisted QA, mathematics, and kernel generation, with testing performed without scaffolds. The ALFWorld ablations are particularly informative: removing query tools, held-out validation, or continual adaptation leads to substantial performance degradation, and the case studies show specific scaffold revisions over training.
Critical weaknesses
W1. The cross-domain improvement claims are not supported by uniformly matched baselines. The main ALFWorld and QA tables explicitly reuse several baseline numbers from previous papers, as do the math comparisons. These scores may follow similar setups, but they do not provide the same level of controlled comparison as rerunning the corresponding RL baselines under identical code, training steps, data, evaluation, and seeds. This matters because some claimed gains are modest: Qwen3-4B math ties SAGE at 70.0%, and several KernelBench gains are only 1–4 points. The strong ALFWorld ablation does not fully resolve this concern for all domains.
W2. Additional teacher information and computational budget confound the efficiency and automation claims. AutoScaffold uses a strong GPT-5.5 teacher, extra rollouts for held-out scaffold validation, and privileged instance-level hints; the mathematical instance-level mechanism follows QuestA in prepending part of a reference solution. Consequently, improvements may partly reflect additional supervision and validation computation rather than adaptive teacher reasoning alone. Comparing 350 versus 1,100 student training steps to QuestA does not establish a proportional end-to-end efficiency improvement. The paper quantifies per-update validation time, but lacks comparisons under a matched total teacher + student compute budget and against simpler teacher-generated static or rule-based adaptive scaffolds. Further, class-level validation measures immediate scaffolded success, not directly the eventual scaffold-free benefit.
Questions for authors
Q1. What is the total end-to-end cost per run, including teacher calls, held-out validation rollouts, student training, and any generation of verified instance hints? How does AutoScaffold compare with baselines at a fixed GPU-hour or wall-clock budget rather than only fixed student update count?
Q2. How much gain remains if the teacher generates a static scaffold once, or if a simple predetermined rule controls injection and revision while using the same teacher-generated content and validation budget?
6 comments
Rebuttal for Reviewer vdNG (2/4)
W2(a): Do the gains come from the strong GPT-5.5 teacher rather than adaptive teacher reasoning?
We demonstrate that one of the main contributions of AutoScaffold is adaptive teacher reasoning, rather than a strong model itself. Our ablation study shows that, with the same GPT-5.5 teacher, only AutoScaffold's adaptive use of the teacher improves over training without a teacher.
To separate the teacher's knowledge from how the teacher is used, we compare ablation variants on ALFWorld that all use the same GPT-5.5 teacher.
| How the same GPT-5.5 teacher is used | Success (%) |
|---|---|
| AutoScaffold: the teacher chooses which failures to inspect, tests its edits on held-out tasks, and keeps revising the scaffold | 93.9 |
| The teacher sees only a fixed random sample of 12 failures (w/o query tools) | 85.3 |
| Same as above, plus 15 rounds of self-refinement | 85.8 |
| The teacher's edits are deployed without testing (w/o validation) | 79.8 |
| The teacher writes the scaffold once, and it is then frozen | 78.1 |
| Reference: GiGPO, trained without any teacher | 86.7 |
All rows except AutoScaffold perform at or below GiGPO. When the teacher cannot choose which failures to inspect, cannot test its edits, or cannot revise the scaffold after writing it, the same strong teacher gives no gain over training without a teacher, and sometimes lowers performance. The gains over the baselines therefore come from adaptive teacher reasoning, not from the strength of the teacher model alone.
W2(b): Do the gains come from privileged instance-level hints?
The gains do not come from privileged instance-level information. ALFWorld, Search QA, and KernelBench use no privileged instance-level information at all. In math, AutoScaffold uses exactly the same privileged information as QuestA and still surpasses it, so the gains come from how the teacher uses this information.
In ALFWorld, Search QA, and KernelBench, AutoScaffold uses no instance-level hints; all guidance is class-level text the teacher writes from the student's trajectories. The gains in these three domains therefore cannot come from privileged information. In math, AutoScaffold's instance-level hints are the same reference-solution prefixes that QuestA uses. The difference is that the teacher decides how much of the prefix to show for each group of problems and when to add class-level guidance instead (Appendix B.2). With identical privileged information, AutoScaffold surpasses QuestA (62.4 vs. 60.6) in one-third of the student steps. This also shows, in a domain other than ALFWorld, that the gains come from the teacher's adaptive scaffolds.
W2(c): How much do validation and teacher calls add to the end-to-end cost?
Including teacher calls and validation, AutoScaffold takes 76.1 h vs. 129.1 h for SAGE, and 79.4 h vs. 196.2 h for QuestA.
We agree that step counts alone do not give the end-to-end cost. The table below separates student training from teacher calls and validation (full comparison in Q1):
| Setting | Method | Steps | Student training | Teacher + validation | Total |
|---|---|---|---|---|---|
| Qwen3-4B | SAGE | 500 | 129.1 h (incl. per-step hinting) | – | 129.1 h |
| Qwen3-4B | AutoScaffold | 500 | 54.2 h | 21.9 h | 76.1 h |
| Nemotron-1.5B | QuestA | 1,100 | 196.2 h | – | 196.2 h |
| Nemotron-1.5B | AutoScaffold | 350 | 63.7 h | 15.7 h | 79.4 h |
Teacher-generated static and rule-based scaffolds are discussed in Q2.
W2(d): Does validation measure the eventual scaffold-free benefit?
We agree that validation does not directly measure the eventual scaffold-free benefit. It measures the current student's success with and without a candidate scaffold. However, the scaffolds that validation selects do lead to higher scaffold-free success later, as shown in Fig 2.
Directly measuring the eventual scaffold-free benefit would require running every candidate branch of the RL run to the end, which is too costly. Instead, we use validation to check whether a proposed scaffold is effective for the current student, and take this as evidence that it will benefit later training.
At each update, validation compares three conditions on held-out tasks at the current checkpoint: the current scaffold, the proposed scaffold, and no scaffold (Section 3.2). It tells the teacher whether a proposed scaffold helps the current student and whether the student still needs support. The dynamic adaptation study (Section 4.4, Fig. 2) confirms that scaffolds selected this way benefit later training: at four branch points during training, we continue training for 20 steps with the newly selected scaffold, the previous scaffold, or no scaffold, and then evaluate without scaffolds. At every branching point, the newly selected scaffold gives the highest scaffold-free success.
Rebuttal for Reviewer vdNG (3/4)
Q1(a): What is the total end-to-end cost per run?
Below we report the end-to-end training time and performance of AutoScaffold and every baseline, including teacher calls and validation.
All times are measured from our own runs (W1), not estimated, and the revision adds this comparison as Table 8. All runs use 4×H200, except the Nemotron-1.5B setting, which uses 8×H200 for both AutoScaffold and QuestA; wall-clock ratios therefore equal GPU-hour ratios.
| Domain | Method | Total time | Time relative to AutoScaffold | Result |
|---|---|---|---|---|
| ALFWorld | GiGPO | 11h34m | 0.74× | 85.7 |
| ALFWorld | AutoScaffold | 15h36m | 1.00× | 93.9 ± 0.3 |
| Search QA | GiGPO | 14h41m | 0.71× | 41.5 |
| Search QA | Skill0 | 19h05m | 0.92× | 40.4 |
| Search QA | AutoScaffold | 20h44m | 1.00× | 45.9 ± 0.6 |
| KernelBench | StitchCUDA | 11h58m | 0.70× | 59 / 50 |
| KernelBench | AutoScaffold | 17h10m | 1.00× | 63 / 54 |
| Math (Qwen3-4B) | SAGE | 129.1h | 1.70× | 69.7 |
| Math (Qwen3-4B) | AutoScaffold | 76.1h | 1.00× | 70.0 ± 0.3 |
| Math (Nemotron-1.5B) | QuestA | 196.2h | 2.47× | 60.6 |
| Math (Nemotron-1.5B) | AutoScaffold | 79.4h | 1.00× | 62.4 ± 0.7 |
Results: success rate (ALFWorld), average exact match (Search QA), Level 1 / Level 2 correctness (KernelBench), average accuracy (math).
AutoScaffold is faster than SAGE because SAGE refreshes its hints at every training step, while AutoScaffold is light and effective, which revises scaffolds only once every 10 steps; it is faster than QuestA because it can provide adaptive scaffold to help the student learn more efficiently , which needs 350 instead of 1,100 training steps. Instance hints add no generation cost (see W2(b)). The teacher API costs $2–6 per run.
Q1(b): How does AutoScaffold compare at a fixed time budget?
With the same training time, AutoScaffold still leads GiGPO by 5.2 and 3.6 points on ALFWorld and Search QA, and in math it matches or exceeds SAGE and QuestA in less time.
On ALFWorld and Search QA, we continued training GiGPO (both domains) and Skill0 (Search QA) from their final checkpoints until they used the same wall-clock as AutoScaffold:
| Method | ALFWorld | Search QA |
|---|---|---|
| GiGPO (standard training) | 85.7 | 41.5 |
| GiGPO (extended to AutoScaffold's time) | 88.7 | 42.3 |
| Skill0 (standard training) | – | 40.4 |
| Skill0 (extended to AutoScaffold's time) | – | 40.6 |
| AutoScaffold | 93.9 ± 0.3 | 45.9 ± 0.6 |
Extra training time helps GiGPO on ALFWorld (+3.0), but AutoScaffold still leads by 5.2 points. On Search QA, it helps little (+0.8 for GiGPO, +0.2 for Skill0), and AutoScaffold leads by 3.6 points.
Rebuttal for Reviewer vdNG (1/4)
We thank the reviewer for the careful reading and the positive assessment. We highly appreciate the reviewer finding AutoScaffold "a meaningful systems contribution addressing how assistance should evolve with student capabilities," highlighting the breadth of the evaluation across embodied tasks, tool-assisted QA, mathematics, and kernel generation, and finding the ALFWorld ablations "particularly informative," since removing query tools, held-out validation, or continual adaptation each leads to substantial degradation.
In summary of our responses:
Changes are highlighted in blue in the revised PDF.
W1(a): Are the baselines compared under identical code, data, evaluation, and seeds?
We reran the closest baselines in our codebase with the same student, training data, evaluation, seeds, and hardware as AutoScaffold. AutoScaffold leads GiGPO by +8.2 and +4.4 points on ALFWorld and Search QA, matches SAGE, and surpasses QuestA.
The StitchCUDA results on KernelBench were already from our own runs. The revised tables report both the published and the reproduced numbers, with mean ± std over 3 seeds (except KernelBench; see W1(b)).
W1(b): How should the small margins in math and KernelBench be read?
On Qwen3-4B, AutoScaffold matches SAGE at 59% of its wall-clock time; on Nemotron-1.5B, it surpasses QuestA in one-third of the steps; on KernelBench, it improves all six metrics over StitchCUDA
On Qwen3-4B, AutoScaffold reaches 70.0 ± 0.3 vs. 69.7 for SAGE, which we describe as matching rather than exceeding. Its value there is that a teacher procedure shared across four domains matches a hinting strategy designed for math, at lower wall-clock time(Q1). On Nemotron-1.5B, AutoScaffold reaches 62.4 ± 0.7 vs. 60.6 for QuestA, in 350 instead of 1,100 steps. On KernelBench, the three runs are used to compute Pass@3: a task counts as solved if any of the three generated kernels is correct (or, for Fast1, faster than the PyTorch reference), so this table has no per-seed standard deviation.