- Rating
- 3
- Confidence
- 4
- Informativeness
- 3
- Presentation
- 3
- Soundness
- 3
Read review
Paper summary
The paper proposes InScribe, a two-stage reinforcement learning recipe for improving coding and terminal agents using training-time scaffolds. In Stage I, the policy generates a task-conditioned harness specification and solves the task under that harness, with separate loss channels for harness generation and task execution. Stage II continues training on tasks that a frozen Stage-I probe fails to solve, mixing clean records with records containing fixed teacher diagnoses informed by failed trajectories and private training verifiers. On Qwen3.5-9B, the full method improves Terminal-Bench 2.0 and SWE-bench Verified by 7.49 and 6.74 percentage points over the authors' 300-update stock-harness RL baseline, with additional transfer results on function-calling and interactive-agent benchmarks.
Critical strengths
-
The paper highlights an important aspect of agent RL: failures can arise from limitations of the execution harness as well as the policy, yet harness optimization has received comparatively less attention during RL training. By allowing the policy to adapt its own harness, InScribe makes execution strategies part of the learning process. Combining this adaptation with temporary diagnostic guidance provides a useful way to improve the policy.
-
The reported improvements are promising and extend beyond a single evaluation harness. Table 3 shows consistent checkpoint ordering under three fixed harnesses, including two absent from training. The broader evaluations also suggest transfer beyond terminal and repository-repair tasks. These results support the claim that the trained weights improve, even though they do not by themselves identify what was internalized. The ablations of Table 4 are reasonable and the results also validate the effectiveness of InScribe recipe.
-
The paper is well writen and easy to follow.
Critical weaknesses
-
The evaluation is limited to a single dense model, Qwen3.5-9B. Although the results demonstrate transfer across tasks and evaluation harnesses, generalization across model sizes and architectures remains untested. Experiments with larger models and alternative architectures, such as mixture-of-experts models, would help establish whether the benefits of InScribe extend beyond the current student model.
-
The baseline comparisons could be more comprehensive. The experiments compare InScribe with stock-harness RL and several ablations, but do not include the other harness-evolution methods discussed in the introduction and related work. Comparisons with representative methods under a common student model and task setting would help establish InScribe's advantages over existing approaches.
Questions for authors
-
Section 4.1 states that the main evaluations omit harness generation, would allowing InScribe to generate harnesses at inference, as it does during training, can further improve performance across the benchmarks?
-
Table 3 shows that the generated harnesses still underperform the hand-designed Terminus2 and OpenHands harnesses. To what extent does the fixed schema and runtime inherited from the stock harness constrain the scope of adaptation? Could a more flexible harness-generation strategy close this gap or outperform a carefully engineered harness?
-
As an exploratory question, does the policy exhibit any distinctive behaviors at inference after InScribe training? For example, when deployed under a fixed harness without an explicit harness-generation step, does it spontaneously construct or adapt tools to the task? Qualitative examples would be interesting for understanding what the policy has internalized.
4 comments
Thanks for the careful explaination. It is a pity that we can not have detailed generalization experiments results and more methods comparison due to heavy training budget.
Accord to the results in section 4.3 and Table 7 of Appendix B.1, although InScribe do help improve the final performance compared to naive Stock-harness RL (16.85 -> 19.85 vs 16.85 -> 17.23), there is still large gap between stock harness and generated harness ( 19.85 vs 23.22). In other words, there is still large room for the "Internalizing".
Another question about the transfer across differencr harness. As shown in Table 3, why training in a stock based harness would improve most in the Terminus 2 harness? Would it be related to the capability of the base harness?
Thank you for the follow-up. With Stock fixed, InScribe scores 19.85% compared with 17.23% for Stock-harness RL. Enabling harness generation raises the same InScribe checkpoint’s score to 23.22%, while it lowers the Stock-harness RL checkpoint’s score by 1.13 percentage points. Writing a useful harness for the task and using it effectively are themselves capabilities the model learns through training.
We believe the harness plays different roles in training and evaluation. On TB2, the base model scores 20.97% with Terminus2 and 16.85% with Stock. With the same Stage I recipe, the checkpoint trained from Stock Bash scores 26.22% under fixed Terminus2, compared with 21.35% for the checkpoint trained from Terminus2 itself (Table 3). We believe Stock Bash’s simpler interaction protocol may provide an easier starting point. Through repeated practice writing and executing task-specific harnesses, the model can gradually learn strategies that help it use more demanding interfaces such as Terminus2 more effectively. This may help explain why the gains are larger under Terminus2.
Thanks for the further feedback. I have no further concerns and will maintain my score of 3 for acceptance.
Response to Reviewer PcyK
Thank you for your careful review and for asking about generalization and what the policy can do under a fixed harness.
W1
We focused our experiments on the dense Qwen3.5-9B student and limit our empirical claims to that model given the time and computing resources required for long-horizon RL. We are keen to evaluate InScribe on larger mixture-of-experts (MoE) models. With harness generation disabled and teacher diagnoses removed at evaluation, InScribe still improves performance on Terminal-Bench 2.0 and SWE-bench Verified under fixed evaluation harnesses (Tables 2 and 3). In future work, we plan to explore training InScribe on larger MoE models and test whether these gains carry over.
W2
In Table 4, InScribe and Stock-harness RL start from the same Qwen3.5-9B initialization, receive 300 optimizer updates each, and are evaluated under the same fixed benchmark harnesses. Relative to this control, InScribe improves Terminal-Bench 2.0 by 7.49 percentage points and SWE-bench Verified by 6.74 percentage points.
We cite HASE and EvoHarness-RL in Section 2 for their related harness adaptation mechanisms. HASE was evaluated on classification and program search, and EvoHarness-RL on household and web interaction tasks. A head-to-head comparison on our terminal and repository tasks would require adapting both methods and running additional long-horizon RL training with the same student, task set, and training budget. For context, one full InScribe training run used 5,695 A100 GPU-hours and launched 64,121 CPU sandbox episodes (Table 9). We would like to explore adapting both methods to our environments for such a comparison in future work.
Q1
We report the inference-time harness generation results in §4.3 and Table 7 of Appendix B.1. With the Stock harness, InScribe scored 54.47 ± 0.70%. Generating a task-specific harness at inference increased the score to 57.27 ± 0.70%, a gain of 2.80 percentage points. Together with the 3.37-percentage-point gain on Terminal-Bench 2.0, this shows that inference-time harness generation improves performance over the Stock harness on both of our primary benchmarks.
Q2
Our generated harnesses use the configuration format and runtime of the Stock harness. For each task, the model can add executable Python tools and adjust the tool-use rules and history retention. As detailed in Appendix C.1 (Table 11), verification and recovery rules are rendered as instructions for the model. For example, a generated configuration can ask the model to run tests after a change. The model still has to invoke a tool to run them. This limits which execution behaviors a generated harness can control directly.
Table 3 compares the same model across complete harnesses. A controlled comparison would be needed to measure how much the fixed configuration and runtime contribute to the gap. Giving the model a choice of mechanisms that the runtime can execute for each task, such as context summarization for long tasks or automatic self-checks before submission, could help narrow the gap. Whether this approach can match or surpass a hand-designed harness remains to be tested with the same checkpoint, tasks, verifier, and total inference budget.
Q3
We observed the policy construct a task-specific tool in a pilot TB2 evaluation of an earlier Stage II checkpoint. The evaluation used a fixed Terminus2 harness with harness generation and teacher hints disabled at inference. In cancel-async-tasks, the policy used the existing terminal interface to build and run a task-specific tool that checked concurrency and cleanup callbacks. After the tool reported an empty cleanup list, the policy revised the implementation and its checks, reran the tool, and passed the benchmark verifier. We describe this case in Appendix C.4 (A task-specific check under a fixed harness) and refer to it in Section 4.3. How often this behavior occurs across different checkpoints remains to be studied.