- Rating
- 2
- Confidence
- 3
- Informativeness
- 2
- Presentation
- 2
- Soundness
- 2
Read review
Paper summary
This paper proposes SWE-AaaV, an adaptive verification framework for rewarding coding agents when fixed unit tests provide incomplete or overly restrictive assessments of candidate patches. A verification agent constructs paired executable tests conditioned on candidate and reference implementations, while a deterministic harness checks their consistency and evidence of issue resolution. Asynchronous verification and a windowed FIFO mechanism integrate these rewards into reinforcement learning. Evaluation covers 618 candidate patches from SWE-bench Verified, using unit-test outcomes as proxy labels alongside sampled human adjudication and ablations. With stronger verifier backbones, the framework reports AUC improvements of 4.55-5.98 points over model-based baselines. In RL experiments, SWE-AaaV improves SWE-bench Verified performance by 1.8 points over standard unit-test rewards, with ~15% additional training wall-clock training time.
Critical strengths
-
The experimental design is thorough and comprehensive, covering offline verifier evaluation, ablation studies, and RL results.
-
Efficiency is well taken into account. The asynchronous reward computation alleviates the slowdown introduced by the complex verification system, showing that the authors have carefully considered the potential challenges of deploying this work in large-scale RL training.
Critical weaknesses
-
The more accurate reward does not yield a clear performance gain in the whole RL pipeline. In Figure 4(a), the accuracy curves of SWE-AaaV and the Test Verifier are nearly identical over the first 500 RL steps, and only in the final 200 steps does a mere 1.8 percentage gap emerge. This inevitably raises doubts about the importance of the reward system claimed in this paper — perhaps the less precise reward provided by the Test Verifier is already sufficient?
-
Lack of novelty. Prior work has already explored using agents as verifiers in RL training, yet this paper lacks comparative experiments against such similar methods [1]. Furthermore, regarding asynchronous reward computation, many mature implementations already exist in industry, e.g., the fully async mode of verl [2]; the authors should supplement experiments demonstrating the performance advantages of their proposed method over these existing implementations.
[1] The Verification Horizon: No Silver Bullet for Coding Agent Rewards
[2] https://verl.readthedocs.io/en/latest/advance/fully_async.html
Questions for authors
- In the full RL pipeline, could you roughly estimate how many samples' correctness is misjudged by the Test Verifier but correctly judged by SWE-AaaV? This could be estimated via manual sampling or LLM-as-a-judge.
4 comments
Thank you for the response. The 1.6–2.4 pp improvement from RL is indeed relatively modest, and integrating asynchronous rewards into an existing RL framework appears to contribute more to the practical implementation of RL systems than to methodological novelty. While I acknowledge the authors' substantial contributions to RL infra, I think the academic novelty of the work remains somewhat limited. Therefore, I maintain my original score.
Thank you for engaging in this discussion! Our previous response focused too heavily on RL infra and did not sufficiently highlight the core verification mechanism.
We distinguish two contributions:
- Methodology: generate candidate-conditioned paired tests, validate their evidence through mandatory execution checks, and determine rewards from candidate execution.
- Engineering: integrate this costly verification process into an existing asynchronous RL framework.
Experiment results. 1. The 1.6-2.4 pp gain is an additional improvement over the best performance achieved with test verifier. Relative to the initial policy, SWE-AaaV achieves an absolute RL improvement of up to 10.6 points. 2. The ablations also support the methodological contribution: "w/o validation" retains the same information and execution access, providing a general baseline of agentic verifier. Removing this mandatory validation reduces AUC by 7.89 points on Qwen and 3.11 points on DeepSeek.
To our knowledge, SWE-AaaV is the first framework to separate paired executable evidence generation from mandatory evidence validation for coding agent rewards. We respectfully ask you to reconsider the methodological novelty and contribution.
Dear Reviewer AdKM,
Thank you again for your time and for continuing the discussion. As the author discussion period ends on November 18 (AoE), we briefly summarize our responses to your concerns below.
-
Reward corrections during training. Manual adjudication estimates that SWE-AaaV corrected the test verifier on 8.2% of rollouts in the easiest training interval and 20.6% in the hardest. An independent RL run also reproduced the gain over the test verifier (+2.2 pp).
-
Methodological contribution. Our latest reply explains how SWE-AaaV generates paired tests for each candidate patch and validates the evidence before assigning a reward. With reference and execution access held fixed, removing validation lowered AUC by 7.89 points on Qwen and 3.11 points on DeepSeek.
If you have a moment before the discussion closes, we would appreciate your thoughts on whether this clarifies the methodological contribution.
Author Response Overview: Thank you for recognizing the breadth of our experiments and training-efficiency work. Below, we add training-time reward adjudication and clarify the relationship to prior verifiers and asynchronous systems.
W1 & C1: Downstream RL Gains and Reward Corrections
To avoid additional test-execution overhead affecting the efficiency measurements, we did not run Test Verifier during SWE-AaaV RL. We instead conducted a post-hoc audit across three curriculum intervals of increasing difficulty. For each interval, we sampled 500 rollouts and ran Test Verifier on their candidate patches offline. We then sampled 10 cases from each disagreement direction within that interval for manual adjudication.
Table R1. Training disagreement rates and SWE-AaaV correctness in audited disagreements. Verdict order: SWE-AaaV / Test Verifier.
Results: Using direction-balanced audit accuracy, the estimated reward-correction rates of 8.2%, 12.0%, and 20.6%, showing a positive association with curriculum difficulty. In the final interval, manual adjudication favors SWE-AaaV in 7/10 cases in each direction. This pattern is also consistent with the late-stage RL gains. A plausible explanation is that complex issues make correct and comprehensive predefined tests harder to construct, while candidate-conditioned verification can target the particular patch. Moreover, an independent RL repetition yields 48.0% versus 50.2%, alongside the original 47.8% versus 49.6%. The two results per method imply a 1.6–2.4 pp gain across cross-comparisons.
W2: Novelty Relative to Prior Agentic Verifiers and Asynchronous Integration
Prior verifiers. In [1], the SWE quality judge assesses task–test alignment for training-data filtering, while the interactive and long-horizon evaluators are not applicable to SWE RL. Dockerless [2] uses read-only repository exploration to judge patches, but we could not locate a public release of its trained verifier. SWE-AaaV's distinguishing feature is paired executable evidence checked by a deterministic harness, with the validated outcome used directly as RL rewards.
Asynchronous integration. We build on an existing asynchronous RL framework and integrate asynchronous reward computation and windowed FIFO to support agentic verification. The systems contribution lies in adapting and evaluating these mechanisms for agentic rewards, whose multi-turn verification creates a substantially greater scheduling burden than a single-response LLM judge. Figure 4(b) quantifies the benefit of this integration within our implementation: relative step time falls from 2.00× to 1.40× with asynchronous rewards, then to 1.15× with windowed FIFO.
[1] Qwen Team. The verification horizon: No silver bullet for coding agent rewards. 2026.
[2] Wenhao Zeng, et al. Dockerless: Environment-free program verifier for coding agents. 2026.