acceptodds
Under review as a conference paper at ICLR 2027

VeriLoop Coder-E1: Toward Verifiable Recursive Self-Improvement through the VeriLoop Spiral

Abstract

Self-modifying coding agents may retain locally successful changes without establishing whether those changes improve subsequent error correction. We present VeriLoop, an evidence-governed framework that separates within-task verification from cross-task method inheritance. VeriLoop organizes the improvement process into six operations: evidence, falsification, inquiry, revision, verification, and verified internalization. Its dual-gate protocol first validates a correction within its declared scope and then conditions inheritance on subsequent independent evidence, while preserving provenance and explicit revocation paths. We instantiate this framework as VeriLoop Coder-E1, built on a frozen Qwen3.6-27B backbone with four detachable Surface Host adapters and an external Self-Harness. VeriLoop achieves 85.20% on SWE-bench Verified, 62.38% on SWE-bench Pro, 76.40% on Terminal-Bench 2.0, and 33.63% on DeepSWE. Three classification adapters improve performance on held-out capability endpoints, while a fourth provides uncertainty estimates. Their LoRA matrices contain 3,481,640 parameters, corresponding to 0.012895% of the backbone. Mechanism audits across 115 episodes document selective admission and intact control boundaries. Together, these results support the feasibility of a practical and reversible method-inheritance architecture, while not establishing that inheritance causally improves downstream task success.

Primary area:
transfer learning, meta learning, and lifelong learning
Keywords:
LLM Agents, Coding Agents, Self-Improving Agents, Evidence-Governed Adaptation, Agentic Harness

Reviews

Among:
Shape:
1234this paper, Bottom 1%
Figure 1. Mean review rating of the reviewed papers at ICLR 2027. This paper averages 1.00 over 3 reviews, above 0% of the others.
Reviewer_ak92OpenReview ↗
Rating
1
Confidence
2
Informativeness
2
Presentation
2
Soundness
1
Read review
Paper summary

The paper proposes VeriLoop, an evidence-governed framework that separates within-task verification from cross-task method inheritance for self-modifying coding agents. A verification gate handles local acceptance, and an inheritance gate conditions promotion on paired gains over later tasks, with provenance tracking and revocation. The system uses a frozen Qwen3.6-27B backbone, four LoRA adapters, and an external Self-Harness, and reports strong scores on four benchmarks together with a mechanism audit over 115 episodes.
The main concern is that the reported evidence does not yet include a direct instantiation of the inheritance gate, so the inheritance mechanism highlighted in the title and abstract appears to remain untested, and the current results seem to support the architecture rather than the claimed mechanism.

Critical strengths
  1. VeriLoop decouples local validation from cross-task promotion and keeps promoted methods conditional with provenance and rollback, while the frozen backbone and lightweight LoRA adapters make it a practical safety-control option rather than a retraining pipeline.
  2. The layered evaluation paradigm is a useful reference for the field. The paper checks whether internal control boundaries operate as designed through end-to-end benchmarks, independent adapter capability tests, and mechanism audits. It also separates architectural feasibility from causal performance gains and avoids pooling samples from different analysis units.
Critical weaknesses
  1. The inheritance gate does not appear to be instantiated in any reported experiment. No values are given for the thresholds in Eq. 2, and no paired gain or regression rate seems to be measured, so the mechanism highlighted in the title appears untested and the results seem to support the architecture rather than the claimed inheritance behavior.
  2. The threat model appears asserted rather than validated. No reported experiment demonstrates the gate rejecting a harmful candidate, both gates consume evidence from the same evaluation infrastructure so a systematic evaluator flaw could inflate the paired gains, and adversarial cases such as rules injected through poisoned trajectories are not discussed.
  3. The system results are difficult to attribute to the framework. There is no baseline with the same backbone and harness but VeriLoop disabled, no ablation removing the inheritance gate, the four benchmarks appear to be single runs without variance, and the comparator sets in Figure 5 are unexplained while DeepSWE falls below its displayed comparator.
  4. Several key formal objects are undefined. The estimators behind the LCB and UCB in Eq. 2, the predicates OpEquivalent and Revisable, and the bound update at line 19 of Algorithm 1 appear unspecified, which makes independent reimplementation of the gate difficult.
  5. Records appear incomplete and reproduction seems difficult at present. Mixture proportions are unrecorded for three of four surfaces, one sequence length is listed as Unknown, three benchmarks lack a reported contamination audit, the before model and chance levels in Table 2 are undefined, and no code or trajectories appear to be released.
  6. The protocol's applicability boundary is unexamined. The verification gate appears to rely on machine-checkable evidence such as tests and exit codes, and the paper does not discuss how it would operate in domains with weaker verifiers such as planning or open-ended tool use.
  7. Defensive writing appears to substitute disclosure for evidence. Nearly every claim is paired with a preemptive disclaimer, and these disclaimers cluster around the same untested core identified in point 1, while six-decimal metrics appear alongside single runs without variance.
Questions for authors
  1. Has the inheritance gate in Eq.2 been evaluated on real episodes? If so, please report a full instantiation with concrete values for nn, the paired gain, the regression rate, δ\delta, ρmax⁡\rho_{\max}, and α\alpha.
  2. Could you run an injection experiment that plants known-bad methods, such as patches exploiting evaluator omissions, and report how often each gate rejects them?
  3. How would the gates handle a systematic evaluator flaw that persists across tasks, given both gates consume evidence from the same infrastructure?
  4. What is the performance of the same backbone and harness with VeriLoop disabled, and of a variant keeping only the verification gate?
Reviewer_NcsvOpenReview ↗
Rating
1
Confidence
3
Informativeness
1
Presentation
1
Soundness
1
Read review
Paper summary

This paper presents VeriLoop, a framework for self-modifying coding agents that separates "this fix worked on the current task" from "this fix should become part of how the agent works going forward". The idea is a dual-gate protocol: a verification gate for local acceptance, and an inheritance gate that requires later independent evidence before a correction is promoted, with provenance and rollback. The authors instantiate it with a frozen open-weight backbone, a few small adapters, and an external harness, and report strong scores on several coding benchmarks along with an audit of the gating mechanism.

Critical strengths
  • The core question is a good one. As agents increasingly edit their own harnesses, memories, and rules, deciding which locally successful changes deserve to persist indeed seems underexplored.

  • I like the emphasis on reversibility and provenance.

  • I appreciate that the authors are upfront about what their evidence does and does not support.

Critical weaknesses
  • In general my biggest concern is that the central claim isn't really tested. The whole motivation is that gated inheritance leads to better error correction on later tasks, but as the paper itself acknowledges, there's no comparison showing that inheritance actually improves downstream success. I'd want to at least see the same backbone with a standard harness, and VeriLoop with vs. without the inheritance gate, on the same tasks and budgets.

  • Also, the paper could be much more concrete. The intro is mostly in abstract terms, and it took me a while to picture what an inherited "method" looks like or when a locally verified fix would become harmful. Similarly, all figures are mostly describing the concept in abstract. One worked example of a correction going through both gates would help a lot.

  • The related work feels too short and likely misses a lot. There's a large body of work on self-improving agents and agent memory (skill libraries, experience/workflow memory, reflection-based methods, etc.), and positioning against these would make it much clearer what the inheritance gate adds beyond, say, validating a memory entry before reusing it.

Questions for authors

Can you explain a bit why do you feel the current harness are lacking for deciding whether to make local vs. global edits?

Reviewer_jkdTOpenReview ↗
Rating
1
Confidence
3
Informativeness
2
Presentation
2
Soundness
1
Read review
Paper summary

This paper studies how coding agents can retain and reuse corrections across tasks while limiting unsupported method updates. It proposes VeriLoop, a dual-gate framework that separates verification of a local correction from its promotion into a reusable method, with promotion conditioned on subsequent independent evidence of benefit and bounded regression risk. The framework is instantiated as VeriLoop Coder-E1 using a frozen 4-bit Qwen3.6-27B backbone, four auxiliary control adapters, and an external harness supporting scoped memory, provenance tracking, and rollback. The evaluation spans three levels: end-to-end scores on four software-engineering benchmarks, held-out endpoints for the individual adapters, and an audit of admission and boundary compliance over 115 episodes. The authors note that these results support the architecture's feasibility but do not directly measure whether cross-task inheritance improves downstream task success.

Critical strengths

A well-motivated separation between local verification and cross-task inheritance. The paper identifies a real failure mode of self-modifying agents: a revision that succeeds on the task that produced it can be promoted into persistent behavior even when that success reflects evaluator omissions, overfitting, or noise. Requiring subsequent independent evidence (Eq. 2), operational non-equivalence, and a revocation path before promotion is a sensible and clearly articulated design principle. This is the main reason the paper is of interest, although its value depends on whether the inheritance gate is shown to work in practice (see Weaknesses).

Critical weaknesses
  1. Insufficient empirical validation of the inheritance gate GIG_I. The values of nmin⁡n_{\min}, δ\delta, ρmax⁡\rho_{\max}, and α\alpha in Eq. 2 are not reported, and no concrete candidate is shown passing or failing GIG_I with its estimated gain and regression bounds. The mechanism audit in Table 3 measures admission, packaging, and guard compliance, but does not demonstrate that promoted candidates satisfy the stated benefit and regression-risk criteria on subsequent independent tasks. These observations therefore do not establish whether the proposed inheritance gate is operationally enforced as specified. Although the authors explicitly refrain from claiming causal downstream improvement, the missing evidence also limits support for the framework's central architectural contribution: distinguishing locally successful corrections from methods justified for future reuse.

  2. The component-level evidence does not establish the contribution of the learned control surfaces to agent behavior. For the three classification adapters, Table 2 reports before/after improvements without clearly defining the "Before" condition or how its predictions are obtained, making the comparisons difficult to interpret. Each capability endpoint uses 80 held-out examples, and the reported classification and regression metrics alone do not establish the usefulness of these signals within the complete agent. Section 3.3 describes the adapters' intended roles, but the concrete decision rules connecting their outputs to harness actions, and evidence of their effects on task outcomes, remain insufficiently specified. No component ablation or controlled comparison isolates the contribution of these learned signals. Consequently, the adapter evaluations and system-level benchmark results do not establish whether the proposed control surfaces improve the agent's decisions or task performance.

Questions for authors
  1. Implementation and validation of the inheritance gate. What values of nmin⁡n_{\min}, δ\delta, ρmax⁡\rho_{\max}, and α\alpha, and which confidence-bound estimators, were actually used for GIG_I? How many provisional candidates were evaluated and promoted? Please provide the available paired later-task results and gain/regression bounds for representative accepted and rejected candidates.

  2. Interpretation and use of the adapter outputs. What exactly does "Before" denote for the three classification adapters in Table 2? How are the trained adapter outputs translated into harness decisions? Please provide any existing execution traces or controlled comparisons that demonstrate their effects on decisions or task outcomes.

  3. Memory during evaluation. Is engineering memory reset between benchmark instances or updated across them? If it persists, please clarify its initial contents, the task ordering, and the feedback available for memory updates and method promotion.

  4. Benchmark protocol and verification boundaries. For each benchmark, please specify the evaluated task set and count, attempts per task, token and time budgets, and evaluator version. Which tests are available to the agent during solving and candidate selection, and how are benchmark-provided evaluation tests and grading feedback isolated from these processes?

  5. Clarification of statistical reporting and figure comparison criteria. Could the authors resolve the following apparent inconsistencies? Table 2's caption specifies Holm adjustment, whereas its column is labeled qBHq_{\mathrm{BH}} and the values appear consistent with Benjamini–Hochberg adjustment. Comparing Tables 3 and 8, the adjusted values appear consistent with a five-test family, although only four endpoints (M2–M5) are reported; please specify the complete testing family and correction procedure.

From OpenReview, CC BY 4.0. Authors as signed there; shown unedited.

Related papers

Loading the map…

Discussion (0)

Sign in to comment.