acceptodds
Under review as a conference paper at ICLR 2027

Inherited, Added, Or Ended Unresolved: Attribution Accounting For Warm-Started Coding Agents

Abstract

A warm start lets a coding agent continue from a repository that another session has modified. Testing the final repository measures system success, but does not separate solutions already present at handoff from those added by the successor. It also leaves unresolved whether the successor lost an inherited solution. We measure these transitions with a prefix-only arm: replay the recorded prefix and submit the inherited tree without starting a successor. This requires no additional model calls. Pairing its score with the resumed session's gives a four-cell accounting and the identity . The same accounting expresses a published takeover benchmark's result for QwenDevstral on already-solved handoffs as both a final solved rate and a preservation rate, rather than a rate of newly added solutions. In our historical -task SWE-bench Pro analysis with SWE-agent at a action cut, of gpt-oss-120b's resumed-arm solves and of deepseek-v4-flash's were already resolved at handoff. These shares condition on branch pairs where replay landed edits in both warm-started arms; under frozen intent-to-treat scoring, their net score changes are and points. On a separate -task cut sweep with a fixed -slot denominator, the inherited share rises from to as handoff moves later, while final score point estimates span points. This non-random subset supports a within-sample trend, not a claim that its levels or curve shape transfer to other pools. We recommend reporting the prefix-only baseline alongside the final score so that system success and successor contribution remain distinct.

Primary area:
generative models
Keywords:
code agent, warm start for agent

Reviews

Among:
Shape:
1234this paper, Bottom 1%
Figure 1. Mean review rating of the reviewed papers at ICLR 2027. This paper averages 1.00 over 3 reviews, above 0% of the others.
Reviewer_sUbKOpenReview ↗
Rating
1
Confidence
2
Informativeness
1
Presentation
1
Soundness
1
Read review
Paper summary

This paper studies the problem of credit assignment for SWE issue resolution tasks in take-over or "handoff" scenarios defined as disjoint, sequential AI agent sessions where one agent does part of the task and then reassigns it to another agent. They identify the issue already being resolved before handover and the agent taking over losing the solution as potential issues with the handover task that better credit assignment/accounting can fix. They propose an accounting scheme that might help separate the successor credit from the overall task success.

Critical strengths

None

Critical weaknesses
  1. Extremely poorly written paper. It is full of Claudeisms and is very hard to follow/understand.
  2. Very narrow and insignificant contribution. This paper at most proposes an evaluation metric that might be useful for a very specific kind of task. Even this is not established with any notions of evaluation metric quality, such as human studies, construct validity, or metric reliability. The utility of the proposed metric/accounting scheme is unclear.
  3. Insufficient empirical contributions. Experiments are limited to one benchmark, two models, and one harness, making the generalizability of any findings questionable.
Questions for authors
  1. What is the downstream utility of the proposed metric?
  2. How do you know if your proposed metric/credit assignment is doing the intended work besides moving some extrinsic evaluation scores in a particular direction?
  3. "Its non-random sample does not establish transfer to other pools." -- What does this mean?
Reviewer_bxpVOpenReview ↗
Rating
1
Confidence
3
Informativeness
1
Presentation
1
Soundness
1
Read review
Paper summary

This paper studies how to measure the contribution of coding agents that resume work from a partially modified repository. The authors propose a prefix-only baseline that evaluates the repository before the successor starts, then compares it with the final result. This distinguishes part of solutions already present from those added or lost during continuation.

Critical strengths

Scoring the starting checkpoint makes for a good baseline.

Critical weaknesses

The technical contribution is thin as the protocol amounts to evaluating agent performance from a starting state. The study uses only two models, one scaffold, and one benchmark family. It does not establish how inherited success varies with agent capability or harness design.

Questions for authors

What scientific conclusion changes because of this work?

The authors are evaluating contributions from handoff/states but the abstract mentions "non-random subset supports a within-sample trend, not a claim that its levels or curve shape transfer to other pools." Furthermore, the conclusion statest "This does not establish a general relationship between capability and inheritance

Does this mean the results are not generalizable?

Reviewer_7K9DOpenReview ↗
Rating
1
Confidence
1
Informativeness
1
Presentation
1
Soundness
1
Read review
Paper summary

This paper studies how to attribute coding-agent performance under warm starts. The authors propose evaluating the inherited repository before continuation and comparing its success with the final repository. This separates inherited solutions, newly added solutions, and previously resolved tasks that become unresolved. Experiments on SWE-bench Pro with two models suggest that a substantial fraction of successful warm-started runs inherit solutions already present at handoff.

Critical strengths

None

Critical weaknesses

Poor presentation. The paper is extremely difficult to follow. A relatively simple evaluation idea is buried under extensive notation, accounting identities, statistical comparisons, and implementation details. The main contributions are difficult to distinguish from routine analysis and experimental logs.

Questions for authors

Q1. Can the authors simplify the presentation and clearly distinguish the main findings from supporting implementation and statistical details?

Q2. Could the authors clarify the definition of a warm start and how it differs from other forms of agent continuation, particularly those that retain conversation history or other context?

From OpenReview, CC BY 4.0. Authors as signed there; shown unedited.

Related papers

Loading the map…

Discussion (0)

Sign in to comment.