acceptodds
Under review as a conference paper at ICLR 2027

Improving Large Language Models for Code through Runtime Program-State Reasoning

Abstract

Large language models receive limited explicit training in reasoning about runtime program states. We study whether training models to reason about runtime program states improves downstream software-engineering capabilities. We introduce two complementary program-state reasoning tasks. Buggy input–output reasoning requires a model to generate a concrete input that exposes a behavioral difference between a buggy program and a hidden correct implementation and to predict the resulting execution behavior. Precondition–postcondition reasoning requires an agent to symbolically characterize a bug-triggering precondition, predict the expected postcondition, explain their causal connection, and instantiate this reasoning as an executable regression test. By incorporating these two tasks into a staged post-training pipeline, we develop Comet-9B, a 9B language model based on Qwen3.5-9B Base. We evaluate the resulting checkpoints on repository-level patch generation, regression-test generation, and security PoC generation. Adding both program-state reasoning tasks to supervised fine-tuning (SFT) on issue resolution improves success rates by 7.25 percentage points on SWE-bench Pro and 9.70 points on SWT-Bench Verified. Sequential reinforcement learning on the two tasks yields further gains of 7.25, 26.79, and 4.67 percentage points on SWE-bench Pro, SWT-Bench Verified, and CyberGym, respectively. Despite having only 9B parameters, Comet-9B achieves a score comparable to the reported GPT-5.2 result on SWE-bench Pro and matches the reported success rate of a GPT-4o-based agent on SWT-Bench Verified.

Primary area:
reinforcement learning
Keywords:
Large Language Models, Reinforcement Learning, Code Reasoning, Software Engineering Agents

Reviews

Among:
Shape:
1234this paper, Bottom 27%
Figure 1. Mean review rating of the reviewed papers at ICLR 2027. This paper averages 2.00 over 3 reviews, above 27% of the others.
Reviewer_NQ5bOpenReview ↗
Rating
2
Confidence
3
Informativeness
2
Presentation
3
Soundness
3
Read review
Paper summary

This paper proposes two program-state reasoning tasks for training code LLMs that act as software-engineering (SWE) agents for bug fixing and regression-test generation: (a) Buggy I/O Reasoning. Given a buggy program, the model must infer unit tests that expose the bug, without access to execution tools. (b) Pre/Post-Condition Reasoning. At the repository level, the model must write a regression test for a bug fix. It has shell access to explore the repository. It must also produce a separate reasoning file that records the symbolic conditions under which the bug is triggered and what the bug causes. These two tasks are added to the standard patch-generation task in an SFT + RL training pipeline. The resulting model, Comet-9B, is competitive with other agentic baselines on bug fixing and test generation in SWE-Bench Pro, SWT-Bench Verified, and CyberGym.

Critical strengths
  1. Intuitive task formulation. Inferring bug-exposing inputs and the pre/post-conditions of a fix are intuitive ways to make a model reason about runtime program state, a skill that patch-only training does not target directly.

  2. Clear data collection and training setup. The pipeline for building task data and the SFT + RL setup are well described, which supports reproducibility.

  3. Clear gains over the base model. Adding the two tasks produces a model that is noticeably better at bug patching and regression-test generation than the base model.

Critical weaknesses

W1. Missing empirical comparison with models trained on SWE-Gym (Pan et al., 2025), R2E-Gym (Jain et al., 2025), SWE-smith (Yang et al., 2025) provided executable repository environments and trajectory data for agent SFT and RL. This will isolate the impact of adding training tasks for reasoning about runtime program states which existing works don't provide.

W2. Missing cost–performance comparison with the strongest baselines. The best agentic baselines clearly outperform Comet-9B: Gemini-3 Flash by 4.12 points on SWE-Bench Pro, and GPT-5-mini by 16.9 points on SWT-Bench Verified. The 16.9-point gap in particular weakens the case for Comet-9B as a standalone contribution. Please report the inference cost of running the agent with each model, for example API dollars per instance, tokens, number of agent turns, and wall-clock time. Please compare these with Comet-9B’s self-hosted serving cost. Can you justify the accuracy loss with the advantages offered by Comet-9B?

W3. Bug detection is not addressed. Both tasks assume that a bug is already known to exist. In practice, an agent must first decide whether a bug exists in a program or codebase, and where, which requires scanning the code and reasoning about program states without executing it. The paper does not address this setting. It should either evaluate it, for instance as bug detection or localization, or state it explicitly as a limitation.

W4. Possible bias toward finding bugs that are not there. Do the two tasks bias the model toward reporting bugs even when none exist? Can you compute Comet-9B’s false-positive rate. One test: give the model a bug report paired with the already fixed program and measure how often it still generates a “bug-exposing” test or a patch. A model that always finds a bug would be of limited use in practice.

W5. Size of fixes and depth of repository/codebase exploration. Please break down results by the size of the gold patch, e.g., lines changed, number of chunks, and number of files touched. Is Comet-9B mainly fixing small, local bugs, or can it handle multi-location fixes? It is also unclear how thoroughly the agent explores the repository while reasoning about a bug. Please report exploration statistics, such as files opened, distinct functions inspected, and number of shell commands, with and without the two auxiliary tasks in training. This would show whether the tasks actually change how the agent navigates code.

W6. Missing per-task ablation. The paper does not clearly isolate the contribution of each task. The RL variants in Tables 1 and 2 have all seen both tasks during SFT, so the effect of each one cannot be separated. Please compare Comet-9B against models trained with SFT + RL on (i) patch generation plus only Buggy I/O Reasoning, and (ii) patch generation plus only Pre/Post-Condition Reasoning. This would show whether both tasks are necessary.

Questions for authors

See Weaknesses section.

3 comments

Thank you for your detailed review and constructive suggestions. We appreciate your recognition of the task formulations, the reproducibility of the training setup, and the gains over the base model. We address each of your concerns below.

W1: Comparison with SWE-Gym, R2E-Gym, and SWE-smith.

We report additional comparisons on SWE-bench Verified using Pass@1 only, excluding results that use multiple candidate patches and verifier-based selection.

ModelParametersAgentPass@1
SWE-Gym-32B [1]32BOpenHands20.6%
R2E-Gym-32B [2]32BOpenHands34.4%
SWE-agent-LM-32B (SWE-smith) [3]32BSWE-agent40.2%
Comet-9B9Bmini-SWE-agent58.0%

Comet-9B resolves 290 of 500 tasks, exceeding the strongest of these published Pass@1 baselines by 17.8 percentage points while using fewer parameters.

[1] Training Software Engineering Agents and Verifiers with SWE-Gym.

[2] R2E-Gym: Procedural Environments and Hybrid Verifiers for Scaling Open-Weights SWE Agents.

[3] SWE-smith: Scaling Data for Software Engineering Agents.

W2: Cost and performance.

We agree that cost–performance comparisons are valuable for deployment decisions. However, our main contribution is the training method: designing explicit program-state reasoning tasks that improve downstream software-engineering capabilities. Comet-9B serves as an empirical validation of this method; we do not claim that it offers a superior deployment cost–accuracy trade-off over commercial models.

Our central comparisons therefore use the same model family and agent settings to study the benefits of program-state reasoning training. A comprehensive deployment comparison would require additional measurements under specified serving conditions. Although API expenditure can be measured, self-hosted cost depends on hardware, utilization, batching, and workload, making a meaningful comparison more involved than comparing API prices with model size. Such a deployment study is beyond the scope of this work. The reported commercial baselines provide performance context, rather than support a claim that Comet-9B should replace the strongest available agents.

W3 and W4: Issue-resolution setting and bug existence.

We address W3 and W4 together because both concern whether the agent must first determine that a reported bug actually exists.

Our main claim is that adding explicit program-state reasoning tasks improves downstream software-engineering capabilities. We study issue resolution under the established SWE-bench setting [1], where a user reports an issue and asks the agent to resolve it. The same setting is used to construct training tasks and environments in SWE-Gym [2], R2E-Gym [3], and SWE-smith [4], and to train models in SWE-RL [5], Co-PatcheR [6], and SWE-Fixer [7]. These works, accepted at top-tier venues, share a focus on resolving reported issues. We follow this established scope. Asking an agent to independently determine whether and where bugs exist is an important but distinct research problem, which falls outside the scope of our study.

To examine whether the benefits extend beyond issue resolution, we also study regression-test generation and generalization to vulnerability reproduction without security-specific post-training. Together, these three downstream tasks provide broad evidence that our program-state reasoning tasks improve software-engineering capabilities across different applications.

[1] SWE-bench: Can Language Models Resolve Real-World GitHub Issues? ICLR 2024.

[2] Training Software Engineering Agents and Verifiers with SWE-Gym. ICML 2025.

[3] R2E-Gym: Procedural Environments and Hybrid Verifiers for Scaling Open-Weights SWE Agents. COLM 2025.

[4] SWE-smith: Scaling Data for Software Engineering Agents. NeurIPS 2025, Datasets and Benchmarks Track.

[5] SWE-RL: Advancing LLM Reasoning via Reinforcement Learning on Open Software Evolution. NeurIPS 2025.

[6] Co-PatcheR: Collaborative Software Patching with Component-specific Small Reasoning Models. NeurIPS 2025.

[7] SWE-Fixer: Training Open-Source LLMs for Effective and Efficient GitHub Issue Resolution. Findings of ACL 2025.

W5: Fix complexity and repository exploration.

Thank you for suggesting this analysis of fix complexity and repository exploration. We compare Pure SWE SFT, Mixed SFT, and Comet-9B. Pure SWE SFT provides an issue-resolution baseline with the agentic capabilities needed to use the evaluation scaffold, which the earlier mathematics/code RL checkpoint has not yet acquired. Mixed SFT adds both program-state reasoning tasks during SFT; Comet-9B further applies program-state reasoning RL.

Performance across fix complexity. We group all 731 SWE-bench Pro tasks by their official gold patches. Changed lines count additions plus deletions, hunks are unified-diff hunks, and files are distinct modified paths.

Gold-patch metricGroupTasksPure SWE SFTMixed SFTComet-9B
Changed lines11–25120.0%25.0%41.7%
Changed lines26–5017126.3%38.6%49.1%
Changed lines51–10019921.1%29.6%36.2%
Changed lines>1003498.6%12.0%17.8%
Hunks0–1195.3%10.5%21.1%
Hunks23823.7%42.1%39.5%
Hunks3–512917.8%28.7%41.1%
Hunks6+54515.4%21.1%27.7%
Files110517.1%29.5%41.0%
Files212618.3%29.4%36.5%
Files3+50015.2%20.4%26.8%

Overall success is 16.01%, 23.26%, and 30.51%, respectively. Tasks involving at least three files contribute 58 of the 106 net additional successes over Pure SWE SFT and account for 134/223 (60.1%) of Comet's successes. On patches exceeding 100 changed lines, Comet resolves 62 tasks versus 30 for Pure SWE SFT. The gains therefore extend to larger, multi-location fixes.

Repository exploration and patch revision. We analyze shell commands on matched tasks. For early exploration, we use a fixed window of the first 50 shell calls and measure the proportion of successful explicit source-reading calls that access at least one file modified by the gold patch. We compute this proportion per task and then average across tasks. All other statistics below use the complete trajectories.

Exploration and revision metricPure SWE SFTMixed SFTComet-9B
Shell calls per task204.1205.8171.9
Distinct source files explicitly read per task12.110.88.4
Share of source-reading calls involving gold-modified files, first 50 shell calls56.5%59.3%69.0%
Inspections of current changes (git diff) per task17.77.94.1
Source re-edits after checking, per task8.36.53.3
Source rollbacks after editing, per task3.32.81.7

A source re-edit after checking is a later edit to the same source file following an earlier edit and an intervening diff inspection, test, or static check. Each later editing call is counted at most once. Source rollbacks count calls that restore previously edited source files using git checkout, git restore, or git reset --hard. We identify these events from explicit source-writing and restoration commands, excluding test files. Source-reading statistics count successful calls that explicitly display source files, including repeated reads. Diff inspections include summaries and exclude patch exports and git diff --check.

Adding the auxiliary tasks during SFT reduces source re-edits after checking from 8.3 to 6.5 per task, and subsequent program-state reasoning RL further reduces this to 3.3. Source rollbacks also decrease from 3.3 to 2.8 and 1.7. Comet's early reading is more concentrated on gold-modified files, reaching 69.0% compared with 56.5% for Pure SWE SFT and 59.3% for Mixed SFT, while it reads fewer distinct source files overall.

On tasks successfully solved by all three models, source re-edits after checking also decrease from 3.0 to 1.9 and 1.1 per task. Across the matched tasks, normalizing by trajectory length gives 3.37, 2.55, and 1.74 such re-edits per 100 shell calls, respectively. These observations are consistent with more focused early repository exploration and fewer back-and-forth patch revisions.

W6: Contribution of the individual training tasks.

Training separate SFT-plus-RL pipelines for every task combination would require substantial computational resources, particularly for agentic RL involving repository interaction and execution. We therefore use incremental evaluation: retain the best overall configuration selected in preceding stages and measure the additional benefit of subsequent components. Stage-by-stage comparisons are also used in Nemotron-Cascade [7] and Qwen3 [8].

Including the same task types in SFT and subsequent RL is common practice in recent model training. DeepSeek-V4 trains domain specialists through domain-specific SFT followed by GRPO [1]. GLM-5, Kimi K3, Qwen3-Coder-Next, and Nemotron 3 Super similarly include reasoning or agentic tasks during SFT before further optimizing those capabilities through RL [2–5]. Nemotron 3.5 Super additionally branches its agentic and reasoning teachers from the same intermediate RLVR checkpoint [6]. These designs use preceding training to establish a capable initial policy, then assess further improvements through specialized optimization.

We follow this practice by initializing both RL branches from Mixed SFT. This fixes the preceding training while measuring the additional benefit of each RL task. Buggy-I/O RL and Pre–Post RL improve SWE-bench Pro performance from 23.26% to 26.54% and 26.81%, respectively, and SWT-Bench Verified performance from 18.71% to 24.71% and 44.80%. Sequentially combining them further improves patch-generation performance to 30.51%. Our existing comparisons therefore already support our main claim: adding explicit program-state reasoning training improves downstream software-engineering capabilities.

[1] DeepSeek-V4: Towards Highly Efficient Million-Token Context Intelligence. 2026.

[2] GLM-5: from Vibe Coding to Agentic Engineering. 2026.

[3] Kimi K3: Open Frontier Intelligence. 2026.

[4] Qwen3-Coder-Next Technical Report. 2026.

[5] Nemotron 3 Super: Open, Efficient Mixture-of-Experts Hybrid Mamba-Transformer Model for Agentic Reasoning. 2026.

[6] Nemotron 3.5 Super and Lightning: Open, Efficient Mixture-of-Experts Hybrid Mamba-Transformer Models for Agentic AI. 2026.

[7] Nemotron-Cascade: Scaling Cascaded Reinforcement Learning for General-Purpose Reasoning Models. 2025.

[8] Qwen3 Technical Report. 2025.

Reviewer_iLzaOpenReview ↗
Rating
2
Confidence
3
Informativeness
2
Presentation
3
Soundness
3
Read review
Paper summary

The paper introduces a new model, Comet-9B, based on Qwen3.5-9B Base, trained on two program state reasoning tasks: (i) buggy input-output reasoning for standalone programs, and (ii) precondition-postcondition reasoning for repository-level state reasoning. The model has been trained in a four-stage pipeline, and the final model checkpoint scores 30.51% on SWE-Bench Pro, 45.50% on SWT-Bench verified, and 10.67% on CyberGym.

Critical strengths
  • The paper is very well written and easy to follow.

  • The results are impressive for a 9B parameter model

  • The generalisation result on CyberGym is interesting

Critical weaknesses
  • For SWE-Bench Pro results, it's surprising to see that the difference between the Qwen3.5 Instruction-based model and the pure SWE-SFT variant is 8%. And adding buggy I/O SFT increases performance but still couldn’t outperform the instruction model.

  • No rationale is provided for the ordering of the four-stage training pipeline. 

  • No variance score or statistical significance is reported, and in the ablation, the differences are very tiny. For example, from Table 7, the difference between Buggy-I/O RL and Pre-post RL is only two tasks. (196 vs 194)

  • The Qwen3.5 models were released in early 2026; the authors don’t mention the pretraining cutoff, and it is not reported whether any of NuminaMath-1.5, TACO, or SWE-Gym overlap with the base model's pretraining data.

Questions for authors
  • Is the huge difference between the SWE-SFT variant and Qwen3.5 Instruction and buggy I/O SFT due to the training data quality? If training data quality is the reason, then it would weaken the results in the ablation study. 

  • Why didn’t the authors include Math+Code RL in the ablation for SWE-Bench Pro and SWT-Bench?

  • Could the authors provide any rationale for the ordering of the training pipeline? 

  • Could the authors report variance over several runs?

  • Could the authors state the pretraining cutoff date for Qwen3.5, and check whether there is any overlap between Comet’s training data and its evaluation benchmarks?

  • Did the authors evaluate Comet on code reasoning benchmarks like CruxEval?

1 comment

Thank you for your careful review and for highlighting the paper's clarity, the results at 9B parameters, and the generalization to CyberGym. We address your questions about the training pipeline, comparisons, ablations, and evaluation below.

Official instruction model and intermediate SFT checkpoints.

We clarify that “Qwen3.5-9B Instruct” denotes the officially released post-trained model, Qwen3.5-9B, with broad reasoning, instruction-following, coding, and tool-use capabilities. Qwen’s published post-training research describes a substantially broader process than task-specific SFT, including long-CoT fine-tuning, reasoning RL, further SFT for thinking-mode fusion, general RL, and teacher–student distillation. Qwen3.5’s official documentation additionally emphasizes large-scale RL across diverse tasks and agent environments. Our SFT variants start separately from Qwen3.5-9B-Base and, following the shared mathematics and code RL stages, receive only SFT for agentic software-engineering tasks. This is only an intermediate stage of our overall training pipeline, so it is not surprising that it underperforms the fully post-trained Qwen3.5-9B. Subsequent program-state reasoning RL brings Comet-9B to 30.51%, exceeding the official model by 6.16 percentage points.

Rationale for the training order.

Our training order follows common practice in staged post-training: first strengthen foundational reasoning and coding capabilities, then specialize the model for downstream agentic tasks. GLM-5 performs reasoning RL on mathematics, science, and code before agentic RL [1], while Nemotron-Cascade 2 places competitive-programming Code RL before SWE RL [2]. Following this principle, we first apply mathematics and code RL to establish a foundation for subsequent software-engineering training.

For downstream specialization, we perform SFT before program-state reasoning RL. Using SFT to initialize the policy before RL is common across recent models, including Qwen3, Qwen3-Coder-Next, Kimi K3, and Phi-4-reasoning-Plus [3–6]. In our pipeline, SFT teaches the required task behavior, tool-call capability, and output formats. Subsequent RL optimizes these behaviors using execution and reasoning rewards.

[1] GLM-5: from Vibe Coding to Agentic Engineering.

[2] Nemotron-Cascade 2: Post-Training LLMs with Cascade RL and Multi-Domain On-Policy Distillation.

[3] Qwen3 Technical Report.

[4] Qwen3-Coder-Next Technical Report.

[5] Kimi K3: Open Frontier Intelligence.

[6] Phi-4-reasoning Technical Report.

Interpreting the two-task difference between RL branches.

Buggy-I/O RL and Pre–Post RL are two alternative training branches initialized from the same Mixed SFT checkpoint, each applying RL to a different program-state reasoning task. They do not form an ablation where one task is removed from the other. Accordingly, the 196-versus-194 comparison measures the difference between these two training choices, rather than the contribution of adding program-state reasoning training. To assess that contribution, the relevant comparisons are between each RL variant and their shared Mixed SFT starting point. On SWE-bench Pro, Buggy-I/O RL improves success from 170/731 (23.26%) to 194/731 (26.54%), while Pre–Post RL improves it to 196/731 (26.81%). Sequentially applying both RL stages produces Comet-9B, which further improves success to 223/731 (30.51%). These results support the usefulness of both tasks and their complementary benefits when combined, rather than establishing the superiority of one based on the two-task difference.

Potential overlap with the base model’s pretraining data.

We could not find a publicly documented pretraining-data cutoff for Qwen3.5. Qwen3.5-9B was released on March 2, 2026, and all three datasets predate this release. Given their widespread use, it is plausible that they were already included in Qwen3.5’s training, although we cannot confirm this from the available documentation.

This possibility aligns with our motivation. Our contribution is a training method that starts from the official base model and outperforms the official post-trained model in both patch generation and regression-test generation, with additional generalization to security PoC generation. These improvements do not depend on access to new datasets collected after the base model’s training cutoff. Instead, we design new program-state reasoning tasks from existing datasets, extracting additional training value from data that the original model may already have encountered.

Importantly, this question concerns potential overlap between our training datasets and the base model’s training corpus, rather than overlap with our evaluation benchmarks. Reusing training data does not itself constitute evaluation-data contamination.

Reviewer_PCV5OpenReview ↗
Rating
2
Confidence
3
Informativeness
2
Presentation
2
Soundness
2
Read review
Paper summary

Comet-9B is trained on two code reasoning tasks: (1) given buggy code and its specification, the model constructs an input that exposes the bug and predicts the output, crash, or timeout without execution tools; and (2) with access to a repository and execution tools, the model identifies the conditions that trigger a bug, explains how execution violates expected behavior, and writes a regression test. These tasks both improve the model's ability to fix bugs and generate regression tests.

Critical strengths

The reasoning tasks seem interesting and they improved the model's ability to fix bugs and generate regression tests.

Critical weaknesses

Experiments use 9B backbone and Python-only reasoning-training tasks, and one attempt per task.

Questions for authors

How does your method compare to SWE-Spot: Building Small Repo-Experts with Repository-Centric Learning?

1 comment

Thank you for your careful review and for recognizing the value of the two reasoning tasks for bug fixing and regression-test generation. We address your questions about SWE-Spot, model scale, training languages, and evaluation below.

Relation to SWE-Spot.

1. Training goal. SWE-Spot aims to build repository experts by training on diverse experiences within target repositories, so that models acquire repository-specific knowledge before deployment. We study whether explicit runtime program-state reasoning training improves downstream agentic software-engineering capabilities. Comet uses a single model across repositories and tasks, without separately specializing a model for each target repository.

2. Task requirements. The closest overlap is that SWE-Spot also trains patch generation and reproduction-test generation. These tasks allow agents to repeatedly modify code or tests and inspect execution feedback until the final artifact succeeds. Tools can directly provide execution results and intermediate variable values; artifact correctness alone does not explicitly require the model to infer bug-triggering conditions or explain how program states propagate. Our tasks make these reasoning requirements explicit. In precondition–postcondition reasoning, the model must infer a symbolic bug-triggering precondition (backward reasoning), specify the behavior a correct implementation should satisfy, and explain how execution under that precondition propagates through functions, branches, and intermediate states to violate the expected postcondition (forward reasoning). It then instantiates this explanation as a regression test. In buggy input–output reasoning, the model must construct a concrete bug-triggering input (backward reasoning) and predict the buggy program’s execution status and output (forward reasoning), without execution tools. Together, these tasks explicitly target both symbolic and concrete-value reasoning, in both backward and forward directions.

3. Optimization. SWE-Spot’s implementation uses supervised fine-tuning on teacher-generated repository experiences. Comet combines SFT with subsequent RL on the two program-state reasoning tasks. In particular, repository-level RL rewards both the regression test’s execution correctness and the reasoning file’s quality, making the stated precondition, postcondition, causal explanation, and consistency with the test explicit targets of optimization.

Model scale and training languages.

Our central aim is to establish the usefulness of explicit runtime program-state reasoning training for downstream agentic software engineering. We study this using a 9B backbone; extending the method to larger models and training on more programming languages are directions for future work. Although both reasoning-training tasks use only Python, our results show transfer to Go, JavaScript, and TypeScript on SWE-bench Pro, languages outside our program-state reasoning training data. CyberGym further provides evidence of transfer to C/C++ codebases and to security PoC generation, a task not included in our post-training. These results indicate that the benefits extend beyond the languages and tasks used for program-state reasoning training.

From OpenReview, CC BY 4.0. Authors as signed there; shown unedited.

Related papers

Loading the map…

Discussion (0)

Sign in to comment.