acceptodds
Under review as a conference paper at ICLR 2027

VECTRA: Verified Episodic Control for Repository-Level Code Translation

Abstract

Repository-level code translation is a long-horizon engineering process that requires establishing a coherent target architecture and cross-file interfaces, reconstructing build configurations, and preserving program behavior. Modern coding agents can perform much of this work, but effective execution also requires deciding which intermediate repository states to retain, which failures to address next under a limited budget, and what knowledge to carry across sessions. We present VECTRA, a control framework that advances repository translation through a sequence of verifier-admitted states. Four coordinated mechanisms form its execution loop: a holistic bootstrap establishes the overall target structure; a frontier scheduler turns current build and test evidence into bounded translation missions; a verification gate accepts candidate changes or rolls them back; and a persistent translation ledger carries verified facts and cross-language decisions across fresh coding-agent episodes. The underlying coding agent remains unmodified and can use its usual tools and subagents. Across 13 translation directions in RepoTransBench, VECTRA achieves an average success rate of 67.9%, outperforming the evaluated agent-based baselines. Ablation results show that removing any of the four mechanisms reduces the success rate. These results suggest that explicit control over intermediate repository states and translation progress can complement advances in code generation.

Primary area:
foundation or frontier models, including LLMs
Keywords:
repository-level code translation, coding agents, software migration, execution feedback, large language models

Reviews

Among:
Shape:
1234this paper, Bottom 2%
Figure 1. Mean review rating of the reviewed papers at ICLR 2027. This paper averages 1.25 over 4 reviews, above 2% of the others.
Reviewer_zrNWOpenReview ↗
Rating
1
Confidence
3
Informativeness
1
Presentation
2
Soundness
1
Read review
Paper summary

The paper presents VECTRA, an outer control framework for repository-level code translation. It wraps an existing coding agent with repository-wide bootstrap, failure-driven scheduling, verification and rollback, and a persistent ledger across episodes. On RepoTransBench, VECTRA reports a 67.9% average success rate over all 13 translation directions, compared with 55.6% for direct OpenCode.

Critical strengths

The evaluation covers all 13 translation directions and 1,864 projects in RepoTransBench.

Critical weaknesses
  • The evaluation setup is absolutely a major concern. The target tests seem to be visible to the agent during translation and repair, while those same tests are also used to define final success. This makes it possible to optimize directly for the evaluation cases, or even hard-code behavior for visible inputs. Passing these tests alone does not show general source-target behavioral equivalence. It is obvious that test cases or evaluation cases can not be visible during the processing of converting the code.

  • The presentation is very hard to follow, no key point or gap from the introduction part, and method, expirement part are full of defensive writing style which is not a good acdemic writing. The Introduction does not clearly explain what prior work already solves, what is still missing, and what specific gap VECTRA addresses. Figure 1 is also not very informative (or information density is low, too big and not clear with no information), it shows the four components, but does not make the actual control flow, state updates, or accept/reject process clear.

  • The baseline comparison is too limited. On the full benchmark, VECTRA is mainly compared with RepoTransAgent and direct OpenCode. ReCodeAgent is only tested on two directions, and several closely related recent systems are not compared experimentally.

  • Harness matters, so we cannot say VECTRA is a good process than existing harness. Prior work [1,2] shows that the coding-agent harness itself can have a large effect on both performance and efficiency. It is therefore unclear whether the gains come from VECTRA itself or from how it interacts with OpenCode.

  • The comparison is also not fair. VECTRA uses much more runtime and many more tokens per project, so part of the gain may simply come from giving the agent more computation.

References

[1] Naman Vats and Oleg Golev. "The Scaffold Effect in Coding Agents: Harness Choice as a Hidden Variable in Coding-Agent Evaluation." arXiv:2607.22585, 2026.

[2] Red Hat Emerging Technologies. "coding_agent_bench: Reproducible Benchmarks for Coding Agents and Models using Harbor." GitHub repository, 2026.

Questions for authors

I have multiple questions for authors, I pick the most critical 4 Qs:

Are the tests shown to the agent the same tests used for final evaluation? If yes, how do you rule out overfitting?

Why are stronger recent repository-translation systems not compared more broadly?

Do the gains still hold with other coding-agent harnesses?

What exactly is new here compared with prior work on planning, repair, rollback, and memory?

Reviewer_uafcOpenReview ↗
Rating
1
Confidence
2
Informativeness
1
Presentation
2
Soundness
1
Read review
Paper summary

This paper presents VECTRA, a verified episodic control framework for repository-level code translation. While keeping the underlying coding agent unmodified, it coordinates intermediate repository states and translation progress through four mechanisms: holistic bootstrap, frontier scheduling, verification gate, and persistent translation ledger. On the RepoTransBench benchmark, VECTRA achieves an average success rate of 67.9%.

Critical strengths

Good Performance: VECTRA achieves an average success rate of 67.9% across all 13 translation directions in the RepoTransBench benchmark, outperforming the evaluated baseline methods.

Critical weaknesses
  1. Limited Conceptual Novelty and Core Scientific Question: While the paper demonstrates the empirical effectiveness of VECTRA, it fails to sufficiently justify the uniqueness of its design choices or provide deep insights into why this specific configuration works. The framework reads more like a concatenation of engineering heuristics rather than addressing a fundamental scientific problem in repository-level code translation.
  2. Inefficient Resource Consumption and Confounding Factors: Equating a uniform maximum budget to actual computational expenditure is misleading, and current experiments fail to rule out compute scale as a confounding factor. VECTRA incurs a severe efficiency penalty, nearly doubling the execution time compared to direct OpenCode. Could the authors provide a controlled comparison under a strictly matched computational budget?
  3. Narrow Model Generalizability: The evaluation is restricted to a single base model (DeepSeek-V4-Flash). It remains unclear whether VECTRA's improvements generalize to other architectures or whether its marginal utility diminishes when applied to more advanced base models (i.e. GPT-5.5+ Opus 4.6+).
  4. Incomplete Baselines and Limited Evaluation Scope: Although the authors cite several advanced methods in related work (e.g., AlphaTrans, Skel, MatchFixAgent), they do not include them in the experimental comparisons. Furthermore, evaluating on only a single benchmark (RepoTransBench) lacks breadth; incorporating additional benchmarks or real-world repository cases would significantly strengthen the evaluation.
Questions for authors

Same as Critical Weaknesses

Reviewer_o1z1OpenReview ↗
Rating
2
Confidence
3
Informativeness
2
Presentation
2
Soundness
2
Read review
Paper summary

This paper investigates the issues of long-term execution and state management of coding agents in repository-level code translation. The authors point out that, although existing coding agents are already capable of repository analysis, cross-file editing and test-driven debugging, they still face problems such as code regression, inefficient task scheduling and loss of context across rounds during prolonged translation processes. To address this, the authors propose VECTRA, which constructs a Verified Episodic Control Framework external to existing coding agents, comprising four components: VECTRA utilises unmodified OpenCode as its underlying executor and achieves an average success rate of 67.9 per cent across all 13 translation directions and 1,864 projects on the RepoTransBench, outperforming the 55.6 per cent achieved by Direct OpenCode and the 20.8 per cent achieved by RepoTransAgent.

Critical strengths
  1. This paper treats Repository-Level Code Translation as a long-term, continuously evolving process requiring external verification, rather than a one-off code generation task. VECTRA separates code modifications from state acceptance decisions, prevents observed test regressions via a Verification Gate, and utilises a Persistent Ledger to retain design decisions and lessons learnt from failures across episodes.

  2. The paper covers all 13 translation directions and 1,864 projects in RepoTransBench, demonstrating good experimental coverage. Compared with Direct OpenCode, which uses the same model and executor, VECTRA increases the average success rate from 55.6 per cent to 67.9 per cent and the compilation rate from 76.9 per cent to 93.2 per cent.

Critical weaknesses
  1. The absence of a crucial naive multi-round execution baseline prevents a thorough isolation of the contributions made by the control mechanism and the additional execution computations. Direct OpenCode typically halts execution well before the budget is exhausted, whereas one of VECTRA’s direct functions is to continue initiating new coding episodes after the agent has stopped. If a baseline were added whereby, when OpenCode stops but not all tests have passed, the Agent is restarted and instructed to continue fixing the currently failing tests until the project budget is exhausted, this baseline would directly verify how much of VECTRA’s benefit stems from additional iterative opportunities and how much from its specific control mechanisms.

  2. The Success Rates for some Baselines in Table 1 cannot be directly calculated from the listed number of projects NN and the integer number of successful projects. For example, for JavaScript→Python, N=188N=188, but the Success Rate (SR) for Direct OpenCode is 60.0 per cent; for C#→Java, N=96N=96, but the SR for Direct OpenCode is 63.9 per cent. These values do not correspond directly to a result with one decimal place obtained from a single run in which all NN projects are included in the scoring.

Questions for authors
  1. It is recommended to add ‘OpenCode + Simple Retry Loop’: when the agent stops but there are still failed tests and remaining budget, restart the agent using the current failed test, without incorporating the Frontier Scheduler, Verification Gate or Ledger.

  2. Why can the success rates for some Baselines in Table 1 not be expressed directly as k/Nk/N? Do these figures derive from the average of repeated runs, different valid sample sizes, or other calculation methods?

Reviewer_THo6OpenReview ↗
Rating
1
Confidence
3
Informativeness
1
Presentation
2
Soundness
1
Read review
Paper summary

This paper presents a framework, VECTRA, designed to improve coding agents on repository-level code translation tasks. The main idea is to manage the coding agent through multiple episodes instead of asking it to complete the entire task in one run. VECTRA combines four main components: 1) holistic repository bootstrap, 2) frontier-based task scheduling, 3) verification-gated state admission, and 4) a persistent translation ledger. These components aim to reduce repeated failures, prevent regressions, and preserve useful information across episodes. The authors evaluate VECTRA on RepoTransBench, which contains around 1.8K projects across 13 translation directions. The results show that VECTRA improves translation success rate compared with direct OpenCode.

Critical strengths
  • Repository-level code translation is more difficult than translating individual functions because it requires handling dependencies, build configurations, cross-file interfaces, and test behavior. Improving how agents track progress and avoid breaking previously working code is a useful direction
  • VECTRA separates code generation from task scheduling, verification, and progress management. The design choice looks useful as it allows the controller to manage the coding agent without changing the underlying model in long-running coding tasks
Critical weaknesses

I have major concerns about the novelty, generality, and evaluation of VECTRA

  • My main concern is that the technical contribution of this paper is not sufficiently clear. The four main components of VECTRA are based on ideas that are already common in existing SWE agents. For example, task scheduling, checkpoint-based rollback, test execution-based verification, and persistent memory are not new ideas in coding agents. The paper combines these ideas into one workflow, but it is not clear what new technical insight this combination provides. The authors need to better explain why VECTRA is more than an engineering integration of existing techniques.

  • The next major concern is the evaluation which is mainly based on only OpenCode with DeepSeek-V4-Flash. This makes it difficult to understand whether the improvements come from the VECTRA controller itself or from how it interacts with this specific agent and model. My question is why didn't the author try popular coding agents (e.g., Codex, Claude Code, PI etc.)?? This shows a clear picture of how close/far VECTRA is from these coding agents in repository level code translation.

  • A proper comparison is required against more recent agent-based approaches under the same experimental settings. Without these experiments, the generality of the proposed framework remains unclear. The current results may be specific to the selected setup. Please convince me with the results why I will use VECTRA instead of Codex or Claude Code to do the translation?

  • VECTRA uses test results to decide whether a candidate repository should be accepted or rejected. This is a reasonable way to prevent some regressions, but passing the available tests does not necessarily mean that the translated code preserves the original behavior. The framework can accept an incorrect implementation if the test suite does not cover the affected behavior. Similarly, the requirement to preserve previously passing tests does not guarantee that the final repository is semantically correct. A stronger analysis is required to investigate how the framework behaves when test coverage is incomplete or tests are not reliable.

Questions for authors
  • (Q1) What is the main technical contribution of the proposed framework beyond intregating task scheduling, checkpoint-based verification, and persistent memory?
  • (Q2) Could the the authors compare VECTRA with other popular coding agent harness with different LLMs?
  • (Q3) How does VECTRA behave when the target test suite has low coverage or misses important behavioral differences?

From OpenReview, CC BY 4.0. Authors as signed there; shown unedited.

Related papers

Loading the map…

Discussion (0)

Sign in to comment.