acceptodds
Under review as a conference paper at ICLR 2027

One Repository, a Thousand Graphs: A Repository Context Interface for Coding Agents

Abstract

Large codebases far exceed the context that large language models can effectively process. Encoding repository knowledge in model parameters is costly, lags behind changes, and is difficult to sustain through continual learning. On-demand search can provide uneven coverage or miss relevant information, while maintaining explicit code graphs outside the model for large repositories poses challenges for covering task-relevant relations and keeping them synchronized as the code evolves. We propose ORTG as an integrated repository interface that organizes entity context and a partial set of prebuilt relations by module, delivers task-relevant information on demand through a two-level structure within a limited context, and specifies a workflow for updating entity outlines and relation fields after edits during task execution. End-to-end evaluation covers SWE-bench Verified, Deep-SWE, and FeatureBench. In the evaluated configurations, complete ORTG achieves higher mean task success rates than the corresponding baselines and ablation configurations. Behavioral statistics on Deep-SWE show fewer interactions and file reads, but higher cumulative token usage including index construction and updates.

Primary area:
generative models
Keywords:
LLM

Reviews

Among:
Shape:
1234this paper, Top 49%
Figure 1. Mean review rating of the reviewed papers at ICLR 2027. This paper averages 2.33 over 3 reviews, above 51% of the others.
Reviewer_HV7fOpenReview ↗
Rating
3(+1)
Confidence
3
Informativeness
2
Presentation
3
Soundness
2
Read review
Paper summary

This paper introduces ORTG, a repository context interface for coding agents. ORTG stores semantic descriptions of individual files together with a bounded number of relation targets, organizes these entries by module, and provides either complete outlines or hierarchical access depending on the context budget. The editing model also updates the corresponding descriptions and relation fields after modifying code. Experiments on SWE-bench Verified, Deep-SWE, and FeatureBench report higher mean task success rates than the corresponding baseline agents. Additional experiments examine relation removal, randomized relation targets, different relation caps, and flat versus hierarchical delivery. Behavioral measurements show fewer interactions and file reads, accompanied by increased cumulative token usage.

Critical strengths

1.The reported improvements are practically meaningful and extend across several evaluation settings. Table 2 reports improvements from 90.5% to 95.6% on SWE-bench Verified, from 46.1% to 52.3% on Deep-SWE, and from 25.2% to 41.0% for Codex on FeatureBench. The comparison with Codex + RPG-Encoder, which achieves 29.8%, is particularly relevant. These results make the proposed interface worth investigating beyond a single illustrative task.

2.The component experiments and cost reporting provide useful diagnostic evidence.Table 3 separates the effects of removing relation fields, corrupting their targets, and changing delivery policies. The entities-only condition reaches 36.4%, compared with 25.2% for Codex and 41.0% for complete ORTG, suggesting that semantic context contributes substantially. The paper also explicitly reports the 26% increase in cumulative tokens and correctly avoids interpreting fewer interactions as demonstrated reductions in monetary cost or latency.

Critical weaknesses

1.All reported ablations retain the update workflow. Without comparing against the same initial index with updates disabled, the experiments cannot establish whether maintaining repository context during editing contributes beyond providing useful initial descriptions.

2.The flat delivery baseline uses fixed ordering and truncation, potentially excluding relevant entries. Its lower performance therefore does not isolate the benefit of hierarchical organization from differences in information coverage.

3.Appendix C acknowledges that outline accuracy, relation precision and recall, and reuse across successive tasks are not evaluated. Whether repeated updates preserve semantic correctness or accumulate errors consequently remains unclear.

4.Appendix B does not specify the exact Deep-SWE and FeatureBench versions, splits, instance counts, or numerical execution budgets. FeatureBench has multiple splits and revised task statements, making these omissions consequential for reproducibility. The RPG-Encoder integration also requires clearer documentation. GitHub

5.The confidence interval calculation is unspecified, and paired uncertainty estimates are absent. Smaller differences, such as 95.6% versus 94.1% on SWE-bench Verified, remain difficult to assess beyond their reported means.

6.Table 4 reports 26% more cumulative tokens despite fewer interactions. Although acknowledged, the absence of latency, billed cost, and separate initialization and update measurements leaves the practical balance between performance gains and resource requirements unresolved.

Questions for authors
  1. Which exact dataset versions, splits, and instance counts underlie the Deep-SWE and FeatureBench results? Please identify the corresponding agent configurations, numerical execution budgets, evaluator versions, and configuration used to integrate RPG-Encoder.

  2. Do you have a comparison between complete ORTG and the same initial outlines and relation fields with index updates disabled, while retaining equivalent source access and inspection instructions? Such a comparison would materially affect my assessment of the distinctive contribution.

  3. How often does each benchmark use complete-outline delivery versus hierarchical access? Under the flat delivery baseline, what fraction of entries is truncated, and can the observed performance difference be separated from differences in accessible information?

  4. How are the confidence intervals computed? Can you provide paired performance differences and their uncertainty for complete ORTG versus the closest graph baseline and the single-level delivery condition?

Reviewer_L9GCOpenReview ↗
Rating
2
Confidence
3
Informativeness
2
Presentation
2
Soundness
2
Read review
Paper summary

This paper studies how to provide and maintain task-relevant context for coding agents working with large, continuously evolving code repositories. It proposes ORTG, which treats files as entities, stores semantic summaries of their functionality, interfaces, and constraints, and retains up to five relation targets per entity. These entries are organized by module: when the complete outline occupies less than half of the context window, it is provided in full; otherwise, the agent first localizes relevant modules and then accesses file entries by module or path. After code edits, the same editing model generates and submits updates to the summaries and relation fields, while file fingerprints indicate content changes. Experiments cover SWE-bench Verified, Deep-SWE, and FeatureBench, comparing base agents, graph-augmented methods, and ablations of relation fields and delivery strategies. Full ORTG achieves higher mean performance across the evaluated configurations, demonstrating the practical potential of this interface.

Critical strengths
  1. The problem is practically relevant, and the interface design is concrete. The paper organizes repository context around file responsibilities, interface constraints, relation hints, and maintenance after editing, while clearly distinguishing persistent state, task-time access, and the update process. Beyond describing a repository representation, it provides a concrete workflow that can be integrated into existing coding agents.

  2. The complete system consistently improves mean performance across multiple settings. In Table 2, the resolution rate on SWE-bench Verified increases from 90.5% to 95.6%, and the rate on Deep-SWE increases from 46.1% to 52.3%. Under the Codex setting on FeatureBench, the reported metric increases from 25.2% to 41.0%, also exceeding the RPG-Encoder baseline at 29.8%. These settings cover both bug fixing and feature development, suggesting that the benefits extend across different task types.

  3. The ablations are targeted, and the limitations of the results are reported candidly. The paper compares conditions with relation fields removed, randomized relation targets, different limits on the number of targets, and single-level delivery. It also explicitly notes that entity descriptions may themselves retain relational information. The authors additionally report increased token usage and present the Sphinx instance as a behavioral case that remains unresolved, helping readers distinguish overall performance, process observations, and cost measures.

Critical weaknesses
  1. The contribution currently appears primarily to be a workflow-level combination of existing ideas, and its substantive advance over closely related methods could be articulated more clearly. ORTG integrates file-level semantic summaries, limited relation hints, capacity-aware delivery, and updates after editing into a unified interface. Prior work such as RepoAgent [1] and RPG-Encoder [2] already incorporates several of these mechanisms. Section 3 and Figure 2 explain design motivations involving semantic coverage, context allocation, and reuse of understanding acquired during editing, while Tables 2–3 support the practical value of the system and several components. However, the current presentation emphasizes the effects of integrating the workflow. More focused comparisons and interpretation of the results are needed to clarify what additional insights these design principles offer relative to existing methods.

  2. The independent benefit of immediate updates after editing remains unclear. The relation-field and delivery-strategy ablations reported in the main text and appendices all retain updates; they do not include a comparison that freezes the initial outline or disables updates after editing. Thus, although the complete interface achieves higher mean performance, the reported results do not disentangle the benefits of the initial semantic overview from those of continued maintenance within a task. Appendix C also states that the semantic accuracy of the outline and the quality of its relations have not been independently quantified. A frozen-outline comparison that holds other conditions constant, together with a sampled analysis of update correctness and subsequent use, would more directly test the value of this mechanism within the current task.

  3. Some key evaluation details require clarification to support interpretation and reproducibility. The metric underlying the FeatureBench scores needs to be specified explicitly. Table 2 uses “resolution/success rates,” whereas the appendix refers to “pass rates.” The official FeatureBench protocol [3] distinguishes the rate of fully resolved tasks from the fraction of F2P tests passed. The statement in Appendix B.1 that the experiments follow the official settings does not substitute for an explicit identification of the metric.

  4. The actual experimental coverage and operating conditions of two-level access remain unclear. Section 4 determines whether to deliver the complete outline or use two-level access based on the ratio of the complete outline length to the context-window size. However, the main text and appendices do not report the actual window sizes, outline-length distributions, proportions of runs using each mode, or results across different repository sizes. The single-level delivery ablation supports the overall benefit of the evaluated delivery configuration, but it remains unclear how much of this benefit comes from two-level access activated when the outline exceeds the capacity threshold. Reporting performance and the number of entries actually visible at different outline-length-to-window-size ratios would more directly establish the range over which this mechanism is useful in large repositories.

[1] Luo Q, Ye Y, Liang S, et al. Repoagent: An llm-powered open-source framework for repository-level code documentation generation[C]//Proceedings of the 2024 Conference on Empirical Methods in Natural Language Processing: System Demonstrations. 2024: 436-464.

[2] Luo J, Yin C, Zhang X, et al. Closing the Loop: Universal Repository Representation with RPG-Encoder[J]. arXiv preprint arXiv:2602.02084, 2026.

[3] Zhou Q, Zhang J, Wang H, et al. Featurebench: Benchmarking agentic coding for complex feature development[J]. arXiv preprint arXiv:2602.10975, 2026.

Questions for authors
  1. Could the authors further distill ORTG’s core design principle relative to existing approaches to semantic summarization, graph navigation, and incremental maintenance, and identify which experimental observations support that principle? I suggest organizing the method comparison, workflow explanation, and ablation interpretation around a common thread to highlight the new insights arising from the combination.

  2. How would disabling updates after editing affect the success rate when the initial outline, relation fields, access tools, and prompts are held constant? Could the authors inspect a sample of updates to assess whether the updated summaries remain consistent with the source code and whether subsequent steps within the same task actually use them?

  3. Please clarify how the FeatureBench metric corresponds to the official resolved rate and passed rate, and provide the instance counts, splits, fixed versions, and main execution configurations for the two benchmarks. The paper already states that initialization does not access the task to be solved. Could the authors additionally provide identifiers linking each task-instance snapshot to its initial outline and clarify when outline generation occurs relative to feature removal and environment cleanup?

  4. How is the ratio of complete outline length to context-window size distributed across the benchmarks, and how many runs actually enter two-level access mode? With the initial entries held constant, how do performance and entry coverage change under different context budgets?

Reviewer_XMUtOpenReview ↗
Rating
2
Confidence
3
Informativeness
1
Presentation
3
Soundness
2
Read review
Paper summary

This paper introduces ORTG, a repository context interface centered on file-level entity semantics with a bounded set of relation cues. ORTG provides mechanisms for localizing responsibility domains, accessing entity-level details, and incrementally updating repository context. The evaluation shows that ORTG can improve average task success rates, although these gains come with increased token consumption.

Critical strengths

1, Repository-level code generation and software engineering with LLM agents is an important and timely research problem.

Critical weaknesses
  1. Motivation for file-level entities. The motivation for choosing files as the entity granularity is unclear. Why is file-level representation preferable to module-, class-, or other semantic-level representations? A single file may contain multiple logically independent components. As a result, file-level entities may introduce substantial noise into the retrieved context. I would like to see a stronger justification for this design choice, ideally supported by an ablation comparing different levels of granularity.

  2. Unclear fundamental novelty over repository/code graph approaches. Code graphs and repository-level structural representations are not new concepts. Moreover, operations such as graph construction/extraction, localization, context access, editing, and incremental updates have been explored in prior works. The paper would benefit from a much clearer articulation of ORTG's fundamental technical novelty beyond existing repository/code graph approaches. Which specific design or algorithmic contribution is new, and why cannot existing code graph approaches achieve the same functionality?

  3. Limited gains relative to additional token cost. In Table 2, the improvement over SWE-agent + RepoGraph appears relatively modest, while Table 4 shows that ORTG consumes approximately 26% more tokens. This raises questions about its cost-effectiveness. The paper should provide a more explicit analysis of the tradeoff between task success rate and token consumption. In addition, why are Codex + RepoGraph/CodexGraph baselines not evaluated on Deep-SWE and FeatureBench? Such comparisons would help separate the benefits of ORTG's representation from those resulting from differences in the benchmark/agent/model.

  4. Limited evaluation on repository-level code generation benchmarks. The evaluation does not appear to cover dedicated repository-level code generation benchmarks such as RepoCraft, RepoZero, or RepoGenesis. Evaluation on these benchmarks would provide stronger evidence that the proposed representation generalizes beyond the currently tested software-engineering tasks and is effective specifically for repository-level code generation.

Questions for authors
  1. What motivates ORTG's choice of file-level entities rather than finer-grained semantic entities such as modules, or classes? A single file may contain multiple logically independent components, potentially introducing irrelevant relations and noisy context. Have the authors evaluated alternative granularities or measured the impact of file-level noise on localization and task performance?

  2. Could the authors clarify the fundamental technical novelty of ORTG compared with prior repository/code graph approaches?

  3. Table 2 appears to show relatively modest improvements over SWE-agent + RepoGraph, while Table 4 indicates approximately 26% higher token consumption. Could the authors provide further analysis of this accuracy-cost tradeoff in all evaluations?

  4. Why are Codex + RepoGraph/CodexGraph baselines not evaluated on Deep-SWE and FeatureBench?

  5. Could the authors evaluate ORTG on at least one repository-level code generation benchmarks such as RepoCraft, RepoZero, or RepoGenesis?

Comments on the paper

1 comment

Dear reviewers,

The authors have submitted their rebuttals. Please take a look and interact with the authors if necessary.

Thanks a lot.

AC

From OpenReview, CC BY 4.0. Authors as signed there; shown unedited.

Related papers

Loading the map…

Discussion (0)

Sign in to comment.