acceptodds
Under review as a conference paper at ICLR 2027

FastCode: Fast and Cost-Efficient Code Understanding and Reasoning

Abstract

Repository-scale code reasoning is a cornerstone of modern AI-assisted software engineering, enabling Large Language Models (LLMs) to handle complex workflows from program comprehension to complex debugging. However, balancing accuracy with context cost remains a significant bottleneck, as coding agents repeatedly process full code accumulated in their exploration history. To address this, we introduce FastCode, a framework that decouples repository exploration from content consumption. FastCode employs a structural scouting mechanism to navigate a lightweight semantic-structural map of the codebase, allowing the system to trace dependencies and pinpoint relevant targets without loading code bodies into the exploration context. By integrating structure-aware navigation with cost-aware context selection, the framework constructs high-value contexts for final reasoning while reducing repeated token overhead. Extensive evaluations on the SWE-QA, LongCodeQA, GitTaskBench, and LOC-BENCH benchmarks show that FastCode achieves favorable trade-offs between reasoning performance and token efficiency, highlighting its effectiveness for repository-scale code reasoning. Source code is available at https://anonymous.4open.science/r/FastCode.

Primary area:
foundation or frontier models, including LLMs
Keywords:
Large Language Models, LLM Agents, Code Reasoning, Context Management

Reviews

Among:
Shape:
1234this paper, Top 15%
Figure 1. Mean review rating of the reviewed papers at ICLR 2027. This paper averages 3.00 over 3 reviews, above 85% of the others.
Reviewer_CRybOpenReview ↗
Rating
2
Confidence
4
Informativeness
2
Presentation
3
Soundness
2
Read review
Paper summary

This paper proposes FastCode, an efficient framework for repository-level code understanding and reasoning. It targets the high token overhead that existing coding agents incur by repeatedly processing full code content during multi-round code exploration. The core idea is to decouple repository exploration from full-code reading, which reduces context cost while preserving iterative exploration. Specifically, FastCode first builds a lightweight semantic-structural representation by combining hybrid retrieval, hierarchical code units, and dependency, inheritance, and call graphs. It then progressively locates relevant code through query augmentation, tool-assisted search, and graph expansion, without loading full code implementations. Finally, a cost-aware context management policy dynamically controls the exploration process based on query complexity, context sufficiency, and resource budget, and selects the relevant code for final reasoning.

Critical strengths
  1. The research problem has practical significance, and the method design is reasonable.
  2. The experiments are fairly comprehensive, and the results are good.
  3. The paper is easy to follow.
Critical weaknesses

I have the following concerns about this paper:

  1. I have some confusion about the optimization objective in the method section. In Equation (1) of Section 3.2, the authors formulate context selection as an optimization problem that balances relevance and computational cost. However, the paper does not seem to clearly specify how the relevance function Rel⁡(C∣q)\operatorname{Rel}(C\mid q), the cost function Ω(C)\Omega(C), and the trade-off coefficient λ\lambda are defined and computed in the actual method. Moreover, Section 4.3 mainly relies on heuristic strategies based on confidence and budget constraints to manage the context, and the relationship between these strategies and the optimization objective in Equation (1) remains unclear.
  2. The method section describes the complementary roles of BM25 and dense retrieval, but it does not specify how the two retrievers are fused, or what content is used to build the vector index. In addition, I have some confusion about the definition of the Information Gain Rate (IGR) in Section 4.3.2. The authors measure the exploration gain as the ratio of the change in the context-sufficiency score to the change in the number of candidate code lines. However, during iterative filtering, the total number of candidate lines may stay the same or even decrease. This could make the denominator of IGR zero, or give it a sign that does not accurately reflect the change in context sufficiency.
  3. FastCode mainly relies on lightweight structural metadata for exploration, but the authors use the LLM’s self-assessed κt\kappa_t to decide whether to continue retrieval or terminate early. Without access to the full code implementation, the model may struggle to judge accurately whether the current candidates already contain all the evidence needed to solve the problem.
  4. In the Cost Calculation paragraph of Appendix A.2, the authors explicitly state: "For baselines, we directly adopt the cost figures reported in their original publications." In other words, FastCode’s cost is computed from the API services and pricing used in this paper, such as Poe and OpenRouter, whereas some baseline costs are taken from numbers reported in prior papers. My biggest concern here is that costs reported in different papers are not necessarily directly comparable. Furthermore, the fairness of the execution protocol on GitTaskBench needs further clarification. FastCode generates an executable .sh script that is run by a predefined automation script, whereas SWE-agent and OpenHands are interactive coding agents. The cost advantage may therefore come partly from differences in execution protocol, rather than entirely from Structural Scouting. In addition, FastCode needs to build a BM25 index, an embedding index, and an AST relation graph, and it performs multiple rounds of exploration. Do the reported costs include these preprocessing operations?
  5. How much of FastCode’s gain comes from "not reading the full code during exploration," and how much comes from query augmentation, graph retrieval, and other engineering components? The ablation study cannot determine this.
  6. I have some concerns about whether the backbones are comparable across methods. In Table 7, FastCode’s best result on LOC-BENCH (86.13%) uses Gemini-3-Flash, while all baselines use Claude 3.5 Sonnet. Under the same backbone, FastCode’s Acc@1/3/5 are all lower than those of LocAgent. Similarly, in Section 5.3, FastCode with Gemini-3-Flash outperforms OpenHands with Claude 3.7 Sonnet, but Gemini-3-Flash is a newer-generation model, and "lightweight" does not mean weaker. I suggest the authors compare methods on the same backbone to isolate the framework's contribution.

Minors

  1. The text in Figure 1 is too small to read comfortably. In addition, Figure 1 uses radar charts to compare Accuracy and Cost across benchmarks, but the left chart lacks a clear numerical scale, and the percentage ticks in the right chart are not evenly spaced.
  2. The paper uses two spellings, "locBench" and "LOC-BENCH", which should be unified.
  3. On P4, "Iniitialization" in Figure 2 is misspelled.
  4. On P5, L238–239, "returns file path and match counts and without loading full files" contains a redundant "and".
Questions for authors

Please see the Weaknesses above; the relevant questions are all included there.

4 comments
AuthorsOpenReview ↗

Response to Reviewer CRyb (1/2)

1. Optimization objective and context management (W1)

Response to W1: We thank the reviewer for the thoughtful question. Eq. (1) expresses our goal of obtaining sufficient query-relevant evidence at low computational cost. The context-selection objective is C∗=arg⁡max⁡C⊂US(C;q)C^*=\arg\max_{C\subset\mathcal{U}}S(C;q), where S(C;q)=Rel⁡(C∣q)−λΩ(C)S(C;q)=\operatorname{Rel}(C\mid q)-\lambda\Omega(C). For each candidate context, the LLM assesses query coverage and evidence completeness using metadata such as signatures, docstrings, retrieval evidence, and structural relations (Sec. 4.2.2). It assigns a sufficiency score κt∈[0,100]\kappa_t\in[0,100], which estimates Rel⁡(C∣q)\operatorname{Rel}(C\mid q) (Appendix A.2.1). Ω(C)\Omega(C) captures two cost components: code volume LtL_t for final context consumption and exploration depth tt for repeated scouting.

The trade-off represented by λ\lambda is realized through the confidence and budget controls in Sec. 4.3. Specifically, exploration stops when the current context is judged sufficient, as determined by the threshold τ\tau. When evidence remains insufficient, exploration continues. Information-gain monitoring links relevance and cost by assessing sufficiency gains relative to added code volume, stopping after consecutive low-gain rounds. Finally, the budget bounds code volume and exploration depth, with final selection prioritizing strongly supported units within it. Together, these mechanisms implement the relevance-cost balance in Eq. (1). We will explicitly add the correspondence between the task objective and this policy in the revised manuscript.

2. Hybrid retrieval and IGR definition (W2)

Response to W2: We thank the reviewer for these questions. Hybrid retrieval combines semantic similarity with lexical matching. Embedding and BM25 candidates are merged by unit ID and ranked by the fusion score S(u)=0.5cos⁡(eq,eu)+0.5 BM25(q,u)/max⁡vBM25(q,v)S(u)=0.5\cos(e_q,e_u)+0.5\,\mathrm{BM25}(q,u)/\max_v\mathrm{BM25}(q,v), where eq,eue_q,e_u are query/unit embeddings and missing scores are zero. Graph expansion follows fusion. Each file/class/function/documentation unit is embedded from its type, name, signature, docstring, summary, and bounded code text. Paths and line ranges remain metadata.

Due to space constraints, the main-text IGR equation does not detail the handling of unchanged or reduced context volume. IGR measures sufficiency gain per added code volume, so its ratio is evaluated only for positive growth. With Δκ=κt−κt−1\Delta\kappa=\kappa_t-\kappa_{t-1} and ΔL=Lt−Lt−1\Delta L=L_t-L_{t-1}, the implementation uses IGR=Δκ/ΔLIGR=\Delta\kappa/\Delta L when ΔL>0\Delta L>0; otherwise it skips division and records zero as a placeholder, avoiding division by zero and sign reversal. This placeholder is assessed jointly with Δκ\Delta\kappa: a round remains productive when the sufficiency score improves meaningfully, even if context shrinks. Low-progress stopping requires two consecutive rounds with either a substantial confidence decline or both a small gain and low IGR. A single corrective decline is tolerated; sufficiency and resource limits apply separately. We will add the fusion rule, embedding inputs, and guarded IGR and stopping definitions to the appendix.

3. Reliability of context-sufficiency assessment (W3)

Response to W3: We thank the reviewer for this question. The LLM assigns the sufficiency score κt\kappa_t based on the candidate units' structured metadata, including signatures, docstrings, retrieval evidence, and structural relations. Function names, parameters, and type annotations provide cues about functionality and interfaces, while docstrings describe intended behavior. Call and dependency relations reveal how units interact and which related implementations may be needed. Together, these signals support assessing whether the candidate set covers the functional roles and dependencies required by the query, grounding the sufficiency estimate in both semantic and structural evidence.

Table 9 provides empirical support on the three-repository SWE-QA subset: base sufficiency thresholds of 65, 80, and 95 yield answer scores of 40.79, 41.52, and 43.19, respectively. The improvement under stricter sufficiency requirements supports the practical value of metadata-based coverage assessment for context selection.

AuthorsOpenReview ↗

Response to Reviewer CRyb (2/2)

4. Cost accounting and execution protocol (W4)

Response to W4: We thank the reviewer for these questions. For the SWE-QA and LongCodeQA costs in Figure 4, we calculate API costs for FastCode and non-commercial baselines using the rates in Table 3. For commercial tools, we use their console-reported costs. For GitTaskBench and LOC-BENCH, matched-backbone comparisons use the same rates across methods: USD 3 per million input tokens and USD 15 per million output tokens for Claude 3.5/3.7 Sonnet [1-2]. This aligns the pricing basis for FastCode and baselines such as OpenHands, SWE-agent, and LocAgent.

On GitTaskBench, the automation script only runs FastCode's generated .sh file; all task-specific operations come from FastCode, with no additional task-solving logic supplied by the executor. SWE-agent and OpenHands can likewise generate and execute scripts through their shell tools to complete tasks. All methods follow the same task requirements and ECR/TPR evaluation criteria.

FastCode's reported API cost includes every task-solving LLM call, covering query augmentation, multi-round exploration, and final reasoning. BM25 and embedding indexes and AST relation graphs are built locally on CPU as repository preprocessing and can be reused across queries. These operations incur no LLM API charges. We will revise Appendix A.2 to distinguish the pricing rules and document the execution protocol and accounting scope.

[1] GitTaskBench: A Benchmark for Code Agents Solving Real-World Tasks Through Code Repository Leveraging. arXiv preprint, 2025.

[2] LocAgent: Graph-Guided LLM Agents for Code Localization. ACL, 2025.

5. Component contributions in the ablation study (W5)

Response to W5: We thank the reviewer for this question. The contributions of these components are already examined in Section 5.5 (Ablation Study) and Fig. 5, as detailed below.

(1) Structural scouting reduces repeated token processing by keeping code bodies out of exploration history. Its tool-based navigation contribution is assessed by comparing w/o C with w/o T&C: with query augmentation retained and cost management disabled in both, removing tool-based scouting increases the reported performance drop from 0.92% to 4.31%.

(2) Query augmentation bridges query intent and code terminology through rewriting and implementation hints. Both w/o T&C and w/o N&C disable tool-based scouting and cost management; additionally removing query augmentation increases the drop from 4.31% to 5.06%.

(3) Graph retrieval captures related code beyond initial similarity matches through call, dependency, and inheritance relations. Removing graph expansion alone (w/o G) reduces performance by 0.83%.

(4) For other components, hybrid retrieval supports semantic matching and precise identifier lookup; removing it (w/o R) causes a 0.60% drop. Cost-aware context management regulates exploration according to sufficiency and budgets; removing it (w/o C) causes a 0.92% drop.

6. Comparisons with the same backbone (W6)

Response to W6: We thank the reviewer for raising this point. We already compare FastCode and LocAgent on LOC-BENCH using the same Claude 3.5 Sonnet backbone in Appendix A.3.1 (Table 7). FastCode achieves 76.28% Acc@1 at USD 0.365 per instance, compared with 77.74% at USD 0.660 for LocAgent. The Acc@1 gap is 1.46 percentage points, while FastCode's reported cost per instance is 44.7% lower.

On GitTaskBench, we also report comparisons using the same backbone for FastCode, SWE-agent, and OpenHands in Section 5.3 (Figure 3) and Appendix A.3.1 (Table 6), covering both Claude 3.5 Sonnet and Claude 3.7 Sonnet. FastCode leads both baselines in Execution Completion Rate (ECR) and Task Pass Rate (TPR) with either backbone. With Claude 3.5 Sonnet, FastCode achieves 61.11% ECR and 46.30% TPR, versus 53.70% and 40.74% for OpenHands. With Claude 3.7 Sonnet, the corresponding values are 74.07% and 57.41% for FastCode and 72.22% and 48.15% for OpenHands. The Gemini-3-Flash results serve as a supplementary evaluation, showing FastCode's performance with an additional LLM backbone.

Minor comments on figures and wording

Thank you. We will enlarge Fig. 1's text and add a scale to its left chart. The right chart's uneven ticks show differences in cost. We will refine them for clarity while preserving that contrast. We will use LOC-BENCH consistently, fix Fig. 2's typo, and remove the extra 'and'.

Reviewer_CRybOpenReview ↗

Final Assessment and Rationale for Maintaining Weak Reject

Thank the authors for their detailed responses and additional experiments. However, after careful reconsideration, I remain inclined to maintain my original score (Weak Reject) for the following primary reasons:

  • Lack of Technical Novelty. This remains my primary concern. FastCode mainly integrates existing techniques, including BM25, Dense Retrieval, Graph-based Retrieval, Query Augmentation, and Agentic Exploration. The authors have not convincingly demonstrated that their core "exploration-consumption decoupling" provides a substantial methodological contribution. In my view, this work represents an engineering integration rather than a significant methodological innovation.
  • Heuristic-based Optimization. The proposed "cost-aware optimization" relies heavily on predefined heuristic confidence thresholds, line budgets, and stopping rules, lacking a principled optimization mechanism.
  • Insufficient Validation. The reliability of the LLM-based context sufficiency assessment remains unverified, and the current ablation studies do not adequately isolate and prove the independent contribution of the core mechanisms.
  • Limited Evidence for Efficiency and Generalization. The claimed efficiency advantage is not consistent across baselines. On SWE-QA, FastCode achieves a higher score (43.28) than Func RAG (39.62) and Sliding RAG (40.22), but its per-question cost (USD 0.0320) is also substantially higher than theirs (USD 0.0112 and USD 0.0200, respectively). Similarly, on LOC-BENCH under the same Claude 3.5 Sonnet backbone, FastCode achieves lower localization accuracy than LocAgent (Acc@1: 76.28% vs. 77.74%), despite its lower cost (USD 0.365 vs. USD 0.660). These results suggest an accuracy-cost trade-off rather than a consistent efficiency advantage. Furthermore, the cross-language evaluation covers only 22 queries across three repositories, with just one query for greenlet, which is insufficient to establish robust generalization across programming languages.

Finally, to be candid, I respectfully disagree with Reviewer XMa8's assessment. While this paper holds a certain degree of practical engineering value, its technical novelty is insufficient to justify a score of 4 (Clear Accept). The completeness of an engineering implementation and the importance of the research problem should not be conflated with technical innovation.

AuthorsOpenReview ↗

Response to Reviewer CRyb's Final Assessment

We thank the reviewer for the follow-up and appreciate the opportunity to address the additional points raised in this comment.

1. Novelty.

We do not regard the use or combination of BM25, dense retrieval, graph-based retrieval, and similar techniques as a contribution of this paper. Our contribution is to decouple repository exploration from code consumption, which changes how the agent loop is organized.

In iterative agents such as SWE-agent and OpenHands, code read during exploration stays in the history and is re-processed in every later round, so cost grows with both the amount of code inspected and the number of rounds. In FastCode, the exploration loop sees only metadata (e.g., signatures, docstrings, and dependency relations), and code bodies are loaded once, after exploration ends. This is a change to the method rather than an added component: exploration stays iterative and adaptive, but no code is re-processed across rounds. The established techniques above serve to realize this idea more effectively: they supply and rank the candidates the agent examines.

The paper shows that an agent can explore a repository without loading code bodies into its context, tracing dependencies and pinpointing the code a query needs by navigating a lightweight semantic-structural map, and that this changes how cost scales. For example, on LongCodeQA, FastCode's cost stays within USD 0.041–0.069 as repositories grow from 32K to 256K tokens, while direct input rises from USD 0.083 to 0.666 (Table 5). On GitTaskBench with the same backbone, FastCode costs USD 0.0878 per task vs. 1.38 (SWE-agent) and 8.95 (OpenHands) (Table 6).

2. Cost-aware optimization.

The cost-aware policy aims to gather sufficient query-relevant evidence at low cost, by balancing estimated context sufficiency against exploration depth and code volume. It has three advantages.

(1) Adaptive. Rather than applying fixed values, the policy itself adjusts the line budget, iteration limit, and sufficiency threshold at run time, according to the estimated query complexity and the structural complexity of the repository.

(2) Efficient and training-free. The policy needs no training and no additional model. It ends exploration once the evidence is judged sufficient or added code brings little further gain, and it works across multiple backbones.

(3) Stable. The policy keeps per-query cost in a narrow range as repositories grow (Table 5).

3. Validation.

(1) Sufficiency assessment. For each candidate, the LLM sees its signature and docstring, which indicate what the unit does; retrieval and tool evidence, which show how it matches the query; and call, dependency, and inheritance relations, which show which other units it relies on. From these, it can judge whether the candidates cover the functions and dependencies the query involves. Two results support that the LLM-based context sufficiency assessment is effective. (a) The sufficiency score tracks answer quality: as its mean rises from 83.0 to 91.7, the answer score rises from 40.79 to 43.40 (Table 9). (b) Context selected under this assessment matches or exceeds full-repository input, which contains all evidence, in nearly all settings (Table 2).

(2) Ablation. Sec. 5.5 (Fig. 5) ablates multiple mechanisms with the rest fixed, and each removal lowers the score: hybrid retrieval (w/o R, −0.60%), graph expansion (w/o G, −0.83%), cost-aware control (w/o C, −0.92%), tool-based scouting (a further −3.39%, w/o C to w/o T&C), and query augmentation (a further −0.75%, w/o T&C to w/o N&C).

Details are in our Responses to W3 and W5.

4. Efficiency and generalization.

An accuracy–cost trade-off is our stated claim: the abstract, Sec. 5.4, and Fig. 4 report favorable trade-offs, not the lowest cost on every benchmark. Cost comparisons are meaningful between methods of comparable accuracy.

(1) SWE-QA and LOC-BENCH. The RAG baselines are not at a comparable level: they score 3.06–3.66 points below FastCode. FastCode matches Cursor and exceeds Claude Code (43.28 vs. 43.17 and 42.08) at 55% and 44% lower cost. Against LocAgent, it is within 1.46 points at Acc@1 at 44.7% lower cost.

(2) LongCodeQA and GitTaskBench. FastCode leads on both accuracy and cost (Claude 3.5). On LongCodeQA, it matches or exceeds full-repository input on every subset from 32K to 256K, at 50.6–91.4% lower cost (Table 5). On GitTaskBench, it exceeds SWE-agent and OpenHands in ECR and TPR at 93.6% and 99.0% lower cost (Table 6).

(3) Cross-language. The paper makes no claim of cross-language generalization and therefore does not evaluate it. The 22-question experiment was added at Reviewer fFXD's request for at least one non-Python repository; we reported every LongCodeQA question for three of them.

We hope these responses address the reviewer's concerns and would appreciate a reconsideration.

Reviewer_fFXDOpenReview ↗
Rating
3(+1)
Confidence
3
Informativeness
2
Presentation
2
Soundness
3
Read review
Paper summary

The author (s) propose Fastcode, a framework for repository-scale code reasoning that decouples exploration, using lightweight metadata (e.g., signatures, docstrings, line ranges), from full code context consumption, via a three-layer (dependency, inheritance, call graph) structure. They also introduced a heuristic, cost-aware control policy which decides which code is relevant and which part to load for final reasoning. They evaluated the framework on four benchmarks (SWE-QA, LongCodeQA, GitTaskBench, and LOC-BENCH) on different baselines like direct-LLM, RAG, and other agentic(SWE-agent) and commercial(Claude Code, Cursor) baselines.

Critical strengths
  • The paper frames a clear and practical problem. It aims to achieve cost-efficient code reasoning without sacrificing accuracy.

  • The evaluation is comprehensive with four benchmarks and multiple baselines.

  • The backbone comparison is insightful, suggesting the framework can improve both small and large models.

Critical weaknesses
  • The performance gain over baselines is small and lacks statistical significance. On the SWE-QA dataset, the comparison is FastCode 43.28 vs. Cursor 43.17 (+0.11) and vs. SWEQA-Agent 42.33 (+0.95). The paper reports only a single-point result without any standard deviation. Without multiple runs, the claim that FastCode “overperforms” the commercial baselines is an overstatement, in my opinion.

  • Table 1 reports a "3-repository subset" of SWE-QA, but in the main text of Section 5.2, 43.28 is reported as “the” SWE-QA result. This makes it unclear whether the accuracy claim applies to the whole benchmark or only the subset.

  • Cost comparisons are not apple-to-apple comparisons, as the authors listed the baseline costs from the original papers. The baselines may be computed with different backbones with different API pricing.

  • The ablation study on different components also lacks statistical significance. With small individual component deltas (w/o R: −0.60%, w/o G: −0.83%, w/o C: −0.92%), the results do not necessarily establish that individual components are necessary.

  • The framework and evaluation are only Python-specific; this narrows the scope of the framework.

Questions for authors
  • Can the authors run a significance test over at least 3 runs for Fastcode vs other commercial tools?

  • Is the 5.2 section result on the 3-repository subset or the whole benchmark? Can the authors report both?

  • Can the authors re-run the baseline using the same backbones and API pricing?

  • Do the authors have any insights into the ablation? Why do the individual components have little significance? Doesn’t it question the architecture of the framework?

  • Can the authors add results of at least one non-Python repository to show the generalizability of the framework?

5 comments
AuthorsOpenReview ↗

Response to Reviewer fFXD (1/2)

1. SWE-QA performance variability (W1)

Response to W1 and Q1: We thank the reviewer for raising this point. We conducted three independent full runs of FastCode on all 144 questions in the three-repository SWE-QA subset using the Table 1 protocol.

Run scoresMean ±\pm SD95% CI for the mean
43.28, 43.42, 43.4543.38±0.0943.38 \pm 0.09[43.16,43.61][43.16, 43.61]

The standard deviation of 0.09 points indicates stable FastCode scores across the three runs. FastCode's mean is 0.21 points above Cursor's reported 43.17 and 1.05 points above SWEQA-Agent's reported 42.33. The 95% confidence interval for FastCode's mean includes Cursor's reported score. We will add these statistics to the appendix and describe FastCode's performance against commercial baselines on this subset as competitive.

2. Scope of the SWE-QA evaluation (W2)

Response to W2 and Q2: We thank the reviewer for pointing this out. The caption of Table 1 identifies the three-repository SWE-QA subset. Section 5.2 refers to Table 1 but leaves the subset scope implicit. All SWE-QA results shown in Table 1 and discussed in Section 5.2 use this same subset of 144 questions. We will explicitly identify this subset in Section 5.2. Appendix A.3.4 (Tables 10–12) reports results for the full SWE-QA benchmark across all 12 repositories.

3. Cost comparability across methods (W3)

Response to W3 and Q3: We thank the reviewer for raising this concern. For the SWE-QA and LongCodeQA costs in Figure 4, we calculate API costs for FastCode and non-commercial baselines using the rates in Table 3. For commercial tools, we use their console-reported costs. For matched Claude comparisons on GitTaskBench and LOC-BENCH [1-2], we use USD 3 per million input tokens and USD 15 per million output tokens for FastCode and the baselines.

Tables 5–7 include comparisons using the same backbone and the same API pricing. At 256K on LongCodeQA, Claude 3.5 Sonnet costs USD 0.057 per instance with FastCode and USD 0.666 with direct input (Table 5). On GitTaskBench (Table 6), FastCode, SWE-agent, and OpenHands cost USD 0.0878, USD 1.38, and USD 8.95 per task with Claude 3.5 Sonnet. The corresponding costs with Claude 3.7 Sonnet are USD 0.1015, USD 1.67, and USD 29.80. On LOC-BENCH, FastCode costs USD 0.365 per instance and LocAgent costs USD 0.660 with Claude 3.5 Sonnet (Table 7). We will clarify the pricing conventions and the sources of the baseline costs in Appendix A.2.

[1] GitTaskBench: A Benchmark for Code Agents Solving Real-World Tasks Through Code Repository Leveraging. arXiv preprint, 2025.

[2] LocAgent: Graph-Guided LLM Agents for Code Localization. ACL, 2025.

AuthorsOpenReview ↗

Response to Reviewer fFXD (2/2)

4. Component roles and nested ablations (W4)

Response to W4 and Q4: We thank the reviewer for raising this point. The single-component ablations measure each module's contribution with the rest of FastCode held fixed. The retained mechanisms can still retrieve relevant evidence. Removing hybrid retrieval (w/o R) lowers the score by 0.60%, while tool-assisted search can still locate relevant code units. Removing graph expansion (w/o G) lowers the score by 0.83%. This drop suggests that graph expansion provides useful evidence beyond hybrid retrieval and tool-assisted search by reaching code units through call, dependency, and inheritance links. Removing cost-aware context management (w/o C) lowers the score by 0.92%. Retrieval and scouting remain active in this variant and can still collect key evidence.

C regulates decisions during tool-assisted scouting (T), so the variants that remove T also disable C. With C disabled and query augmentation retained in both variants, moving from w/o C to w/o T&C removes tool-assisted scouting and increases the reported score drop from 0.92% to 4.31%. The additional drop shows that tool-assisted scouting is effective in improving answer quality even with retrieval and query augmentation in place (Fig. 5).

Highly relevant code units often have high semantic similarity to the query or share its key terms. Targeted tool searches can also locate these units by matching identifiers or usage patterns. Because multiple routes can surface the same units, the retained mechanisms are more likely to find the most relevant evidence and preserve much of the answer score. Complementary mechanisms can recover secondary evidence, such as callers or dependencies that clarify how the main implementation is used. These complementary contributions are important for answer completeness even when the core implementation has already been identified.

5. Language support and evaluation coverage (W5)

Response to W5 and Q5: We thank the reviewer for raising this point. Our evaluation focuses primarily on repositories in the Python ecosystem. FastCode supports multiple languages through language-specific parsers that extract code units with shared metadata, including names, signatures, and source ranges. For example, FastCode extracts Rust and C/C++ code units from syntax trees produced by Tree-sitter. Retrieval, scouting, and context management use this common representation across languages.

Several benchmarks in our evaluation, including SWE-QA, LongCodeQA, and GitTaskBench, contain source code in languages beyond Python. We counted source bytes by file extension in the repository snapshots, excluding shell, PowerShell, and batch scripts. The table below shows the breakdown.

BenchmarkRepos with non-Python codeNon-Python share of source bytes
SWE-QA (full benchmark)8/1212.4%
LongCodeQA26/9820.3%
GitTaskBench4/1828.4%

Following the LongCodeQA setup in Section 5.2 (Table 2), we use Gemini 2.5 Pro to compare full-repository input with FastCode-selected context on three repositories dominated by Rust (90.3%), C++ (71.38%), and C (66.16%), respectively. For example, the questions ask about a musl-target crash involving the Rust crates pyembed and pyo3 and server-side cursor behavior implemented in psycopg2's C extension.

Across the 22 questions, FastCode answered 21 correctly versus 19 with full-repository input. The results suggest that FastCode can also select task-relevant evidence in repositories dominated by languages other than Python while using fewer code tokens for final reasoning. We will add the language breakdown and the task results for these repositories to the appendix.

RepositorySubsetShare of source bytesFull repository+ FastCode
PyOxidizer1MRust 90.3%92.86% (13/14)100.00% (14/14)
greenlet128KC++ 71.38%, headers 19.87%100.00% (1/1)100.00% (1/1)
psycopg2512KC 66.16%, headers 12.18%71.43% (5/7)85.71% (6/7)

Thank you again for your careful review. We have provided additional experiments and clarifications in response to your comments. We would appreciate your feedback on whether these address your concerns, along with any further comments or suggestions you may have.

Reviewer_XMa8OpenReview ↗
Rating
4
Confidence
3
Informativeness
2
Presentation
3
Soundness
3
Read review
Paper summary

This paper targets an important problem in coding agents: how to efficiently explore code repo without harming performance. The authors propose a comprehensive pipeline that help coding agent progressively fill in its contexts. There are three major designs: (1) a better retriever that combines program structural information (e.g., call graph, dependence graph) and textual information (semantic embedding or BM25 index); (2) a progressive exploration tool set that returns func signatures before return the whole content. So that the model can decide whether to open a file based on signatures without filling the contexts; (3) an adaptive context control that use heuristics to compute the budget at a given step, and adaptively decide when to end the exploration.

The evaluation is conducted on four datasets. The compared baseline includes commercial agents like Claude Code and different RAG based approaches. The results demonstrate slightly better performance with much better efficiency.

Critical strengths

This paper works on a very important direction that can reduce the cost in coding tasks while without affecting performance. While LLMs are becoming more capable, it remains a challenge how to efficiently obtain information from code repo. This work incorporates deep domain knowledge into the proposed harness, achieving much better efficiency than existing solutions.

The paper is well presented and the engineering design is comprehensive. It would be a useful tool based on the shown results.

Critical weaknesses

None

Questions for authors
  1. For Fig4, I didn't find performance for RAG-baselines?

  2. Why not evaluating on some common coding tasks such as SWE-Bench? Is CodeQA different from bug fixing tasks? What is the generalization boundary of the proposed harness?

3 comments
AuthorsOpenReview ↗

Response to Reviewer XMa8

1. RAG baseline performance and cost (Q1)

Response to Q1: We thank the reviewer for this question. The RAG performance results are reported in Tables 1 and 2. On SWE-QA, Func RAG and Sliding RAG score 39.62 and 40.22, respectively, compared with FastCode's 43.28 using the same backbone. On LongCodeQA, Table 2 reports File BM25 RAG across context lengths. At 256K with Claude 3.5 Sonnet, File BM25 RAG, full-repository input, and FastCode achieve 25.95%, 66.60%, and 78.50% accuracy, respectively.

We have additionally calculated the SWE-QA RAG costs. Following the SWE-QA implementation, both baselines use voyage-code-3 to build embedding indexes. We include answering costs and amortize one-time indexing charges over the 144 questions. Func RAG scores 39.62 at USD 0.0112 per question, and Sliding RAG scores 40.22 at USD 0.0200. FastCode achieves a balance between performance and cost, scoring 43.28 at USD 0.032 per question, compared with Cursor's 43.17 at USD 0.071. We will add the RAG performance–cost points to Figure 4a.

MethodSWE-QA total scoreCost per question (USD)
Func RAG39.620.0112
Sliding RAG40.220.0200
Cursor43.170.0710
FastCode43.280.0320

For LongCodeQA, Table 2 shows that File BM25 RAG has substantially lower accuracy than both full-repository input and FastCode across the context lengths where their results are available. Figure 4b focuses on cost scaling with context length for direct LLM input and FastCode. We will clarify this scope in the revised manuscript.

2. Evaluation scope and applicability to bug fixing (Q2)

Response to Q2: We thank the reviewer for this question.

(1) Why not SWE-Bench. The core of FastCode is to efficiently acquire task-relevant context, rather than patch generation or test-based validation. SWE-Bench’s resolve rate compounds localization, editing, and validation, and is strongly influenced by the backbone’s editing ability and the harness’s retry loop, so it cannot isolate the stage we contribute to. We therefore evaluated that stage directly on SWE-Bench data: LOC-BENCH (Table 7) consists of 274 issue–patch pairs from SWE-Bench-Lite and measures whether the files modified by the gold patch are correctly localized. With the same backbone (Claude 3.5 Sonnet), FastCode achieves localization accuracy close to that of LocAgent (76.28% vs. 77.74% Acc@1) while reducing cost by 44.7%.

(2) CodeQA vs. bug fixing. The two tasks share a common first step and differ in what follows. Both begin by understanding the repository to identify the code that the request concerns, and this is the step FastCode optimizes. Bug fixing then goes further: it must produce precise code edits and verify them through execution, such as reproducing the failure and running tests, and this edit–verify cycle is typically repeated until the fix passes. Repository understanding is thus an important foundation for bug fixing, though it does not cover the whole process.

(3) Generalization boundary. FastCode is motivated by the difficulty of balancing accuracy with context cost in repository-scale code reasoning, as coding agents repeatedly process the full code accumulated in their exploration history. It is therefore suited to code understanding and reasoning tasks, where the outcome depends on identifying the relevant code in a repository and reasoning over it, such as repository-level question answering (SWE-QA, LongCodeQA), issue localization (LOC-BENCH), and tasks built on existing repositories (GitTaskBench). It is less suited to tasks whose success depends mainly on repeated edit–verify cycles or on runtime information, where code understanding is only one part of the process.

The response addresses my questions. I will keep my positive rating.

Thank you for your thoughtful feedback and constructive discussion. We appreciate your time and positive assessment of our work.

Comments on the paper

1 comment
AuthorsOpenReview ↗

Summary of the Rebuttal

Dear Area Chair, Senior Area Chairs, and Program Chairs,

We sincerely thank you for your time and effort in handling our submission, and we are grateful to all reviewers for their constructive comments. Given the substantial workload of going through the manuscript and the full discussion history, we provide this brief summary of the rebuttal process to aid your assessment.

We greatly appreciate the reviewers’ recognition of our work. They acknowledged that (1) FastCode targets an important and practical problem: cost-efficient repository-scale code reasoning without sacrificing accuracy (XMa8, fFXD, CRyb); (2) the method design is reasonable and comprehensive, incorporating deep domain knowledge (XMa8, CRyb); (3) the evaluation is comprehensive, with four benchmarks and multiple baselines (fFXD, CRyb), shows much better efficiency than existing solutions (XMa8), and includes an insightful backbone comparison (fFXD); and (4) the paper is clear and easy to follow (XMa8, CRyb).

We provided detailed, point-by-point responses to every weakness and question, summarized in six points.

(1) Statistical reliability (fFXD): we added repeated independent runs confirming stable results, refined the related claim, and clarified the benchmark scope.

(2) Cost fairness (fFXD, CRyb): we clarified the pricing conventions, cost accounting scope, and execution protocol, and showed that FastCode’s cost advantage holds under the same backbone and pricing.

(3) Evaluation scope (XMa8, fFXD): we added RAG baseline costs, explained how localization on SWE-Bench data isolates the stage we contribute to, stated FastCode’s applicability boundary, and added experiments on Rust-, C++-, and C-dominated repositories, where FastCode remains effective.

(4) Method details and validation (CRyb): we specified how the optimization objective maps to the context-management policy, the retrieval fusion rule, and the information-gain definition, and provided empirical support for the sufficiency assessment.

(5) Ablation (fFXD, CRyb): we explained that the ablation study covers every key component and that each one contributes, as every removal lowers the score; that single-component drops are limited because complementary mechanisms can still recover the key evidence; and that the nested ablations isolate each component’s contribution, with tool-based scouting contributing the most.

(6) Second-round comments on novelty, the cost-aware policy, and efficiency (CRyb): we clarified that our contribution is not the combination of existing retrieval techniques but the decoupling of repository exploration from code consumption, which reorganizes the agent loop and changes how cost scales with repository size; that the cost-aware policy is adaptive, training-free, and stable; and that the efficiency results are consistent with the favorable accuracy–cost trade-off stated in the paper.

After the rebuttal, Reviewer XMa8 confirmed that our response addressed the questions and maintained a rating of 4 (Clear Acceptance). Reviewer fFXD acknowledged that most concerns were addressed and raised the score from 2 to 3 (Weak Acceptance). Reviewer CRyb engaged with us through two rounds of discussion: after the first round, the reviewer thanked us for the detailed responses and additional experiments and maintained the original score. In the second round, we clarified and answered each of the follow-up comments point by point.

Throughout the rebuttal, we have made every effort to address each concern raised by the reviewers, and we hope the outcomes of this process will be fully considered in your final decision. Thank you again for your time and consideration.

From OpenReview, CC BY 4.0. Authors as signed there; shown unedited.

Related papers

Loading the map…

Discussion (0)

Sign in to comment.