acceptodds
Under review as a conference paper at ICLR 2027

DF-Fuzz: Dependency-Guided Context Retrieval for LLM-Assisted Language Processor Fuzzing

Abstract

Language processor fuzzing assisted by large language models (LLMs) has shown substantial promise in synthesizing test programs that satisfy high-level syntactic and semantic specifications to penetrate deep code validations. However, existing LLM-driven constraint-solving fuzzers face a severe semantic scarcity bottleneck when testing large-scale compiler systems. Conventional physical function slicing retains only the enclosing function containing the target bottleneck basic block, severing critical semantic linkages to external type declarations, preprocessor macros, and class hierarchies. Consequently, LLM-generated test programs suffer from low compilation pass rates. Meanwhile, interactive retrieval designs incur prohibitive round-trip latency and token overhead, frequently triggering format hallucinations and state-machine deadlocks. To overcome these limitations, we present DF-Fuzz, a dependency-guided context retrieval framework for language processor fuzzing. DF-Fuzz introduces the Dual-Flow Hybrid Dependency Graph (DF-HDG) to unify intra-procedural control constraints and inter-procedural, cross-file data dependencies within an offline-constructed representation. Operating via a dual-track architecture, DF-Fuzz performs reverse dominator tree slicing to isolate the minimal condition chains governing branch reachability, while concurrently employing inter-procedural symbol resolution with namespace wrapping to reconstruct self-contained scopes. To mitigate context bloat, DF-Fuzz integrates Jaccard-similarity-based semantic pruning with an API co-occurrence few-shot retrieval mechanism. Furthermore, DF-Fuzz incorporates an adaptive feedback-driven self-healing loop that translates local compiler diagnostic errors into dynamic exemption weights, repairing omitted context in subsequent iterations at zero additional LLM interaction cost. We evaluated DF-Fuzz across eight real-world language processors spanning six programming languages, including Clang++, Flang, Hermes, and Lua. In 24-hour comparative evaluations, DF-Fuzz achieved 100,153 edge coverage on Clang++, outperforming AFL++ by 78.8% and the state-of-the-art LLM baseline HLPFuzz by 5.7%, while reducing API operational costs by 60.3% and increasing execution throughput by 26.9%. In extended testing across all targets, DF-Fuzz discovered 45 previously unknown vulnerabilities, including 10 heap use-after-free and 8 heap buffer overflow vulnerabilities. Our code and artifacts are publicly available at https://osf.io/dkqz4/overview?view_only=d75600e5f99f41a6a849bbc36515e919.

Primary area:
applications to computer vision, audio, language, and other modalities
Keywords:
Fuzzing, Compiler Testing, Large Language Models, Context Retrieval Dependency Analysis, Vulnerability Detection

Reviews

Among:
Shape:
1234this paper, Top 49%
Figure 1. Mean review rating of the reviewed papers at ICLR 2027. This paper averages 2.33 over 3 reviews, above 51% of the others.
Reviewer_KkPjOpenReview ↗
Rating
3
Confidence
3
Informativeness
2
Presentation
3
Soundness
3
Read review
Paper summary

This paper presents DF-Fuzz, a dependency-guided context retrieval framework for LLM-assisted language processor fuzzing. The method constructs a dual-flow dependency graph that combines local control-flow constraints with cross-file data, type, macro, and inheritance dependencies. It then serializes a compact context for single-round mutation, using semantic pruning, API co-occurrence exemplars, and a compiler-feedback-driven self-healing loop to recover omitted definitions. The evaluation compares DF-Fuzz with conventional, symbolic, grammar-based, and LLM-guided fuzzers across several open-source language processors. The paper reports improvements in coverage, bug discovery, execution throughput, and API efficiency, together with an extended campaign that found previously unknown issues.

Critical strengths

The paper identifies a concrete failure mode in LLM-assisted compiler fuzzing: isolated function slices omit the declarations and relationships needed to produce valid inputs. Unifying control reachability with cross-file dependency resolution in an offline graph is a sensible and technically motivated response, and the single-round design has clear practical appeal for high-throughput fuzzing.

The pruning, exemplar retrieval, and diagnostic-driven repair components address different sources of failure and are connected through an understandable end-to-end workflow. The modular design and walkthrough make the system easy to follow and extend, and distinguish it from a monolithic prompt-engineering contribution.

Critical weaknesses

The causal claim is not isolated. The paper argues that missing cross-file context causes invalid LLM-generated inputs, but it only measures end-to-end coverage and bugs. It does not report input validity directly. That baseline's configuration is also unclear, so the benefit of dependency reasoning cannot be separated from simply adding more context or engineering differences.

Questions for authors

Can the authors report frontend pass rates with and without retrieval? Can they also compare against the LLM baseline under the same model and prompt budget?

4 comments

Thank you for the detailed rebuttal. The added pass-rate results address my main concern about input validity. It would also help to report the average prompt tokens for each strategy, ideally with a matched-budget comparison, to further separate dependency selection from context size. Overall, I will keep my score of weak accept.

AuthorsOpenReview ↗

Response to Reviewer KkPj

Dear Reviewer,

Thank you for the follow-up. This suggestion helps us separate which dependencies are selected from how much context is provided, which makes the evaluation clearer. We have compiled the prompt-size statistics below and completed the matched-budget experiments.

1. Average prompt tokens per strategy. All numbers are average input tokens per call, taken from our existing experiment logs (Clang++, 24h runs, averaged over 5 runs). The three DF-HDG variants use the same LSP lookup, few-shot exemplars, and polymorphic recall as the "Cross-file symbol lookup" baseline. The difference lies only in the retrieval module: DF-HDG selects dependencies anchored on the uncovered branch and its dominating predicates, whereas the baseline uses HLPFuzz's native cross-file symbol lookup.

Retrieval strategyAvg. input tokens / callPass rateEdge coverage
No retrieval (enclosing function only)1.2k18%85,426
Cross-file symbol lookup (HLPFuzz-style)4.3k39%94,810
DF-HDG, control edges only3.5k51%98,214
DF-HDG, data edges only3.8k48%97,953
Full DF-HDG3.7k59%100,153

Full DF-HDG uses fewer input tokens per call than the HLPFuzz-style baseline (3.7k vs. 4.3k) while achieving a higher pass rate and higher coverage.

2. Why the HLPFuzz-style baseline uses more input tokens. HLPFuzz builds context iteratively (Sections 5.1–5.3 of its paper). It starts from the enclosing function and, after each failed attempt, expands the context via divergence analysis and LSP symbol lookup, with a maximum of 5 iterations. Its own evaluation reports that a successful solution needs 2.64 iterations on average, and that only 5%–9.1% of solutions succeed in a single iteration (Section 7.3 and Table 6, under their GPT-4o-mini setting). One possible explanation is that, in the baseline, failed attempts trigger further rounds of context expansion, which increases the per-call input size. We note that the HLPFuzz paper does not state this link itself; it is our reading of its design together with our measured 39% pass rate.

3. Matched-budget comparison. An equal-token comparison is the most direct way to separate selection from size, so we ran it on Clang++ under the same protocol (24h, 5 runs averaged, same seeds and hardware). In both settings, the prompt is hard-truncated at the stated limit, and this is applied identically to both systems.

Budget (input tokens / call)StrategyPass rateEdge coverage
3.7k capCross-file symbol lookup (HLPFuzz-style)30%90,457
3.7k capFull DF-HDG51%97,186
4.3k capCross-file symbol lookup (HLPFuzz-style)36%92,235
4.3k capFull DF-HDG55%98,804

Under both budgets, Full DF-HDG achieves a higher pass rate (+21 and +19 percentage points) and higher edge coverage (+7.4% and +7.1%) than the baseline. This suggests that the improvement comes largely from which dependencies are selected rather than from how much context is provided.

Action. We will add the per-call token statistics, the matched-budget results, and the explanation of the baseline's token overhead to Section 5.4.2, and reference them in Section 5.1.

Thank you again for the constructive feedback, which has made the evaluation more rigorous.

Best regards,

The Authors

AuthorsOpenReview ↗

Follow-up on Our Response to Your Feedback

Dear Reviewer,

We have added the relevant experiments, including the prompt-size statistics and a matched-budget comparison, and we hope they address your concern. If you have any further questions, we would be happy to discuss and resolve them.

Best regards,

The Authors

Thank you for the follow-up. The token statistics and matched-budget results address my remaining concern. I will maintain my rating of weak accept.

Reviewer_yUUwOpenReview ↗
Rating
2
Confidence
3
Informativeness
2
Presentation
2
Soundness
2
Read review
Paper summary

This paper introduces DF-Fuzz, a context-retrieval framework for LLM-assisted language processor fuzzing. It combines local control constraints with cross-file information about types, macros, functions, and inheritance. The system retrieves this information ahead of generation, prunes the context, adds API examples, and uses compiler diagnostics to recover missing dependencies. The authors report five independent 24-hour runs on four main targets and extended testing on eight language processors. On Clang++, they report 5.7% higher coverage than HLPFuzz, 60.3% lower API cost, and 26.9% higher throughput. The extended campaign reports 45 previously unknown issues.

Critical strengths
  1. The system targets a practical bottleneck. Missing context and repeated retrieval can make LLM-assisted fuzzing expensive and unreliable. Combining dependency prefetching, context pruning, and local diagnostic feedback is a plausible way to address this problem.
  2. The reported improvements cover several targets and include ablations. Mean coverage is higher than HLPFuzz on all four main targets. Removing RAG brings Clang++ coverage close to the HLPFuzz result, which supports the possibility that retrieval is contributing useful information. Repeated runs, shared initial corpora, and cost measurements are also helpful parts of the evaluation.
Critical weaknesses
  1. The paper does not clearly explain how implementation-level dependencies lead to valid target-language inputs.
  2. The main comparisons and vulnerability claims need stronger supporting records.
Questions for authors
  1. Could you provide a reproducible example showing the original seed, retrieved context, full prompt, generated input, execution command, and target-branch coverage? How does this work for Fortran, JavaScript, and Lua?
  2. Was HLPFuzz’s original LSP retrieval fully enabled? Were the model, decoding settings, and budgets matched? What are the valid-generation and target-branch success rates?
  3. How are bugs aggregated and deduplicated across the five runs? Could you provide an anonymized list of the 45 findings, including confirmation status and representative reproductions?
  4. How many regeneration attempts does self-healing require, and are they included in the cost figures? Also, did the implementation use intersection counts or actual Jaccard similarity, and how does serialization handle dependency cycles?
2 comments
AuthorsOpenReview ↗

Response to Reviewer yUUw

We thank the reviewer for the detailed review and for recognizing the practical bottleneck we target. Below we address how implementation-level dependencies yield valid target-language inputs (W1), and the evidence behind our comparisons and bug claims (W2). All revisions are highlighted in the paper.

1. Implementation-level context and target-language inputs (W1 / Q1)

Answer. The LLM generates only test programs in the target language, never compiler source code. The retrieved implementation context (types, enums, macros, class hierarchies) is read-only. It tells the LLM which internal symbols and states appear in the path condition of the target branch, and the LLM expresses them as target-language constructs. Flang, Hermes, and Lua are all implemented in C/C++, so the same pipeline applies to all three without change. Only a short target-spec file and the code-fence tag differ.

Worked example (Lua 5.5.0, tag-method fallback in lvm.c).

  • Seed and target: the seed print(x + y) over integers takes the scalar fast path. The target is the code after the predicate !notm(luaT_gettmbyobj(...)) in callbinTM.
  • Retrieved context: the TMS enum, the luaT_gettmbyobj declaration, and the dominating predicate, serialized before the target function.
  • Generated input and evidence: a Lua program that defines __add in a metatable and applies + to two tables. An afl-showmap replay shows the code after the predicate newly covered, and the test case enters the queue.

Flang and Hermes. The same process applies to the other two targets. For example, for Flang, the target is a constant-folding branch in FoldOperation, and the generated input is a static-bound array expression. For Hermes, the target is the capture-group branch in regExpExec, and the generated input is a regular expression with capture groups.

Action: We have added a case study as Appendix A.2, with pointers from Section 3, and uploaded the corresponding case materials to the anonymous OSF repository.

2. Baseline fairness and generation validity (Q2)

Answer. We appreciate your constructive feedback. HLPFuzz was run with its original LSP-based symbol retrieval fully enabled. Apart from the retrieval strategy, all conditions were identical: the same GPT-5 backend, hardware, seed corpora, and 24-hour, N=5 protocol. The "w/o RAG" variant in Table 4 keeps HLPFuzz's native symbol lookup and accordingly matches HLPFuzz closely (94,810 vs. 94,782 on Clang++).

Valid-generation rate. In a controlled Clang++ experiment (24h, GPT-5) that varies only the retrieval module, the compile rate is 18% with the enclosing function only, 39% with HLPFuzz-style cross-file lookup, and 59% with full DF-HDG, while edge coverage rises from 85,426 to 94,810 to 100,153.

Action: We have stated the matched-configuration details in Section 5.1 and added the pass-rate results to Section 5.4.1.

3. Bug deduplication and supporting evidence (Q3 / W2)

Answer. We appreciate your constructive feedback. The 45 bugs come from one continuous one-month campaign. Crashes were deduplicated by ASan stack trace, manually triaged, reproduced on the latest trunk, and reported upstream. Developers have confirmed 34, and 11 are awaiting review. Upstream report pages are tied to the reporters' identities, so the anonymous OSF repository linked in our abstract contains representative bugs, with descriptions and detailed reproduction steps.

Action: We have added the deduplication, triage, and disclosure protocol, together with the confirmation breakdown, to Section 5.3.

4. Self-healing cost, scoring function, and cycles (Q4)

Self-healing and cost. Regeneration after a failed attempt is a retry step shared with HLPFuzz, capped at 5 attempts per target. Self-healing changes only how the context is repaired between attempts: diagnostics are parsed locally, missing symbols are recovered, and candidate nodes are re-ranked, with no LLM call. The reported cost is total API spend divided by campaign duration ($0.23/h for DF-Fuzz vs. $0.58/h for HLPFuzz) and includes all retry tokens of both systems. The saving therefore comes from better-repaired context and fewer wasted retries.

Scoring function. The implementation uses the full Jaccard similarity, ∣Sdep∩Sanchor∣/∣Sdep∪Sanchor∣|S_{dep}\cap S_{anchor}| / |S_{dep}\cup S_{anchor}|. Section 3.4 presented only the numerator, and we have corrected the formula.

Cycles. Graph expansion keeps a visited set, so traversal terminates on cyclic dependencies. During serialization, Kahn's algorithm emits every node it can order, and nodes that remain in cycles are emitted as a group ahead of their dependents. Every node therefore appears exactly once and no context is lost, and Top-KK pruning (K=6K=6) keeps such groups small.

Action: We have added the retry cap and cost accounting to Section 5.4.2, corrected the scoring formula in Section 3.4, and described cycle handling in Section 3.3.

AuthorsOpenReview ↗

Follow-up on Our Response to Your Review

Dear Reviewer yUUw,

Thank you again for your time and valuable feedback. As the discussion period ends soon, we would like to kindly check whether our response has addressed your concerns.

If any points remain unclear, we would be happy to provide further clarification or additional results before the deadline. We hope our response helps address your concerns, and we would be grateful if you could take it into account in your assessment.

We understand that you may have a busy schedule, and we would sincerely appreciate it if you could find the time to review and respond to our reply.

Best regards,

Authors of Submission #32472

Reviewer_UwxdOpenReview ↗
Rating
2
Confidence
3
Informativeness
2
Presentation
2
Soundness
2
Read review
Paper summary

This paper presents DF-Fuzz, a dependency-guided context retrieval framework for LLM-assisted language processor fuzzing. The proposed approach addresses the limitations of existing LLM-based fuzzers that rely on isolated function-level code slices, which often lack the semantic information required to generate compilable test programs. DF-Fuzz introduces a Dual-Flow Hybrid Dependency Graph (DF-HDG) to capture both intra-procedural control dependencies and cross-file semantic relationships. It further integrates dependency-guided retrieval, semantic pruning, API exemplar retrieval, and compiler-feedback-driven self-healing to provide relevant context for LLM-based test generation.

Critical strengths
  • Meaningful problem and practical significance.

The paper addresses an important limitation of LLM-assisted fuzzing, where insufficient semantic context prevents generated test programs from passing compiler validation.

  • Reasonable integration of existing techniques.

Although individual components such as dependency analysis, retrieval-augmented generation, and compiler feedback are not entirely new, their integration into a unified framework for language processor fuzzing is meaningful.

  • Comprehensive experimental evaluation.

The experiments cover multiple language processors and include coverage measurements, vulnerability discovery, ablation studies, inference efficiency, and LLM backend generalization.

Critical weaknesses
  • Insufficiently justified motivation and lack of representative baselines in the pilot study.

The motivation presented in Section 2 is not sufficiently supported by comparisons with representative state-of-the-art methods. In Section 2.1, the authors argue that conventional function slicing provides only the isolated body of the enclosing function to the LLM, resulting in missing semantic dependencies and poor compilation success rates. However, it is unclear which existing state-of-the-art approaches actually follow this restricted slicing strategy. Similarly, the evaluation in Section 2.2 considers an interactive retrieval baseline in which the LLM dynamically requests missing definitions through {DEFINITION LOOKUP} commands. The paper does not establish whether this implementation represents an existing state-of-the-art retrieval approach or merely a simplified baseline constructed for the pilot study. This weakens the motivation for the proposed framework and makes it difficult to assess the significance of its claimed contributions.

  • Limited methodological novelty and insufficient isolation of the proposed dependency-guided retrieval mechanism.

DF-Fuzz combines several established techniques, including control-flow slicing, cross-file dependency analysis, retrieval-augmented generation, semantic pruning, and compiler-feedback-based refinement. While their integration is practically meaningful, similar ideas have been explored in existing work on repository-level retrieval and dependency-aware fuzzing, such as RepoCoder (Shrivastava et al., 2023), CKGFuzzer (Xu et al., 2025), and dependency-based RAG for kernel fuzzing (Zhang et al., 2025). The paper does not sufficiently clarify the unique methodological contribution of DF-HDG beyond combining these existing techniques. Furthermore, the ablation study in Table 4 mainly evaluates the removal of individual components, while the "w/o RAG" configuration only demonstrates the benefit of providing additional context compared with isolated function slicing. It does not establish whether the proposed dependency graph is more effective than simpler retrieval alternatives. The authors could strengthen the contribution by comparing DF-HDG with conventional AST-based retrieval, cross-file symbol lookup, or other dependency-based retrieval strategies under comparable experimental settings.

  • Insufficient validation of the reported practical benefits and vulnerability discoveries.

Although the evaluation covers several language processors and reports promising coverage improvements, some of the most important empirical claims require stronger supporting evidence. First, Section 5.3 reports 45 previously unknown vulnerabilities, including 10 heap use-after-free and 8 heap buffer overflow issues, but provides limited details about their validation, deduplication, reproducibility, and confirmation by upstream developers. Moreover, the extended one-month vulnerability discovery campaign does not include corresponding baseline results, making it difficult to assess the improvement in actual vulnerability discovery capability. Second, the efficiency analysis in Table 5 focuses primarily on runtime throughput and API costs without clearly accounting for the offline construction and maintenance overhead of DF-HDG. Finally, the coverage improvements over HLPFuzz are relatively modest on some targets, such as Clang++ (+5.7%) and Flang (+3.8%), and additional statistical analysis would help establish their robustness. Providing more detailed vulnerability evidence, comprehensive cost accounting, and stronger statistical comparisons would make the reported practical advantages more convincing.

Questions for authors
  1. Which existing state-of-the-art methods are represented by the isolated function slicing and interactive retrieval baselines in Sections 2.1 and 2.2?

  2. What are the key methodological advantages of DF-HDG over existing dependency-based retrieval approaches?

  3. Could the authors provide stronger evidence supporting the validation and uniqueness of the 45 reported vulnerabilities?

2 comments
AuthorsOpenReview ↗

Response to Reviewer Uwxd

We thank the reviewer for recognizing the practical significance of our problem and the breadth of our evaluation. Below we address each concern with evidence from the paper and new experiments. All revisions are highlighted in the paper.

1. Which existing methods do Sections 2.1 and 2.2 represent?

Both pilot settings reproduce context-acquisition mechanisms of HLPFuzz (USENIX Security 2025), the state-of-the-art LLM-assisted fuzzer for language processors. Our Introduction did not name HLPFuzz; we make the link explicit here.

  • Section 2.1 . HLPFuzz starts each constraint from the enclosing function as its narrow context, and acknowledges that such a context can miss information needed to understand the constraint. This is the setting in which we observe 90% compilation failure.
  • Section 2.2 . HLPFuzz Sec. 5.3 lets the LLM request symbol definitions, served by clangd via LSP. KernelGPT (ASPLOS 2025) follows the same pattern with up to five extraction rounds.

HLPFuzz's own statistics agree with our pilot: it needs 2.64 iterations on average, and only 5% (Flang) to 9.1% (Hermes) of its solutions succeed in the first iteration. Narrow context alone rarely suffices, and the extra rounds are the cost Table 1 quantifies.

We have cited HLPFuzz and KernelGPT in Section 2 and clarify their relation to our pilot.

2. Is DF-HDG separated from simpler retrieval, and how does it differ from prior work?

New controlled experiment. The "w/o RAG" row in Table 4 removes only DF-HDG and keeps HLPFuzz's native LSP symbol lookup, so it already represents cross-file symbol lookup (hence 94,810 vs. HLPFuzz's 94,782 on Clang++). The label "Enclosing Slice" was misleading. To isolate DF-HDG further, we ran a new experiment on Clang++ (24h, GPT-5) varying only the retrieval module. Pass rate is the fraction of generated programs that compile.

Retrieval strategyPass rateEdge coverage
No retrieval (enclosing function only)18%85,426
Cross-file symbol lookup ("w/o RAG")39%94,810
DF-HDG, control edges only51%98,214
DF-HDG, data edges only48%97,953
Full DF-HDG59%100,153

Symbol lookup is already a strong baseline (+11.0%), yet full DF-HDG adds +5.6% coverage and +20 points pass rate, and each edge type alone beats it.

Difference from prior work. We agree that dependency analysis, retrieval, and compiler feedback are established ingredients. Our contribution lies in tailoring them to compiler fuzzing and resolving its two key problems: semantic scarcity in isolated slices, and the high latency and fragility of interactive retrieval.

  • Branch-anchored retrieval. RepoCoder retrieves around a completion cursor, CKGFuzzer and kernel RAG around APIs or system calls. DF-Fuzz anchors on an uncovered branch and its dominating predicates, so control edges decide which data dependencies are fetched.
  • Scope reconstruction. Namespace wrapping rebuilds enclosing scopes (e.g., clang::serialization), since a correct name in the wrong namespace still fails compilation.
  • Zero-LLM-cost self-healing. Local compiler diagnostics re-rank the graph and recover pruned definitions without extra dialogue turns (+3.9% coverage on Clang++).

We have relabeled the "w/o RAG" row, added the controlled comparison to Section 5.4.1, and expanded the comparison with prior work in Section 6.

3. Validation of practical benefits

Bugs. The 45 bugs come from one continuous one-month campaign. Crashes were deduplicated by ASan stack trace, triaged manually, reproduced on the latest trunk, and reported upstream. 34 are confirmed by developers and 11 await review. Since some reports are in public discussions, we do not list every bug here to protect anonymity. We have newly uploaded representative bugs with descriptions and detailed reproduction steps to the anonymous OSF repository linked in our abstract.

Baselines. The campaign was designed for discovery, not comparison. The controlled comparison is Table 2 (24h, N=5): DF-Fuzz finds 11 bugs vs. 8 for HLPFuzz and 2 for AFL++.

Construction cost. "Offline" means DF-HDG is built alongside AFL++, outside the fuzzing loop, not as a pre-built repository-level graph. When AFL++ stagnates, DF-Fuzz builds the per-target subgraph in a separate process, reusing cached syntax trees and the same infrastructure as HLPFuzz. No extra preparation is needed and fuzzing is not interrupted.

Statistics. The smallest margins are +5.7% (Clang++) and +3.8% (Flang), but (i) the gain has the same sign on all four targets, (ii) all 20 ablation cells lie below the full system, (iii) the cost (-60.3%) and throughput (+26.9%) margins are far larger, and (iv) DF-Fuzz finds more bugs (11 vs. 8).

We have replaced "offline" with a precise description in Sections 1 and 3, added the deduplication, triage, and disclosure protocol to Section 5.3, and added the statistical discussion to Appendix D.

AuthorsOpenReview ↗

Follow-up on Our Response to Your Review

Dear Reviewer Uwxd,

Thank you again for your time and valuable feedback. As the discussion period ends soon, we would like to kindly check whether our response has addressed your concerns.

If any points remain unclear, we would be happy to provide further clarification or additional results before the deadline. We hope our response helps address your concerns, and we would be grateful if you could take it into account in your assessment.

We understand that you may have a busy schedule, and we would sincerely appreciate it if you could find the time to review and respond to our reply.

Best regards,

Authors of Submission #32472

Comments on the paper

3 comments

Thank you for the detailed response and the additional experiments. The new pass-rate results and baseline details address my main concerns. I will keep my score.

AuthorsOpenReview ↗

Response to Reviewer KkPj

Dear Reviewer,

Thank you for your prompt reply and for confirming that the new pass-rate results and baseline details address your main concerns. We are grateful for your supportive assessment.

We have included all additional experiments and clarifications in Sections 5.1, 5.4.1, and 5.4.2. We hope this supports the paper's contribution, and we would be glad to address any remaining questions during the discussion period.

Thank you again for your constructive feedback, which has made the paper stronger.

Best regards,

The Authors

AuthorsOpenReview ↗

General Response to AC and the All Reviewers

We thank the reviewers and the AC for their time and feedback. The reviews note that the paper targets a practical problem in LLM-assisted compiler fuzzing, that single-round retrieval suits high-throughput fuzzing, and that the evaluation covers multiple targets with ablations. Revisions are marked in blue in the updated PDF. Individual responses are posted under each review.

1. Isolating DF-HDG and measuring input validity (KkPj, Uwxd, yUUw)

We added a controlled experiment on Clang++ (24h, GPT-5, same seeds and hardware) that varies only the retrieval module. Pass rate is the fraction of generated programs that pass the compiler frontend.

Retrieval strategyPass rateEdge coverage
No retrieval (enclosing function only)18%85,426
Cross-file symbol lookup (HLPFuzz-style)39%94,810
DF-HDG, control edges only51%98,214
DF-HDG, data edges only48%97,953
Full DF-HDG59%100,153

Pass rate and coverage increase together at each step. Cross-file lookup raises pass rate by 21 points over no retrieval. Full DF-HDG adds a further 20 points and 5.6% coverage over cross-file lookup, and each edge type alone exceeds cross-file lookup. Raising K from 6 to 8 and 10 adds definitions and lowers coverage (100.2k → 98.9k → 95.4k), so dependency selection affects results in addition to context size. (Sec. 5.4.1, 5.4.2)

2. Baseline configuration (KkPj, Uwxd, yUUw)

HLPFuzz was run with its original LSP symbol retrieval enabled. The GPT-5 backend, hardware, seed corpora, and 24h, N=5 protocol were identical. The former "w/o RAG" row keeps this lookup, is relabeled "w/o DF-HDG (HLPFuzz-style cross-file lookup)", and matches HLPFuzz (94,810 vs. 94,782). The two pilot settings in Sec. 2 follow HLPFuzz's context acquisition (enclosing function; LSP definition requests) and KernelGPT's multi-round extraction. HLPFuzz's reported average of 2.64 iterations is consistent with our measurements. (Sec. 2, 5.1)

3. Bug validation (Uwxd, yUUw)

Crashes were deduplicated by ASan stack trace, manually triaged, reproduced on the latest trunk, and reported upstream. 34 of the 45 bugs are confirmed by developers and 11 await review. Representative bugs with descriptions and reproduction steps are in the anonymous OSF repository. In the controlled 24h, N=5 comparison, DF-Fuzz finds 11 bugs, HLPFuzz 8, and AFL++ 2. (Sec. 5.3)

4. From implementation context to target-language inputs (yUUw)

Appendix A.2 adds a Lua 5.5.0 case study with the seed, retrieved context, generated input, and afl-showmap evidence. The LLM generates only target-language programs. The read-only context indicates which internal symbols appear in the path condition of the target branch. The same pipeline applies to Flang (constant-folding branch in FoldOperation) and Hermes (capture-group branch in regExpExec). Case materials are in the OSF repository.

5. Cost and statistics (Uwxd, yUUw)

  • Construction: when AFL++ stagnates, the per-target subgraph is built in a separate process, reusing cached syntax trees and HLPFuzz's Clangd infrastructure. There is no pre-built repository-level graph, and AFL++ is not interrupted. (Sec. 1, 3)
  • Retries and cost: retries are capped at 5 per target and shared with HLPFuzz. Cost is total API spend, including all retry tokens, divided by duration: $0.23/h vs. $0.58/h. (Sec. 5.4.2)
  • Statistics: Appendix D discusses the +5.7% (Clang++) and +3.8% (Flang) margins. The gain has the same sign on all four targets, all 20 ablation cells fall below the full system, and the cost (−60.3%) and throughput (+26.9%) margins are larger.

6. Relation to prior work (Uwxd)

Sec. 6 lists three differences from RepoCoder, CKGFuzzer, and kernel RAG: (i) retrieval is anchored on the uncovered branch and its dominating predicates, so control edges select data dependencies; (ii) namespace wrapping rebuilds enclosing C++ scopes; (iii) local diagnostics recover pruned definitions without an extra LLM call (+3.9% Clang++ coverage).

7. Technical clarifications (yUUw)

Sec. 3.4 gives the full Jaccard formula, |S_dep ∩ S_anchor| / |S_dep ∪ S_anchor|, which matches the implementation. Sec. 3.3 describes cycle handling: a visited set bounds expansion, and cyclic nodes are emitted as a group ahead of their dependents, so each node appears once.

Revised sections: Sec. 1, 3 (construction timing, cycles, Jaccard formula); Sec. 2 (pilot baselines); Sec. 5.1 (baseline configuration); Sec. 5.3 (bug protocol); Sec. 5.4.1, 5.4.2 (controlled comparison, cost); Sec. 6 (prior work); Appendix A.2, D.

Thank you again for your valuable comments. We welcome further discussion, and we hope the revised manuscript is now clearer and of higher quality.

From OpenReview, CC BY 4.0. Authors as signed there; shown unedited.

Related papers

Loading the map…

Discussion (0)

Sign in to comment.